Commit Graph

747 Commits

Author SHA1 Message Date
Matthieu Gautier 533541cf45 Write the articleCount and mediaCount metadata in the OPDS stream.
This is not standard OPDS. But clients need this information.
2020-04-07 12:22:44 +02:00
renaud gaudin 7155c788e2 attach taskbar to `<head>` instead of `<head>\n`
Fixed a regression introduced in block-external-links feature.

For cleaner source, the taskbar (and the block-external JS file) were both
attached to `<head>\n`.
Unfortunately, this isn't safe enough as some ZIM files might have all kinds of HTML
syntax. Sotoki for instance have no CR after head, rendering the attachment impossible.

Note: realizing this method is somehow fragile as any HTML content with extra attribute
on the `<head>` tag or without a `<head>` tag would break the taskbar and the block external feature.
2020-04-03 16:53:43 +02:00
renaud gaudin 4709a42f4f disable external links blocking on 500 handler 2020-03-30 14:42:37 +00:00
renaud gaudin d04d9bf7f3 Unblock external link in catch page in JS code
Instead of disabling the blocking for the handler, the JS code detects it is
displaying the handler and allows external links to go through
2020-03-27 12:26:22 +00:00
renaud gaudin 412f0d9c61 moved blockExternalLink outside of taskbar
- `setBlockExternalLinks()` on server
- zero-dependency JS code
- JS script added in `inject_externallinks_blocker()`
- changed URL to `/catch/external?source=<source>`
2020-03-27 11:25:39 +00:00
renaud gaudin 0ad8bf45fc Add external links blocking in serve
In many use cases, it is not wanted to have user accidentaly click on external links
and leave the served ZIM content.
This could be because the result is unpredictible (reader not implementing this properly)
or because the serve user knows there's no backup internet connexion or because there is
an induced cost behind external links that doesn't affect served content.

using a new flag (`blockExternalLinks`) on `Response`/`setTaskBar`, a piece of JS code
is injected into the taskbar code.
This code adds a JS handler on all link click events and verifies the destination.
If the destination appears to be an external link (1), the link target is changed to
a specific URL:

```
/external?source=<original_uri>
```

(1) external is a link that's not on the same origin and starts with either `http:` `https:` or `//`.

Server implements a new handler on `/external` that displays a new page (`captured_external.html`)
which returns a generic message explaining the situation and offering to click on the link
again should the user really want to.
This is done by specifically asking `set_taskbar` to not block external requests on that page.

This approach allows integrators using a reverse proxy to handle that endpoint differently (rebrand it)

1. `Server` now has an `m_blockExternalLinks` defaulting to `false`
1. `Server.setTaskbar` is extended to support an additional bool to set the variable.
1. `Response` now has an `m_blockExternalLinks`
1. `Response` constr expects an additional bool for `blockExternalLinks`.
1. `Response.set_taskbar` is extended to support an additional bool to set the variable.
1. JNI/Java Wrapper reflects the extensions.
1. New resource file `templates/block_external.js` (included in head_part). Should it be in skin?
1. New resource file `templates/captured_external.html` for `handle_captured_external()`
1. Added a comment on `head_part.html` to help with JS insertion at the right place
1. `introduce_taskbar()` conditionnaly inserts the JS inside the taskbar
2020-03-26 12:06:36 +00:00
Matthieu Gautier 064d5f3fa6 Make the search argument constant. 2020-03-06 12:08:05 +01:00
Matthieu Gautier 46626a3f98 Add the method get bookByPath in library. 2020-03-06 12:08:05 +01:00
Matthieu Gautier 76c293e403 [JAVA] Add a method to get the size of an article.
Fix #327
2020-03-04 16:40:03 +01:00
Matthieu Gautier 2e60a088ab [JAVA] Use a long to store the offset of a article in the zim file.
Fixes kiwix/kiwix-android#1769
2020-02-19 14:27:51 +01:00
Matthieu Gautier ea29557a33 [Java] Add a wrapper on method to update book from another book or reader. 2020-02-11 17:44:14 +01:00
Matthieu Gautier b53f531f2b Fix typo getTagStr in the wrapper 2020-02-10 17:25:31 +01:00
Matthieu Gautier 1632e9c55b Fix launching command with path containing spaces.
We must correctly quote path with space on windows.
This is needed as we can't launch command using a array of string on
windows but by giving only one string using space as separator.

Fix kiwix/kiwix-desktop#268
2020-02-06 11:46:33 +01:00
Kelson 6a975994cc Include stdexcept to fix GCC v10 compilation 2020-02-01 13:52:27 +01:00
Matthieu Gautier ce6e956434 [OPDS] Add the url argument to filter by size and name.
Fix kiwix/kiwix-tools#231
2020-01-30 19:02:33 +01:00
Matthieu Gautier f560a1f815 Be able to filter the books by name. 2020-01-30 19:02:33 +01:00
Matthieu Gautier 34257cfc1f Trust the library.xml information by default.
Do not try to read the zim file and update the book when parsing a
library.xml.
Needed by kiwix/kiwix-tools#319
2020-01-30 18:22:07 +01:00
Matthieu Gautier a756e7f8f3 Add flavour attribute to book.
Fix #259
Fix kiwix/kiwix-tools#316
2020-01-30 17:48:56 +01:00
Matthieu Gautier bc257d2d6d Add method to get value of tag from a book.
Only the `getTagStr` method is available on android because we need a
proper exception handling on wrapping side.

Fix #298
2020-01-30 17:48:56 +01:00
Matthieu Gautier 7275f9b8e3 Move function to convert and use tags inside otherTools. 2020-01-30 17:48:56 +01:00
Matthieu Gautier 2881face70 Add missing setting of attribute. 2020-01-30 15:42:54 +01:00
Matthieu Gautier 77ba09c310 Reorder setting and dumper of book attribute.
No real change. Reordering setting and dumping of attribute in the same
order (mostly) they are declared in book.h make it easier to detect missing
attribute.
2020-01-30 15:42:54 +01:00
Matthieu Gautier 49aa0fbb9f Use a macro to write the filters. 2020-01-30 15:42:54 +01:00
Matthieu Gautier 7846b45bef Fix opds filtering by tag. 2020-01-30 15:42:54 +01:00
Matthieu Gautier 7e26f3502d Fix opds dumper/parser.
Add missing attributes.
2020-01-30 15:42:54 +01:00
luddens 4b6c26bd0b add parameter option to start a download with aria2
Downloader::startDownload has a new parameter option which is a vector of
pair that represents the options that can be set for adding a uri with aria2
with the function Aria2::addUri.

Aria2::addUri uses this parameter to set the struct of parameters for the
aria2 command
2020-01-29 15:38:09 +01:00
luddens 6bcecc2677 remove useless error handling functions
Some of the changes from https://github.com/kiwix/kiwix-lib/pull/292
that doesn't work properly are removed
2020-01-28 16:58:55 +01:00
Matthieu Gautier 82afb804e1 Add a small java test on the kiwix-lib.
Will the compilation should be made by meson.
It seems it is not possible to specify a existing jar
to link with. Use a custom script for now.
2020-01-28 12:08:18 +01:00
Matthieu Gautier 0951546356 Create the jar library when creating the java wrapper. 2020-01-28 12:08:18 +01:00
Matthieu Gautier f09c739c1f Be able to create a wrapper for java.
Android is a specific wrapper.
Java is another one.
2020-01-28 12:08:18 +01:00
Matthieu Gautier df9ddd5451 Use correct mutex on android and java. 2020-01-28 12:08:18 +01:00
Matthieu Gautier fe513951d3 Use a macro to print error log.
This allow use to compile the JNI wrapper not for
android.
2020-01-28 12:08:18 +01:00
Matthieu Gautier 7f0d509a88 Add the filter functionality. 2020-01-28 12:08:18 +01:00
Matthieu Gautier 54f671b2f1 Add some methods to get information from the library.
Else, the library is useless.
2020-01-28 12:08:18 +01:00
Matthieu Gautier c2c89c6c86 Rename the JNIKiwixLibrary class to Library.
This mainly use the "new memory system". No need to call dispose function.
Rename the class to Library to conform with the naming semantics
(JNIKiwix* use old memory system)
2020-01-28 12:08:18 +01:00
Matthieu Gautier 75652d0e9f WIP Add a wrapper around the kiwix::manager class.
The JNIKiwixManager is used to manage (insertion of book in) the library.

It is created, as needed, using an existing Library as input.
It is then used to add books, parse library.xml or opds content.
Then it can be destruct (and must be) with the `dispose` method.

```java
library = JNILibrary(...);
manager = JNIManager(library);
manager.parseOpds(opdscontent);
manager.dispose();
// library contains the books declared in the opds content.
// Use the library methods to get the books' info.
```
2020-01-28 12:08:18 +01:00
Matthieu Gautier 6535dc2e38 Add a wrapper for the Book class. 2020-01-28 12:08:18 +01:00
Matthieu Gautier 6b2f768c8f Use template function c2jni to convert c++ type to jni. 2020-01-28 12:08:18 +01:00
luddens ff21a095cb update book even if the members aren't empty
remove the conditions to always update the book
2020-01-27 17:50:44 +01:00
Matthieu Gautier 91db055d86 Remove function to read file using a native path.
All path must be utf8. This is already the case in all our project.
(If this not the case, this is a bug)

So we don't need to have a version with a native and utf8 path.
2020-01-13 16:59:58 +01:00
Matthieu Gautier 5540149e2b Correctly open the library path on windows.
We need to convert the path to wstring on Windows to handle directory
with accented characters.

Fix kiwix-desktop#269
2020-01-13 16:54:09 +01:00
Matthieu Gautier 071e9e3fec Correctly filter the catalog when we don't what to filter.
Set the different filter's fields only when we are requested to filter
them. Else, we ends to requests that some fields are empty.

If the request has no argument, we raise an exception (catched) and so
we don't set the corresponding field in the filter.

Fix #303
2020-01-07 17:29:43 +01:00
Matthieu Gautier 4a01303438 Correctly set the id of the opds stream.
Set the id of the stream *after* the uuid is generated.
2020-01-07 17:29:29 +01:00
Kelson b7c5e5f339 Remove trailing spaces 2019-12-08 11:52:16 +01:00
Kelson 52e165cf78 Reintroduce kiwix-serve taskbar 2019-11-26 11:54:00 +01:00
Kelson 9c4867a95a Update Changelog 2019-11-20 13:06:24 +01:00
Emmanuel Engelhart de7b7c34b5 Remove absolute internal URL support 2019-11-07 18:05:58 +01:00
luddens 20a2c78733 add get aria2 launch cmd method 2019-11-01 15:27:21 +01:00
luddens 9850be7267 add Curl error message 2019-11-01 15:27:21 +01:00
luddens 0dd996c6a3 add try catch around aria2 first commands 2019-11-01 15:27:21 +01:00
luddens c9a15c9961 Add a parameter to getBookmarks fct to get valid bookmarks only
The default value of this parameter is false, in this case all the bookmarks
are returned, otherwise only those who are related to books of the library.
2019-10-31 14:05:21 +02:00
luddens 9975e0b369 add setPort() method 2019-10-28 15:56:49 +01:00
Aditya-Sood 2af9ba4eab Readd original getNextSuggestion() 2019-10-01 13:30:35 +05:30
Aditya-Sood c007373b46 Re-add comment 2019-10-01 13:30:34 +05:30
Aditya-Sood e1acf9acff Code & local repository cleanup 2019-10-01 13:30:34 +05:30
Aditya-Sood daaadf3e1c Comment out previous definitions 2019-10-01 13:30:34 +05:30
Aditya-Sood 74bd482335 Preliminary work 2019-10-01 13:30:34 +05:30
Matthieu Gautier 67170709bb Convert path get from windows environment to utf8.
Fix kiwix/kiwix-desktop#203
2019-09-25 18:07:42 +02:00
Matthieu Gautier 0db06d98a8 Add missing implementation of android's getArticleCount and getMediaCount.
Fix #281
2019-09-24 11:47:05 +02:00
Matthieu Gautier 598dd3c175 [API Break] Fix pathTools (and a bit stringTools).
Api changes :
 - removeLastPathElement do not takes extra arguments
   `removePreSeparator` and `removePostSeparator`.
   This is not needed as path do not need special tailing separator.
 - Only one function `split`. Arguments can be implicitly convert to
   string. No need for overloading functions to explicitly cast them.
 - `split` function takes another argument `trimEmpty`. If true, empty
   element are removed.

Path manipulation now almost pass trough a vector<string> to store each
path's part.

Most of the complex works is now made in the normalizeParts function.
2019-09-19 18:16:06 +02:00
Matthieu Gautier 2f4636e2df Fix stringTools join function. 2019-09-17 16:22:28 +02:00
Matthieu Gautier 9b4419f3fc [ABI Break] Correctly detect the executable path in appimage.
There are two executable path :
- The user one (the appimage path)
- The real one (in the appimage archive)

When we search of `library.xml` we need the user one.
But when we search of `aria2c` or `kiwix-serve` we need the real one.

Fix kiwix/kiwix-desktop#256
2019-09-17 11:23:16 +02:00
Matthieu Gautier 6ee174b546 Add a method to get the value of a specific tag.
Fix #258
2019-09-17 10:37:53 +02:00
Matthieu Gautier 2a6772b76d [API Change] Convert tags to the new convention.
Use the new convention describe here : https://wiki.openzim.org/wiki/Tags
2019-09-17 10:30:24 +02:00
Matthieu Gautier 660d5d7fb7 [API Change] Rename getMatatag to getMetadata. 2019-09-16 10:36:04 +02:00
Matthieu Gautier 157c1c939c Add a string tool to join a list of strings together. 2019-09-16 09:42:10 +02:00
Matthieu Gautier bd91e89785 Add missing method to get the zim metadata.
According to https://wiki.openzim.org/wiki/Metadata
2019-09-12 15:33:07 +02:00
Matthieu Gautier 1245d4e467 Use a macro to get the content of the metadata. 2019-09-12 15:26:53 +02:00
Matthieu Gautier 420be55bfa Reorder methods to get metadata.
Use the same order than https://wiki.openzim.org/wiki/Metadata
2019-09-12 15:24:17 +02:00
Matthieu Gautier e42e061d45 Add a way to specify a library to use with kiwix-serve.
If kiwix-desktop use a `library.xml` in the same directory than the
executable, we need to use it instead of the default one.

Instead of detect again the `library.xml` to use, let `kiwix-desktop` set
the library to use.

This also fix a issue when `/` is not a valid path separator in windows.
2019-09-11 14:04:21 +02:00
Matthieu Gautier 3294508d87 Correctly cast double to int.
Ms cl compiler complains about the implicit conversion.
2019-09-10 14:10:40 +02:00
Matthieu Gautier a32363e6a2 Correctly detect the executable path if we use a AppImage.
AppImage works by decompressing the "program" in a temporary directory.
So the executable path is not the path of the AppImage file.

By using the environment variables set by appimage we can find the correct
"path" of the executable.

Fix kiwix/kiwix-desktop#46
2019-09-09 18:27:53 +02:00
Matthieu Gautier 87dc145dc7 Correctly set searcher information even if resultStart equals resultEnd. 2019-09-09 14:43:51 +02:00
Matthieu Gautier a13244dc0e Rename `hasResult` to `hasResults` 2019-09-09 14:43:51 +02:00
Matthieu Gautier 78dbd66522 [HTML Rendering] Do not render page navigation buttons if only one page. 2019-09-09 14:43:51 +02:00
Matthieu Gautier fdc291b7c2 [HTML Rendering] Do not do division by zero.
We must correctly handle the case if resultStart is equal to resultEnd.
2019-09-09 14:43:51 +02:00
Kelson d0833bdcd4 Fix fulltext search link in kiwix-serve suggestions 2019-09-04 17:07:05 +02:00
Matthieu Gautier 12a93c3e29 Fix use of strtok on windows.
On windows, strtok_r is called strto_s.
2019-08-19 17:35:22 +02:00
Emmanuel Engelhart cea201b394 Specs says ZIM favicon should be at '-/favicon', should be tried first 2019-08-19 16:27:55 +02:00
mhutti1 8672aede97 Add JNIKiwixString constructors 2019-08-13 17:55:48 +02:00
Matthieu Gautier 6ab52e2b9e Add a JNI method to check if an url exists and is a redirection. 2019-08-13 11:19:39 +02:00
Matthieu Gautier d90a27af11 Fix nameMapper initialization. 2019-08-12 16:16:26 +02:00
Matthieu Gautier a65e192f0f [JNI] Allow android to know that an article is a redirect.
Android need to handle the redirection by doing a redirection in the web
view, not by providing the content of the targeted article.

This is already what we do in kiwix-serve or ios.

The API should be far better by returning a Entry but for now,
we just change the given url if the article is a redirection.
2019-08-12 13:03:20 +02:00
Matthieu Gautier 513cc9c90f [JNI] Fix log typo. 2019-08-12 12:43:52 +02:00
Matthieu Gautier 231ae095f6 Correctly set that book's path is valid when updating it from a reader. 2019-08-12 12:43:52 +02:00
Matthieu Gautier 9a0c6da018 Library construction doesn't take argument 2019-08-12 12:43:52 +02:00
Matthieu Gautier 52299ef767 Fix computeAbsolutePath.
Correctly delete the duplicated string.
Use strtok_r to be thread safe.
2019-08-12 12:43:52 +02:00
Matthieu Gautier c4963268ba Fix regexTools.
The buildMatcher must not take a rvalue as it will keep a reference
to it.
2019-08-12 12:05:51 +02:00
Matthieu Gautier 73a29ccb24 [JNI] Fix implementation of setDataDirectory.
Now that setDataDirectory is a static method, we need to take an
jclass instead of a jobject.
2019-08-11 15:09:26 +02:00
Matthieu Gautier 7060afae66 [SERVER] Catch any error and return a 500 response instead of crashing.
The server will be running some code on the behalf of the calling code.
We really don't what to crash the library (and the binary) because
of a wrong request.
2019-08-11 11:30:43 +02:00
Matthieu Gautier 4d3df4e889 Add JNI wrapper around the library and the server. 2019-08-11 11:30:43 +02:00
Matthieu Gautier cd050ddcc8 Use camelCase. 2019-08-11 11:30:43 +02:00
Matthieu Gautier 635d4438e5 Make the server take a pointer to the library instead of a reference. 2019-08-11 11:30:43 +02:00
Matthieu Gautier ce09375c6c Reduce complexity of handle_search. 2019-08-11 11:30:43 +02:00
Matthieu Gautier fae0918f49 Reduce complexity of handle_catalog. 2019-08-11 11:30:43 +02:00
Matthieu Gautier d90f8b0f05 Add a name_mapper mapping the HumanReadable name to the id. 2019-08-11 11:30:43 +02:00
Matthieu Gautier abb5db0193 Clean up includes in server.cpp 2019-08-11 11:30:43 +02:00
Matthieu Gautier e452f5cf36 Ensure that the root is correctly formatted. 2019-08-11 11:30:43 +02:00
Matthieu Gautier c890e1c87e Add support of a binding to a specific ip address. 2019-08-11 11:30:43 +02:00
Matthieu Gautier e5ef3780db Rename humanReadableBookId to bookName.
`humanReadableBookId` is a bit long and doesn't represent what it is
(this is not a id).
`bookName` is far better.
2019-08-11 11:30:43 +02:00