Commit Graph

702 Commits

Author SHA1 Message Date
Kelson 9c4867a95a Update Changelog 2019-11-20 13:06:24 +01:00
Emmanuel Engelhart de7b7c34b5 Remove absolute internal URL support 2019-11-07 18:05:58 +01:00
luddens 20a2c78733 add get aria2 launch cmd method 2019-11-01 15:27:21 +01:00
luddens 9850be7267 add Curl error message 2019-11-01 15:27:21 +01:00
luddens 0dd996c6a3 add try catch around aria2 first commands 2019-11-01 15:27:21 +01:00
luddens c9a15c9961 Add a parameter to getBookmarks fct to get valid bookmarks only
The default value of this parameter is false, in this case all the bookmarks
are returned, otherwise only those who are related to books of the library.
2019-10-31 14:05:21 +02:00
luddens 9975e0b369 add setPort() method 2019-10-28 15:56:49 +01:00
Aditya-Sood 2af9ba4eab Readd original getNextSuggestion() 2019-10-01 13:30:35 +05:30
Aditya-Sood c007373b46 Re-add comment 2019-10-01 13:30:34 +05:30
Aditya-Sood e1acf9acff Code & local repository cleanup 2019-10-01 13:30:34 +05:30
Aditya-Sood daaadf3e1c Comment out previous definitions 2019-10-01 13:30:34 +05:30
Aditya-Sood 74bd482335 Preliminary work 2019-10-01 13:30:34 +05:30
Matthieu Gautier 67170709bb Convert path get from windows environment to utf8.
Fix kiwix/kiwix-desktop#203
2019-09-25 18:07:42 +02:00
Matthieu Gautier 0db06d98a8 Add missing implementation of android's getArticleCount and getMediaCount.
Fix #281
2019-09-24 11:47:05 +02:00
Matthieu Gautier 598dd3c175 [API Break] Fix pathTools (and a bit stringTools).
Api changes :
 - removeLastPathElement do not takes extra arguments
   `removePreSeparator` and `removePostSeparator`.
   This is not needed as path do not need special tailing separator.
 - Only one function `split`. Arguments can be implicitly convert to
   string. No need for overloading functions to explicitly cast them.
 - `split` function takes another argument `trimEmpty`. If true, empty
   element are removed.

Path manipulation now almost pass trough a vector<string> to store each
path's part.

Most of the complex works is now made in the normalizeParts function.
2019-09-19 18:16:06 +02:00
Matthieu Gautier 2f4636e2df Fix stringTools join function. 2019-09-17 16:22:28 +02:00
Matthieu Gautier 9b4419f3fc [ABI Break] Correctly detect the executable path in appimage.
There are two executable path :
- The user one (the appimage path)
- The real one (in the appimage archive)

When we search of `library.xml` we need the user one.
But when we search of `aria2c` or `kiwix-serve` we need the real one.

Fix kiwix/kiwix-desktop#256
2019-09-17 11:23:16 +02:00
Matthieu Gautier 6ee174b546 Add a method to get the value of a specific tag.
Fix #258
2019-09-17 10:37:53 +02:00
Matthieu Gautier 2a6772b76d [API Change] Convert tags to the new convention.
Use the new convention describe here : https://wiki.openzim.org/wiki/Tags
2019-09-17 10:30:24 +02:00
Matthieu Gautier 660d5d7fb7 [API Change] Rename getMatatag to getMetadata. 2019-09-16 10:36:04 +02:00
Matthieu Gautier 157c1c939c Add a string tool to join a list of strings together. 2019-09-16 09:42:10 +02:00
Matthieu Gautier bd91e89785 Add missing method to get the zim metadata.
According to https://wiki.openzim.org/wiki/Metadata
2019-09-12 15:33:07 +02:00
Matthieu Gautier 1245d4e467 Use a macro to get the content of the metadata. 2019-09-12 15:26:53 +02:00
Matthieu Gautier 420be55bfa Reorder methods to get metadata.
Use the same order than https://wiki.openzim.org/wiki/Metadata
2019-09-12 15:24:17 +02:00
Matthieu Gautier e42e061d45 Add a way to specify a library to use with kiwix-serve.
If kiwix-desktop use a `library.xml` in the same directory than the
executable, we need to use it instead of the default one.

Instead of detect again the `library.xml` to use, let `kiwix-desktop` set
the library to use.

This also fix a issue when `/` is not a valid path separator in windows.
2019-09-11 14:04:21 +02:00
Matthieu Gautier 3294508d87 Correctly cast double to int.
Ms cl compiler complains about the implicit conversion.
2019-09-10 14:10:40 +02:00
Matthieu Gautier a32363e6a2 Correctly detect the executable path if we use a AppImage.
AppImage works by decompressing the "program" in a temporary directory.
So the executable path is not the path of the AppImage file.

By using the environment variables set by appimage we can find the correct
"path" of the executable.

Fix kiwix/kiwix-desktop#46
2019-09-09 18:27:53 +02:00
Matthieu Gautier 87dc145dc7 Correctly set searcher information even if resultStart equals resultEnd. 2019-09-09 14:43:51 +02:00
Matthieu Gautier a13244dc0e Rename `hasResult` to `hasResults` 2019-09-09 14:43:51 +02:00
Matthieu Gautier 78dbd66522 [HTML Rendering] Do not render page navigation buttons if only one page. 2019-09-09 14:43:51 +02:00
Matthieu Gautier fdc291b7c2 [HTML Rendering] Do not do division by zero.
We must correctly handle the case if resultStart is equal to resultEnd.
2019-09-09 14:43:51 +02:00
Kelson d0833bdcd4 Fix fulltext search link in kiwix-serve suggestions 2019-09-04 17:07:05 +02:00
Matthieu Gautier 12a93c3e29 Fix use of strtok on windows.
On windows, strtok_r is called strto_s.
2019-08-19 17:35:22 +02:00
Emmanuel Engelhart cea201b394 Specs says ZIM favicon should be at '-/favicon', should be tried first 2019-08-19 16:27:55 +02:00
mhutti1 8672aede97 Add JNIKiwixString constructors 2019-08-13 17:55:48 +02:00
Matthieu Gautier 6ab52e2b9e Add a JNI method to check if an url exists and is a redirection. 2019-08-13 11:19:39 +02:00
Matthieu Gautier d90a27af11 Fix nameMapper initialization. 2019-08-12 16:16:26 +02:00
Matthieu Gautier a65e192f0f [JNI] Allow android to know that an article is a redirect.
Android need to handle the redirection by doing a redirection in the web
view, not by providing the content of the targeted article.

This is already what we do in kiwix-serve or ios.

The API should be far better by returning a Entry but for now,
we just change the given url if the article is a redirection.
2019-08-12 13:03:20 +02:00
Matthieu Gautier 513cc9c90f [JNI] Fix log typo. 2019-08-12 12:43:52 +02:00
Matthieu Gautier 231ae095f6 Correctly set that book's path is valid when updating it from a reader. 2019-08-12 12:43:52 +02:00
Matthieu Gautier 9a0c6da018 Library construction doesn't take argument 2019-08-12 12:43:52 +02:00
Matthieu Gautier 52299ef767 Fix computeAbsolutePath.
Correctly delete the duplicated string.
Use strtok_r to be thread safe.
2019-08-12 12:43:52 +02:00
Matthieu Gautier c4963268ba Fix regexTools.
The buildMatcher must not take a rvalue as it will keep a reference
to it.
2019-08-12 12:05:51 +02:00
Matthieu Gautier 73a29ccb24 [JNI] Fix implementation of setDataDirectory.
Now that setDataDirectory is a static method, we need to take an
jclass instead of a jobject.
2019-08-11 15:09:26 +02:00
Matthieu Gautier 7060afae66 [SERVER] Catch any error and return a 500 response instead of crashing.
The server will be running some code on the behalf of the calling code.
We really don't what to crash the library (and the binary) because
of a wrong request.
2019-08-11 11:30:43 +02:00
Matthieu Gautier 4d3df4e889 Add JNI wrapper around the library and the server. 2019-08-11 11:30:43 +02:00
Matthieu Gautier cd050ddcc8 Use camelCase. 2019-08-11 11:30:43 +02:00
Matthieu Gautier 635d4438e5 Make the server take a pointer to the library instead of a reference. 2019-08-11 11:30:43 +02:00
Matthieu Gautier ce09375c6c Reduce complexity of handle_search. 2019-08-11 11:30:43 +02:00
Matthieu Gautier fae0918f49 Reduce complexity of handle_catalog. 2019-08-11 11:30:43 +02:00
Matthieu Gautier d90f8b0f05 Add a name_mapper mapping the HumanReadable name to the id. 2019-08-11 11:30:43 +02:00
Matthieu Gautier abb5db0193 Clean up includes in server.cpp 2019-08-11 11:30:43 +02:00
Matthieu Gautier e452f5cf36 Ensure that the root is correctly formatted. 2019-08-11 11:30:43 +02:00
Matthieu Gautier c890e1c87e Add support of a binding to a specific ip address. 2019-08-11 11:30:43 +02:00
Matthieu Gautier e5ef3780db Rename humanReadableBookId to bookName.
`humanReadableBookId` is a bit long and doesn't represent what it is
(this is not a id).
`bookName` is far better.
2019-08-11 11:30:43 +02:00
Matthieu Gautier 2aeed65205 Better handling of invalid request.
Do no crash if we can get a book or a reader for the requested content.
2019-08-11 11:30:43 +02:00
Matthieu Gautier c1faf55ae8 Introduce the server functionality in the kiwix-lib.
This code is mainly copied from kiwix-tools.

But :
- Move all the response thing in a new class Response.
- This Response class is responsible to handle all the MHD_response
  configuration. This way the server handle a global object and do
  no call to MHD_response*
- Server uses a lot more the templating system with mustache.
  There are still few regex operations (because we need to
  change a content already existing).
- By default, the server serves the content using the id as name.
- Server creates a new Searcher per request. This way, we don't have
  to protect the search for multi-thread and we can do several search
  in the same time.
- search results are not cached, this will allow future improvement in the
  search algorithm.
- the home page is not cached.
- Few more verbose information (number of request served, time spend to
  respond to a request).

TOOD:
 - Readd interface selection.
 - Do Android wrapper.
 - Remove KiwixServer (who use a external process).
 -
2019-08-11 11:30:43 +02:00
Matthieu Gautier 64dfea2547 Move the search html renderer in a different class than the searcher.
This is two different functionnalies, we don't need to polute the searcher
api with things to render the html.
2019-08-11 10:19:48 +02:00
Matthieu Gautier cca5980b27 Remove limitation of the search len in the Searcher.
The limitation should be made elsewhere (the code using the searcher).
2019-08-11 10:19:48 +02:00
Matthieu Gautier ce8fff0b42 Make the library create the reader. 2019-08-11 10:19:48 +02:00
Matthieu Gautier e56335109c Make appendToFirstOccurence take argument by reference. 2019-08-11 10:19:48 +02:00
Matthieu Gautier 656bf183b7 Make getHumanReadableFromPath method const. 2019-08-11 10:19:48 +02:00
Matthieu Gautier cbe8e20118 Fix include in otherTools.h 2019-08-11 10:19:48 +02:00
Matthieu Gautier e013d38cc6 Add startWith function in stringTools.h 2019-08-11 10:19:48 +02:00
Matthieu Gautier 6234457920 Fix include in stringTools.h 2019-08-11 10:19:48 +02:00
Matthieu Gautier 72223d69fe Fix include in pathTools.h 2019-08-10 11:02:23 +02:00
Matthieu Gautier ddeb862395 Add getMimeTypeForFile in pathTool. 2019-08-10 10:59:13 +02:00
Matthieu Gautier 61c28f0e3d Make the regexTool thread safe. 2019-08-10 10:59:13 +02:00
Matthieu Gautier 654a8e304c Set the verbosity of the underlying zim::Search. 2019-07-30 17:03:44 +02:00
Sean Mac Gillicuddy 5adf7891cc #239 ReLinker should be used to load library 2019-07-29 18:31:57 +02:00
luddens c17abdae5e port can be set with the constructor
add a method to return the port
2019-06-27 16:50:22 +02:00
Matthieu Gautier 31c9375a3a Better API to filter books in a library.
Instead of having a single method `listBooksIds` that tries to be
exhaustive about all the filter and sort option, split the method in
two separated methods `filter` and `sort`.

The `filter` method takes a `Filter` object that represent on what we are
filtering. This object has to be construct before calling `filter`.

```cpp
Filter filter;
filter.query("Astring");
filter.acceptTags({"nopic"});
// return all book in eng and with "Astring" in the tile or description".
library.filter(filter);
//equivalent to
library.listBooksIds(ALL, UNSORTED, "Astring", "", "", "", {"nopic"});
// or better
library.filter(Filter().query("Astring").acceptTags({"nopic"}));
```

The method `listBooksIds` has been marked as deprecated.

Add a small test on the library.
2019-06-26 16:41:01 +02:00
Matthieu Gautier 3d689e790b Force the exit of the forked process.
The `exit` function will call the functions registered.
Qt may (and does) register function and may (and does) hangup waiting
for some mutex to be free.

Here we are in a forked process and we want to process to exit without
doing any cleanup (because we are a clone of a process that will continue
and we don't want to mess it).
2019-06-25 16:52:46 +02:00
luddens 87f5b56b72 add isRunning method 2019-06-24 15:05:20 +02:00
luddens df76db4f47 Fix crash searchbar's suggestions
The suggestion's list was filled without checking if the current suggestion
was wrong which caused crash in this case.
2019-06-24 11:48:14 +02:00
Matthieu Gautier 6fe6f88b10 Fix waitpid option.
`WEXITED` is an option for the `waitid` function not `waitpid`.

We don't need to pass a option to `waitpid`.
2019-06-24 11:34:03 +02:00
Matthieu Gautier 4e746916a7 Remove unnecessary include.
And unistd.h is not available on windows.
2019-06-12 14:27:15 +02:00
luddens 5dc96d7145 kiwix-serve integration 2019-06-11 10:35:16 +02:00
Matthieu Gautier 48347825a9 suggestions method return a unique_ptr instead of a raw pointer. 2019-05-28 15:08:16 +02:00
luddens 519dd110f5 add tags in opds stream 2019-05-14 10:51:22 +02:00
luddens 491b6d655b add remove fct with aria2 2019-05-13 10:30:19 +02:00
luddens ec8f1ffe9c Add pause and unpause functions for aria2
This functions enable to stop and resume download with aria2.

The Downloader's constructor now checks the paused downloads with the
function "tellWaiting()" to get them at the start of kiwix-desktop.
2019-05-13 10:30:19 +02:00
luddens 12ffad55f7 update download's status when we get it 2019-05-13 10:30:19 +02:00
Matthieu Gautier 0b2c9fa7ce Revert Do not use this_thread::sleep.
We already have a custom sleep method.
2019-05-03 15:38:47 +02:00
Matthieu Gautier fe682f855a Do not use this_thread::sleep.
Mingw doesn't implement it. So, we should not use it.
I suppose that it was working before because mingw package for debian trusty
simply no provides a "thread" header.
We may face to include the native "thread" header.
2019-04-17 15:44:14 +00:00
cyrillemoureaux e1980d190f Detect and reject infinite redirect loops.
This will prevent kiwix/kiwix-tools#207.
2019-04-15 16:55:48 +02:00
Vitaly Zaitsev 28f144796d
Removed Rpath from compiled binaries.
Rpath is not needed to run installed binaries. Most of GNU/Linux
distributions strictly forbid it.

Signed-off-by: Vitaly Zaitsev <vitaly@easycoding.org>
2019-04-15 16:09:53 +02:00
Matthieu Gautier e7e88617d5 Remove unused include unistd.h
We don't use it anymore and it is not present on Windows.
2019-04-15 10:50:26 +02:00
Matthieu Gautier f8522fb26e Remove getNetworkInterfaces and getBestPublicIp function.
They are not used at all and the windows version need some extra libs
that complexify the code.

Remove them for now. If it happens that they are needed, we will readd
them.
2019-04-04 17:00:45 +02:00
Matthieu Gautier 938e2a81c1 Rewrite makeTmpDirectory to not use Uuid methods on windows.
`UuidCreate`, `UuidToString` and `RpcStringFree` need special library
on windows. Lets not use them.
2019-04-04 16:11:36 +02:00
Matthieu Gautier d40982f760 Remove error message when trying to opening a wrong zim file.
The library should not print a message.

Fix kiwix/kiwix-tools#235
2019-04-02 15:28:46 +02:00
Matthieu Gautier c6254d9504 Allow the library to be filtered by tags.
This add an argument to `listBooksIds` to filter by tags.
So, this is an API break.
2019-03-07 17:08:39 +01:00
Matthieu Gautier 93af3aa2d1 Fix the language mapping.
The previous mapping was taken from an unknown (:/) source.

The new mapping is generated with a script taking
https://www.loc.gov/standards/iso639-2/php/code_list.php as source.

The source list is sanitized to keep only language for which we
(http://library.kiwix.org/) have content.
2019-03-05 17:53:37 +01:00
Matthieu Gautier af9734c87f Correctly initialize variable. 2019-02-19 14:05:37 +01:00
Matthieu Gautier 35d812a5f7 Ensure the book's `path` is absolute.
We must use absolute path whenever possible.
Relative path has sense only related to the "interaction" with the user
(current directory, library location, ...).
2019-02-07 15:22:33 +01:00
Matthieu Gautier 432f9c30a3 Remove unused variable url. 2019-02-07 15:20:18 +01:00
Matthieu Gautier af7689e3e8 [API break] Move all the tools in the tools directory instead of common.
The `common` name is from the time where kiwix was only one repository
for all the project (android, desktop, server...).

Now we have split the repositories and kiwix-lib is the "common" repo,
the "common" directory is somehow nonsense.
2019-01-23 15:31:38 +01:00
Matthieu Gautier b996a2877c Correctly initialize retVal. 2019-01-23 11:51:30 +01:00
Matthieu Gautier be498c3b16 Make the string Tools functions available in android. 2019-01-09 18:29:20 +01:00
Matthieu Gautier 92c9a47a0d Move the templating system to mustache instead of ctpp2.
Mustache templating system is a bit simpler than ctpp2 and ctpp2 is no
more maintained (see #189).
We are moving to the kainjow's Mustache project
(https://github.com/kainjow/Mustache).

It simplify a lot our system has it is header only and we don't have to
precompile the template.

Fix #21
2019-01-09 18:28:48 +01:00