Commit Graph

120 Commits

Author SHA1 Message Date
Matthieu Gautier e42e061d45 Add a way to specify a library to use with kiwix-serve.
If kiwix-desktop use a `library.xml` in the same directory than the
executable, we need to use it instead of the default one.

Instead of detect again the `library.xml` to use, let `kiwix-desktop` set
the library to use.

This also fix a issue when `/` is not a valid path separator in windows.
2019-09-11 14:04:21 +02:00
Matthieu Gautier cd050ddcc8 Use camelCase. 2019-08-11 11:30:43 +02:00
Matthieu Gautier 635d4438e5 Make the server take a pointer to the library instead of a reference. 2019-08-11 11:30:43 +02:00
Matthieu Gautier d90f8b0f05 Add a name_mapper mapping the HumanReadable name to the id. 2019-08-11 11:30:43 +02:00
Matthieu Gautier e452f5cf36 Ensure that the root is correctly formatted. 2019-08-11 11:30:43 +02:00
Matthieu Gautier c890e1c87e Add support of a binding to a specific ip address. 2019-08-11 11:30:43 +02:00
Matthieu Gautier c1faf55ae8 Introduce the server functionality in the kiwix-lib.
This code is mainly copied from kiwix-tools.

But :
- Move all the response thing in a new class Response.
- This Response class is responsible to handle all the MHD_response
  configuration. This way the server handle a global object and do
  no call to MHD_response*
- Server uses a lot more the templating system with mustache.
  There are still few regex operations (because we need to
  change a content already existing).
- By default, the server serves the content using the id as name.
- Server creates a new Searcher per request. This way, we don't have
  to protect the search for multi-thread and we can do several search
  in the same time.
- search results are not cached, this will allow future improvement in the
  search algorithm.
- the home page is not cached.
- Few more verbose information (number of request served, time spend to
  respond to a request).

TOOD:
 - Readd interface selection.
 - Do Android wrapper.
 - Remove KiwixServer (who use a external process).
 -
2019-08-11 11:30:43 +02:00
Matthieu Gautier 64dfea2547 Move the search html renderer in a different class than the searcher.
This is two different functionnalies, we don't need to polute the searcher
api with things to render the html.
2019-08-11 10:19:48 +02:00
Matthieu Gautier 68a768f58a Introduce the new class NameMapper.
This class is used to map an id (uuid) to a name (potentially human
readable).

This will be use by the server to or renderer to use a different "name"
than the id.

The default NameMapper provided use the id as a name.
2019-08-11 10:19:48 +02:00
Matthieu Gautier ce8fff0b42 Make the library create the reader. 2019-08-11 10:19:48 +02:00
Matthieu Gautier e56335109c Make appendToFirstOccurence take argument by reference. 2019-08-11 10:19:48 +02:00
Matthieu Gautier 656bf183b7 Make getHumanReadableFromPath method const. 2019-08-11 10:19:48 +02:00
Matthieu Gautier cbe8e20118 Fix include in otherTools.h 2019-08-11 10:19:48 +02:00
Matthieu Gautier e013d38cc6 Add startWith function in stringTools.h 2019-08-11 10:19:48 +02:00
Matthieu Gautier 6234457920 Fix include in stringTools.h 2019-08-11 10:19:48 +02:00
Matthieu Gautier 72223d69fe Fix include in pathTools.h 2019-08-10 11:02:23 +02:00
Matthieu Gautier ddeb862395 Add getMimeTypeForFile in pathTool. 2019-08-10 10:59:13 +02:00
Matthieu Gautier 61c28f0e3d Make the regexTool thread safe. 2019-08-10 10:59:13 +02:00
Matthieu Gautier c8e719101e Add a new tool `Lock` to lock a mutex.
And automaticlly unlock it when the `Lock` got out of scope.
2019-08-10 10:59:13 +02:00
luddens c17abdae5e port can be set with the constructor
add a method to return the port
2019-06-27 16:50:22 +02:00
Matthieu Gautier 31c9375a3a Better API to filter books in a library.
Instead of having a single method `listBooksIds` that tries to be
exhaustive about all the filter and sort option, split the method in
two separated methods `filter` and `sort`.

The `filter` method takes a `Filter` object that represent on what we are
filtering. This object has to be construct before calling `filter`.

```cpp
Filter filter;
filter.query("Astring");
filter.acceptTags({"nopic"});
// return all book in eng and with "Astring" in the tile or description".
library.filter(filter);
//equivalent to
library.listBooksIds(ALL, UNSORTED, "Astring", "", "", "", {"nopic"});
// or better
library.filter(Filter().query("Astring").acceptTags({"nopic"}));
```

The method `listBooksIds` has been marked as deprecated.

Add a small test on the library.
2019-06-26 16:41:01 +02:00
Matthieu Gautier 21592af8b2 Remove invalid method declaration in kiwix::Manager.
Those methods were totally obsolete. They were even not implemented.
2019-06-26 16:41:01 +02:00
luddens 87f5b56b72 add isRunning method 2019-06-24 15:05:20 +02:00
Matthieu Gautier 4e746916a7 Remove unnecessary include.
And unistd.h is not available on windows.
2019-06-12 14:27:15 +02:00
luddens 5dc96d7145 kiwix-serve integration 2019-06-11 10:35:16 +02:00
luddens 491b6d655b add remove fct with aria2 2019-05-13 10:30:19 +02:00
luddens ec8f1ffe9c Add pause and unpause functions for aria2
This functions enable to stop and resume download with aria2.

The Downloader's constructor now checks the paused downloads with the
function "tellWaiting()" to get them at the start of kiwix-desktop.
2019-05-13 10:30:19 +02:00
Matthieu Gautier f8522fb26e Remove getNetworkInterfaces and getBestPublicIp function.
They are not used at all and the windows version need some extra libs
that complexify the code.

Remove them for now. If it happens that they are needed, we will readd
them.
2019-04-04 17:00:45 +02:00
Matthieu Gautier c6254d9504 Allow the library to be filtered by tags.
This add an argument to `listBooksIds` to filter by tags.
So, this is an API break.
2019-03-07 17:08:39 +01:00
Matthieu Gautier af7689e3e8 [API break] Move all the tools in the tools directory instead of common.
The `common` name is from the time where kiwix was only one repository
for all the project (android, desktop, server...).

Now we have split the repositories and kiwix-lib is the "common" repo,
the "common" directory is somehow nonsense.
2019-01-23 15:31:38 +01:00
Matthieu Gautier be498c3b16 Make the string Tools functions available in android. 2019-01-09 18:29:20 +01:00
Matthieu Gautier 92c9a47a0d Move the templating system to mustache instead of ctpp2.
Mustache templating system is a bit simpler than ctpp2 and ctpp2 is no
more maintained (see #189).
We are moving to the kainjow's Mustache project
(https://github.com/kainjow/Mustache).

It simplify a lot our system has it is header only and we don't have to
precompile the template.

Fix #21
2019-01-09 18:28:48 +01:00
Matthieu Gautier 5159d985c6 Remove support for external index.
This feature is considered obsolete for a while.
In fact, it was already not supported since June 2018 as we were compiling
xapian without the chert backend support.

Assume that we don't support it and remove it from the code.
See kiwix/kiwix-tools#245

This is a API break. library.xml files will still work but the indexPath
and indexType will be dropped silently from the file.
2019-01-07 16:47:08 +01:00
Matthieu Gautier 12498e2cfe Add bookmarks support.
The library now contains (simple) methods to handle bookmarks.
The bookmark are stored in a separate xml file.

Bookmark are mainly a couple (`zimId`, `articleUrl`).
However, in the xml we store a bit more data :
- The article's title (for display)
- The book's title, lang and date (for potential update of zim files)
2018-12-02 15:47:29 +01:00
Matthieu Gautier b5ce60a627 Move the dump of the library into library.xml in a specific class.
The same way the dump into a opds feed is in a specific class.
2018-11-28 12:09:28 +01:00
Matthieu Gautier 062124a2a0 Add Book::getFaviconUrl 2018-11-15 17:47:41 +01:00
Matthieu Gautier 9ab44e6a5f Get information about the total number of book of a search.
When we do a search and paging the result, we need to display to the
user the total number of book, not only the `itemsPerPage`.

So, we need to parse correctly the xml to keep information of the total
number of book.
2018-11-02 17:04:55 +01:00
Matthieu Gautier 5f4c04e79e Fix use of getAsI when parsing download rpc.
The value is store as a string in in the xml, so we cannot use getAsI.
We have to get the string and parse it to an int.
We cannot use strtoull because android stdc++ lib doesn't have it.

We have to implement our how parseFromString function using a
istringstream.
2018-11-02 17:03:03 +01:00
Matthieu Gautier a1876e3b27 Add a method converta2toa3 to convert language code alpha2 to alpha3.
Qt give use alpha2 language code but we use alpha3.
2018-10-26 14:18:06 +02:00
Matthieu Gautier c6206edfb4 Do not always download the favicon of a book. Download as needed.
When parsing a opds feed, the favicon is a url, not a dataurl.
If we download the favicon all the times, it may take a lot of time to
parse the feed.

We store the url and download the favicon only when needed (when displayed)
2018-10-24 11:56:05 +02:00
Matthieu Gautier c20ae18bff An opds feed can also be the openSearch result.
We must be able to set the correct entry in the feed for a searchResult.
2018-10-24 11:51:38 +02:00
Matthieu Gautier b1508c0b98 Better listBooksIds supported mode.
Only have REMOTE or LOCAL is a bit restrictive. By using flags a user
can specify for complex request.
2018-10-24 11:50:11 +02:00
Matthieu Gautier 34021994cd Fix for Android
- No std::to_string. We have to implement it with a ostringstream
- No pthread_cancel. So we use pthread_kill to send a signal to the thread.
2018-10-24 10:48:53 +02:00
Matthieu Gautier 910ce5f10d Fix for Windows
- "winsock2.h" needs to be included before "windows.h". But if a
  compilation unit include "windows.h" and after "networkTools.h", we
  fails and it is complicated to handle. The include must not be in the
  header but in the cpp
- windows define some ERROR macro. It is a pitty but we cannot use `ERROR`
  in our enum.
- If build statically using mingw we need to define `CURL_STATICLIB`
2018-10-24 10:47:12 +02:00
Matthieu Gautier ad69fdd8c0 Move the download method from the downloader to networkTools.
The download method is a simple method to download content.
It use curl to download the content instead of aria.
2018-10-24 10:47:12 +02:00
Matthieu Gautier fe6d5fa93e Store the downloadId in the book (and in the library). 2018-10-24 10:47:12 +02:00
Matthieu Gautier 43ff8565d1 Add a Download class to encapsulate a aria2 download. 2018-10-24 10:47:12 +02:00
Matthieu Gautier f718c4c472 Add a LibraryManipulator.
Library client (kiwix-desktop) need to know when a book is added to
library by the manager. By using a LibraryManipulator, we can do
dependency injection.
2018-10-24 10:47:12 +02:00
Matthieu Gautier 8176a6eded Be more resilient to potential aria2 error. 2018-10-24 10:47:12 +02:00
Matthieu Gautier bb1f777078 Store the aria2 session and recover from it. 2018-10-24 10:47:12 +02:00