Commit Graph

561 Commits

Author SHA1 Message Date
Matthieu Gautier c6254d9504 Allow the library to be filtered by tags.
This add an argument to `listBooksIds` to filter by tags.
So, this is an API break.
2019-03-07 17:08:39 +01:00
Matthieu Gautier 93af3aa2d1 Fix the language mapping.
The previous mapping was taken from an unknown (:/) source.

The new mapping is generated with a script taking
https://www.loc.gov/standards/iso639-2/php/code_list.php as source.

The source list is sanitized to keep only language for which we
(http://library.kiwix.org/) have content.
2019-03-05 17:53:37 +01:00
Matthieu Gautier af9734c87f Correctly initialize variable. 2019-02-19 14:05:37 +01:00
Matthieu Gautier 35d812a5f7 Ensure the book's `path` is absolute.
We must use absolute path whenever possible.
Relative path has sense only related to the "interaction" with the user
(current directory, library location, ...).
2019-02-07 15:22:33 +01:00
Matthieu Gautier 432f9c30a3 Remove unused variable url. 2019-02-07 15:20:18 +01:00
Matthieu Gautier af7689e3e8 [API break] Move all the tools in the tools directory instead of common.
The `common` name is from the time where kiwix was only one repository
for all the project (android, desktop, server...).

Now we have split the repositories and kiwix-lib is the "common" repo,
the "common" directory is somehow nonsense.
2019-01-23 15:31:38 +01:00
Matthieu Gautier b996a2877c Correctly initialize retVal. 2019-01-23 11:51:30 +01:00
Matthieu Gautier be498c3b16 Make the string Tools functions available in android. 2019-01-09 18:29:20 +01:00
Matthieu Gautier 92c9a47a0d Move the templating system to mustache instead of ctpp2.
Mustache templating system is a bit simpler than ctpp2 and ctpp2 is no
more maintained (see #189).
We are moving to the kainjow's Mustache project
(https://github.com/kainjow/Mustache).

It simplify a lot our system has it is header only and we don't have to
precompile the template.

Fix #21
2019-01-09 18:28:48 +01:00
Matthieu Gautier 5159d985c6 Remove support for external index.
This feature is considered obsolete for a while.
In fact, it was already not supported since June 2018 as we were compiling
xapian without the chert backend support.

Assume that we don't support it and remove it from the code.
See kiwix/kiwix-tools#245

This is a API break. library.xml files will still work but the indexPath
and indexType will be dropped silently from the file.
2019-01-07 16:47:08 +01:00
Matthieu Gautier 29046bfc05 Create the datadirectory to not fail to write the aria2 session file.
Fix kiwix/kiwix-desktop#69
2018-12-14 15:24:13 +01:00
Matthieu Gautier e6d7ba06fb Convert the standard opds date to our format (YYYY-MM-DD) 2018-12-11 17:02:02 +01:00
Matthieu Gautier 0f812c6584 The update entry of the book should be the date of the book, not the feed. 2018-12-11 17:01:33 +01:00
Matthieu Gautier 716c87dd20 Remove duplicate language attribute in the libxml dumper.
Silly copy/paste.
2018-12-11 17:00:56 +01:00
Matthieu Gautier 12498e2cfe Add bookmarks support.
The library now contains (simple) methods to handle bookmarks.
The bookmark are stored in a separate xml file.

Bookmark are mainly a couple (`zimId`, `articleUrl`).
However, in the xml we store a bit more data :
- The article's title (for display)
- The book's title, lang and date (for potential update of zim files)
2018-12-02 15:47:29 +01:00
Matthieu Gautier b5ce60a627 Move the dump of the library into library.xml in a specific class.
The same way the dump into a opds feed is in a specific class.
2018-11-28 12:09:28 +01:00
Matthieu Gautier 94a053e821 Wait a bit more between attempts to connect to aria2c rpc. 2018-11-12 16:14:21 +01:00
Matthieu Gautier 4b9692bbd5 Correctly run aria2c when packaged with kiwix-desktop in appimage.
By default, we are searching in the PATH env var.
However, with an appImage, the executable directory is not in the PATH,
so we have to use an absolute path if we can.

If we cannot find the aria2c executable in the executable directory let's
try to use the system one.
2018-11-12 14:35:17 +01:00
Matthieu Gautier 4b31842c4a Correctly convert filesize from Kbyte to byte.
`reader.getFileSize()` return the size of the zim in Kbyte in a
`unsigned int` (32 bits). This is ok as it would overflow if the size
of the size is greater than 4294967295 kbytes (so ~4Tbytes).

However, we need to convert the return size into a unsigned 64 bits integer
else, when converting to bytes, we will overflow at 4Gbytes.
Even in `m_size` is a uint64_t.
2018-11-12 12:16:05 +01:00
Matthieu Gautier cf1cfe774e Correctly check for ArticleCount and MediaCount before writing them. 2018-11-12 10:58:10 +01:00
Matthieu Gautier 8c4b9fbe95 Add missing `en`->`eng` mapping to `codeisomapping`.
The most common used language was missing :/

Fix kiwix/kiwix-desktop#51
2018-11-12 10:36:45 +01:00
Matthieu Gautier ab63cb2fb8 Sort `codeisomapping` alphabetically.
This is only code formating, no real change.
2018-11-12 10:34:19 +01:00
Matthieu Gautier 3958b2a06f Make the internal map codeisomapping static.
Symbole should not be visible outside of the compilation unit.
2018-11-12 10:33:35 +01:00
Matthieu Gautier d4ecda40ff Use the correct separator when computing relativePath. 2018-11-03 12:18:54 +01:00
Matthieu Gautier 9ab44e6a5f Get information about the total number of book of a search.
When we do a search and paging the result, we need to display to the
user the total number of book, not only the `itemsPerPage`.

So, we need to parse correctly the xml to keep information of the total
number of book.
2018-11-02 17:04:55 +01:00
Matthieu Gautier 5f4c04e79e Fix use of getAsI when parsing download rpc.
The value is store as a string in in the xml, so we cannot use getAsI.
We have to get the string and parse it to an int.
We cannot use strtoull because android stdc++ lib doesn't have it.

We have to implement our how parseFromString function using a
istringstream.
2018-11-02 17:03:03 +01:00
Matthieu Gautier bb07ff5610 Do not add NULL at end of commandLine on Windows. 2018-10-31 13:56:42 +01:00
Matthieu Gautier 1787e30440 Better launch of the aria2 process.
By setting the ApplicationName to NULL, CreateProcessW will
search for the application in the path.
2018-10-31 13:56:42 +01:00
Matthieu Gautier ccb3d8639d Use correct name for aria2c on windows. 2018-10-30 18:43:30 +01:00
Matthieu Gautier 29e554b47b Include pthread 2018-10-29 14:30:35 +01:00
Matthieu Gautier 68dc4d40b5 Include windows.h before synchapi.h 2018-10-29 12:20:00 +01:00
Matthieu Gautier 2682fa8f9c Remove unecessary variable or output. 2018-10-26 14:19:10 +02:00
Matthieu Gautier a22f962722 Correctly store the size of the book in the library.
`reader.getFileSize()` return ko.
2018-10-26 14:18:40 +02:00
Matthieu Gautier a1876e3b27 Add a method converta2toa3 to convert language code alpha2 to alpha3.
Qt give use alpha2 language code but we use alpha3.
2018-10-26 14:18:06 +02:00
Matthieu Gautier ad654ead08 Do not force the download port to be 80.
We may want to use url with port != 80.
2018-10-24 11:56:38 +02:00
Matthieu Gautier c6206edfb4 Do not always download the favicon of a book. Download as needed.
When parsing a opds feed, the favicon is a url, not a dataurl.
If we download the favicon all the times, it may take a lot of time to
parse the feed.

We store the url and download the favicon only when needed (when displayed)
2018-10-24 11:56:05 +02:00
Matthieu Gautier c20ae18bff An opds feed can also be the openSearch result.
We must be able to set the correct entry in the feed for a searchResult.
2018-10-24 11:51:38 +02:00
Matthieu Gautier b1508c0b98 Better listBooksIds supported mode.
Only have REMOTE or LOCAL is a bit restrictive. By using flags a user
can specify for complex request.
2018-10-24 11:50:11 +02:00
Matthieu Gautier 34021994cd Fix for Android
- No std::to_string. We have to implement it with a ostringstream
- No pthread_cancel. So we use pthread_kill to send a signal to the thread.
2018-10-24 10:48:53 +02:00
Matthieu Gautier 910ce5f10d Fix for Windows
- "winsock2.h" needs to be included before "windows.h". But if a
  compilation unit include "windows.h" and after "networkTools.h", we
  fails and it is complicated to handle. The include must not be in the
  header but in the cpp
- windows define some ERROR macro. It is a pitty but we cannot use `ERROR`
  in our enum.
- If build statically using mingw we need to define `CURL_STATICLIB`
2018-10-24 10:47:12 +02:00
Matthieu Gautier c66c7e9c20 Store the size of the book in OPDSFeed. 2018-10-24 10:47:12 +02:00
Matthieu Gautier ad69fdd8c0 Move the download method from the downloader to networkTools.
The download method is a simple method to download content.
It use curl to download the content instead of aria.
2018-10-24 10:47:12 +02:00
Matthieu Gautier a73ef23f6e Keep the book size in byte in memory (instead of in kb)
We keep the size in kb in library.xml for compatibility.
2018-10-24 10:47:12 +02:00
Matthieu Gautier fe6d5fa93e Store the downloadId in the book (and in the library). 2018-10-24 10:47:12 +02:00
Matthieu Gautier 43ff8565d1 Add a Download class to encapsulate a aria2 download. 2018-10-24 10:47:12 +02:00
Matthieu Gautier f718c4c472 Add a LibraryManipulator.
Library client (kiwix-desktop) need to know when a book is added to
library by the manager. By using a LibraryManipulator, we can do
dependency injection.
2018-10-24 10:47:12 +02:00
Matthieu Gautier 8176a6eded Be more resilient to potential aria2 error. 2018-10-24 10:47:12 +02:00
Matthieu Gautier bb1f777078 Store the aria2 session and recover from it. 2018-10-24 10:47:12 +02:00
Matthieu Gautier 829c34dd69 Store in the book instance if the given path is valid.
The path may exist and not be valid if the zim file is not truncated
(ie, interrupted download)
2018-10-24 10:47:12 +02:00
Matthieu Gautier 9c0f9696ed Better beautifyInteger and beautifyFileSize. 2018-10-24 10:47:12 +02:00