Commit Graph

493 Commits

Author SHA1 Message Date
Matthieu Gautier c9eac04050 Make the Library`s book vector private.
Move a lot of methods from Manager to Library. Because books is private
and thoses methods are better in Library.
2018-09-06 18:30:37 +02:00
Matthieu Gautier 741c67786a Add `update` method to `Book`. 2018-09-06 18:30:37 +02:00
Matthieu Gautier db9000f706 Make the downloader use the aria2c wrapper instead of the aria2 library. 2018-09-06 18:30:34 +02:00
Matthieu Gautier 0a93cb0872 Add aria2 downloader using subprocess aria2c. 2018-09-06 18:29:49 +02:00
Matthieu Gautier f4846c1ac8 Add a tool's function to get the data directory.
The data directory is where kiwix application should store data.
2018-08-29 15:28:52 +02:00
Matthieu Gautier 9b516ac35d Add a small wrapper around pugixml do handle xmprpc. 2018-08-29 15:28:52 +02:00
Matthieu Gautier 79b780b75b Move the function to convert from xml_node to string in otherTools.
This can be usefull elsewhere than in opds_dumper
2018-08-29 15:28:52 +02:00
Matthieu Gautier f3dd83907d Add backend to launch subprocess.
The windows backend is not tested.
2018-08-29 15:28:52 +02:00
Matthieu Gautier 7c634738dd Update jni build script to java jdk 8+.
With jdk8, `javac` has an option `-h` to generate the header files of
native classes.
So there is no need to run `javah` several times.

As there is now only one command to run (`javac`), there is no need for
the wrapper script `gen_kiwix.sh`.

Fix #167
2018-08-20 12:17:29 +02:00
Matthieu Gautier fd9b6569af Include unistd.h only on unix platform. 2018-08-03 19:22:39 +02:00
Matthieu Gautier 3cf58b5f5b Make libaria2 an optional dependency.
We don't compile libaria2 on Windows.
2018-08-03 19:22:39 +02:00
Matthieu Gautier dbcc9140b9 [Android] Better error message when failing to read in zim file.
Let's print the exception's message to allow us to better understand
what went wrong.
2018-07-31 11:49:44 +02:00
mhutti1 d61580f599 Set JNI values to NULL on error 2018-07-27 10:11:24 +02:00
mhutti1 3227b29c90 Follow redirects in favicons 2018-07-27 10:11:24 +02:00
mhutti1 709baae934 Convert all JNI cerrs to Android log messages 2018-06-29 16:35:11 +02:00
mhutti1 ea8cd9f1a9 Correctly pass 0 through JNI if ZIM file is corrupted 2018-06-29 15:30:13 +02:00
Matthieu Gautier 5025ee4963 Fix icudt version.
We have move to icu version 58 a while ago.
2018-06-11 14:38:32 +02:00
Matthieu Gautier 9aaf82a36d Explicitly use icu namespace.
Fix #145
2018-06-11 14:36:34 +02:00
mhutti1 fa99cce68d Decode reserved characters in URLs 2018-05-20 18:41:39 +01:00
Matthieu Gautier 622d2fc23d Remove unused static resources.
Stop words are not use anymore since a long time now that indexing has
been moved to libzim. No need to embedded them in kiwix-lib.
2018-05-15 11:30:30 +02:00
Matthieu Gautier c0b1c6013e Fix parsing of url
Fix kiwix/kiwix-tools#193
2018-05-14 17:41:05 +02:00
Matthieu Gautier 1dd828e79c Fix pathExists and check for correct path for xapian index.
The correct path for xapian database should be "X/fulltext/xapian",
not "Z//fulltextIndex/xapian".

So lets check for the right path and fallback to the wrong one (but
used in old zims).

The double '/' in the path is a bug of zimwriterfs and is specific
to the xapian database.
We must handle this correctly in `hasFulltextIndex` and not (buggly) in
`pathExists`.
(Hopefully, it seems that pathExists were used only by hasFulltextIndex)
2018-04-19 18:04:07 +02:00
Matthieu Gautier 135028c16a Introduce better API to manipulate entries in a zim file.
The previous API suffer different problems:
- It was difficult to handle articles redirecting to other article.
- It was not possible to get few information (title) without getting
  the whole content.

The new API introduce the new class `Entry` that act as a proxy to an
article in the zim file.

Methods of `Reader` now return an `Entry` and the user has to call
`Entry`'s methods to get useful information.
No redirection is made explicitly.
If an entry is not found, an exception is raised instead of returning
an invalid `Entry`.

The common pattern to get the content of an entry become :

```
std::string content;
try {
  auto entry = reader.getEntryFromPath(path);
  entry = entry.getFinalEntry();
  content = entry.getContent();
} catch (NoEntry& e) {
  ...
}
```

Older methods are keep (with the same behavior) but are marked as
deprecated.
2018-04-19 18:04:07 +02:00
Matthieu Gautier 47ce044e3e Add method to `Manager` to populate the library from a opds stream.
The library's books are created in the metadata in the opds.
As the opds stream is by definition a distant "library", there is no
zim to read to complete missing information.

This can lead to incomplete `library.xml`.
2018-04-19 17:53:08 +02:00
Matthieu Gautier 1f091da3f4 Add a downloader tools to download files.
The downloader is using libaria2.

For now, only one download can be run a the time.
A download will start only if (and as soon as) no download is running.
2018-04-19 17:53:08 +02:00
Matthieu Gautier d4fefd1a57 Add a function to create a temporary directory. 2018-04-19 17:53:05 +02:00
Matthieu Gautier 9f86b59d1d Add a function to get the content of a file. 2018-04-19 17:53:02 +02:00
Matthieu Gautier 2164faba44 Add a potential search description link in the opds stream. 2018-04-19 17:08:01 +02:00
Matthieu Gautier ad92af928b Be able to filter a library.
This generate a new library only with the corresponding books.
2018-04-19 17:08:01 +02:00
Matthieu Gautier ee51c470b4 Allow the manager to dump the opds feed of the whole library. 2018-04-19 17:08:01 +02:00
Matthieu Gautier ec097ab267 Try to compile kiwix-lib without warning. 2018-04-18 16:57:27 +02:00
Matthieu Gautier d686de7ec3 Set the RPATH of kiwix-lib.
As we cannot change (DY)LD_LIBRARY_PATH on macos, we have to use rpath.
2018-04-17 16:27:31 +02:00
Matthieu Gautier 313f6731b0 Correctly pre-increment loopCounter.
If we check the later the `loopCounter` with 42, we must pre-increment the
content. Else, in case of infinite loop, the `loopCounter` will be 43.

Related to kiwix/kiwix-tools#168
2018-03-25 17:21:40 +02:00
Matthieu Gautier ee6831d665 Check `internal->_search` before using it.
If a search has been set and a user try to get the nextResult or
restart the search, `internal->_search` will be NULL.
2018-03-12 17:45:18 +01:00
Matthieu Gautier f8a2e4c503 Only add a reader to the searcher if the reader as fulltext index.
`libzim` will not search in zim file without embedded fulltext index.
If we don't want to mess up with result index, we must not store "wrong"
reader.

Fix #111
2018-03-12 17:34:45 +01:00
Matthieu Gautier 57a197d38d Make getCurrentBookId const. 2018-03-12 17:34:45 +01:00
Matthieu Gautier b6ba10af2a Remove unnecessary currentArticleOffset.
This protected member is never used.
2018-03-12 17:34:45 +01:00
Matthieu Gautier f93f50087b Remove unnecessary setBookIndex.
We can use default argument instead of creating a new method.
2018-03-12 17:34:45 +01:00
Matthieu Gautier 6adf95c329 Make JNIKiwixReader throw an exception if something goes wrong at creation.
If the `nativeHandle` is null, the JNIKiwixReader is invalid and we must
not use it.

Throwing an exception for the caller code to handle this properly.
And previously, user code has no way to detect something went wrong :/
2018-02-01 17:18:54 +01:00
mhutti1 97bcf57d53 Allow JNI to access search snippets 2017-12-15 16:02:49 +00:00
mhutti1 f303c7502d Fix JNI to work with kiwix-android 2017-12-14 17:32:03 +00:00
Matthieu Gautier 16bd34e6a6 Add a method in the JNI API to get direct access information.
For binary content (not compressed), it could be interesting to
directly read the content in the zim file instead of using `kiwix-lib`.

This method returns the needed information to do so (if possible).
2017-12-13 17:22:26 +01:00
Matthieu Gautier 5a953f191b Remove a small warning. 2017-12-13 17:11:10 +01:00
Matthieu Gautier 41c92cfc3c Better calculate the start of the last search page.
The increment between pages should always be a multiple of
`resultCountPerPage`.
2017-11-27 12:39:04 +00:00
Matthieu Gautier 64dc5131c0 Be able to specify the global contentHumanReadableId without a index.
Even if we use the add_reader method to search into embedded full text
index, we need to specify the global `contentHumanReadableId` as it will
be used to generate "page links".
2017-11-27 12:37:13 +00:00
kelson42 28b0588df4 Better URL encoding 2017-11-23 19:26:41 +01:00
Matthieu Gautier 4e5d9f0360 Add a API to get only a part of a article content.
Add the jni method `getContentPart` to get only a part of the artcicle
content.

The method can be used to get a part of the content or to know the size
of the full content.
2017-11-22 19:06:54 +00:00
mhutti1 520c1edf31 Fix JNI android integration 2017-11-22 14:54:03 +00:00
mhutti1 d2f7503cfa Fix JNI for android integration 2017-11-22 14:54:03 +00:00
Matthieu Gautier 7a59779b77 Change JNI API of kiwix-lib (mainly to support multi-zims search)
This is a major API break. User code will have to be rewritten.

Before this commit, API was a unique object wrapping the library and
handle a global state with one `Reader` and one `Writer` at the time.

Now, the API is axed around three main objects :
 - The `JNIKiwixReader`, a wrapper around a `kiwix::Reader` (who allow to
   read one zim)
 - The `JNIKiwixSearcher`, a wrapper around a `kiwix::Searcher` (who allow
   to search through one or more reader(s))
 - The `JNIKiwixSearcher.Result` a result of a search. Allowing to get all
   information about a result (title, url, content, snippet, ...)
2017-11-22 14:54:03 +00:00