This functions enable to stop and resume download with aria2.
The Downloader's constructor now checks the paused downloads with the
function "tellWaiting()" to get them at the start of kiwix-desktop.
Mingw doesn't implement it. So, we should not use it.
I suppose that it was working before because mingw package for debian trusty
simply no provides a "thread" header.
We may face to include the native "thread" header.
Rpath is not needed to run installed binaries. Most of GNU/Linux
distributions strictly forbid it.
Signed-off-by: Vitaly Zaitsev <vitaly@easycoding.org>
They are not used at all and the windows version need some extra libs
that complexify the code.
Remove them for now. If it happens that they are needed, we will readd
them.
We must use absolute path whenever possible.
Relative path has sense only related to the "interaction" with the user
(current directory, library location, ...).
The `common` name is from the time where kiwix was only one repository
for all the project (android, desktop, server...).
Now we have split the repositories and kiwix-lib is the "common" repo,
the "common" directory is somehow nonsense.
Mustache templating system is a bit simpler than ctpp2 and ctpp2 is no
more maintained (see #189).
We are moving to the kainjow's Mustache project
(https://github.com/kainjow/Mustache).
It simplify a lot our system has it is header only and we don't have to
precompile the template.
Fix#21
This feature is considered obsolete for a while.
In fact, it was already not supported since June 2018 as we were compiling
xapian without the chert backend support.
Assume that we don't support it and remove it from the code.
See kiwix/kiwix-tools#245
This is a API break. library.xml files will still work but the indexPath
and indexType will be dropped silently from the file.
The library now contains (simple) methods to handle bookmarks.
The bookmark are stored in a separate xml file.
Bookmark are mainly a couple (`zimId`, `articleUrl`).
However, in the xml we store a bit more data :
- The article's title (for display)
- The book's title, lang and date (for potential update of zim files)
By default, we are searching in the PATH env var.
However, with an appImage, the executable directory is not in the PATH,
so we have to use an absolute path if we can.
If we cannot find the aria2c executable in the executable directory let's
try to use the system one.
`reader.getFileSize()` return the size of the zim in Kbyte in a
`unsigned int` (32 bits). This is ok as it would overflow if the size
of the size is greater than 4294967295 kbytes (so ~4Tbytes).
However, we need to convert the return size into a unsigned 64 bits integer
else, when converting to bytes, we will overflow at 4Gbytes.
Even in `m_size` is a uint64_t.
When we do a search and paging the result, we need to display to the
user the total number of book, not only the `itemsPerPage`.
So, we need to parse correctly the xml to keep information of the total
number of book.
The value is store as a string in in the xml, so we cannot use getAsI.
We have to get the string and parse it to an int.
We cannot use strtoull because android stdc++ lib doesn't have it.
We have to implement our how parseFromString function using a
istringstream.
When parsing a opds feed, the favicon is a url, not a dataurl.
If we download the favicon all the times, it may take a lot of time to
parse the feed.
We store the url and download the favicon only when needed (when displayed)
- "winsock2.h" needs to be included before "windows.h". But if a
compilation unit include "windows.h" and after "networkTools.h", we
fails and it is complicated to handle. The include must not be in the
header but in the cpp
- windows define some ERROR macro. It is a pitty but we cannot use `ERROR`
in our enum.
- If build statically using mingw we need to define `CURL_STATICLIB`
Library client (kiwix-desktop) need to know when a book is added to
library by the manager. By using a LibraryManipulator, we can do
dependency injection.
It is up to the book to manage its attribute.
Also remove the `absolutePath` (and `indexAbsolutePath`). The `Book::path` is always stored
absolute.
The fact that the path can be stored absolute or relative in the
`library.xml` is not relevant for the book.
With jdk8, `javac` has an option `-h` to generate the header files of
native classes.
So there is no need to run `javah` several times.
As there is now only one command to run (`javac`), there is no need for
the wrapper script `gen_kiwix.sh`.
Fix#167
The correct path for xapian database should be "X/fulltext/xapian",
not "Z//fulltextIndex/xapian".
So lets check for the right path and fallback to the wrong one (but
used in old zims).
The double '/' in the path is a bug of zimwriterfs and is specific
to the xapian database.
We must handle this correctly in `hasFulltextIndex` and not (buggly) in
`pathExists`.
(Hopefully, it seems that pathExists were used only by hasFulltextIndex)
The previous API suffer different problems:
- It was difficult to handle articles redirecting to other article.
- It was not possible to get few information (title) without getting
the whole content.
The new API introduce the new class `Entry` that act as a proxy to an
article in the zim file.
Methods of `Reader` now return an `Entry` and the user has to call
`Entry`'s methods to get useful information.
No redirection is made explicitly.
If an entry is not found, an exception is raised instead of returning
an invalid `Entry`.
The common pattern to get the content of an entry become :
```
std::string content;
try {
auto entry = reader.getEntryFromPath(path);
entry = entry.getFinalEntry();
content = entry.getContent();
} catch (NoEntry& e) {
...
}
```
Older methods are keep (with the same behavior) but are marked as
deprecated.
The library's books are created in the metadata in the opds.
As the opds stream is by definition a distant "library", there is no
zim to read to complete missing information.
This can lead to incomplete `library.xml`.
The downloader is using libaria2.
For now, only one download can be run a the time.
A download will start only if (and as soon as) no download is running.
If we check the later the `loopCounter` with 42, we must pre-increment the
content. Else, in case of infinite loop, the `loopCounter` will be 43.
Related to kiwix/kiwix-tools#168
`libzim` will not search in zim file without embedded fulltext index.
If we don't want to mess up with result index, we must not store "wrong"
reader.
Fix#111
If the `nativeHandle` is null, the JNIKiwixReader is invalid and we must
not use it.
Throwing an exception for the caller code to handle this properly.
And previously, user code has no way to detect something went wrong :/
For binary content (not compressed), it could be interesting to
directly read the content in the zim file instead of using `kiwix-lib`.
This method returns the needed information to do so (if possible).
Even if we use the add_reader method to search into embedded full text
index, we need to specify the global `contentHumanReadableId` as it will
be used to generate "page links".
Add the jni method `getContentPart` to get only a part of the artcicle
content.
The method can be used to get a part of the content or to know the size
of the full content.
This is a major API break. User code will have to be rewritten.
Before this commit, API was a unique object wrapping the library and
handle a global state with one `Reader` and one `Writer` at the time.
Now, the API is axed around three main objects :
- The `JNIKiwixReader`, a wrapper around a `kiwix::Reader` (who allow to
read one zim)
- The `JNIKiwixSearcher`, a wrapper around a `kiwix::Searcher` (who allow
to search through one or more reader(s))
- The `JNIKiwixSearcher.Result` a result of a search. Allowing to get all
information about a result (title, url, content, snippet, ...)
This is a small quick and dirty API to do geo query.
It is not possible with this API to do a query search and a geo search.
It's either one or the other.
We should think about a better global API to do searching and provide
both of them in the same time (libzim does it).
70 is a too small limit for the number of results.
Users need at least 100.
As the html rendering will fails with more than 144 results,
explicitly limits the number of search to 140.
Fixeskiwix/kiwix-tools#92
Ctpp2 templates have a limit step number. If the template to render is
too big, the rendering fails, throwing an exception.
From our tests, it seems that, with the template we have, the default
step limit allow us to render 68 results only.
By doubling the limit, we can render up to 144 results.
ctpp2c is used to pre-compile the template resource.
However, on OSX, ctpp2c seems to be difficult to compile, as we don't need
ctpp2 at all on OSX/iOS, lets just stop to force the use of ctpp2c.
We always need a humanReadableName associated with a content to search in.
Do not separate the two values (human readable name and zim) in two
different functions.
This way, we avoid miss-use of the Searcher who could lead to segfault.
Add a `title` write argument to `getContent*` methods.
This argument is filled with the title of the content get.
Also update the JNI accordingly.
Related to kiwix/kiwix-android#214
This was not necessary when searching in only one zim file as `url` was
enough to get the article (and so the content).
If we want to search in several zim in the same time, we need a way to get
the content directly.
If a embedded fulltext database is present, suggestion will search in it :
- insensitive case search.
- search for terms in the middle of the title.
- xapian will try to complete the last word of the query (as if a '*'
were added at the end)
Matthieu Gautier
luddens
cyrillemoureaux
Vitaly Zaitsev
mhutti1
kelson42
Chris Li