It is up to the book to manage its attribute.
Also remove the `absolutePath` (and `indexAbsolutePath`). The `Book::path` is always stored
absolute.
The fact that the path can be stored absolute or relative in the
`library.xml` is not relevant for the book.
With jdk8, `javac` has an option `-h` to generate the header files of
native classes.
So there is no need to run `javah` several times.
As there is now only one command to run (`javac`), there is no need for
the wrapper script `gen_kiwix.sh`.
Fix#167
The correct path for xapian database should be "X/fulltext/xapian",
not "Z//fulltextIndex/xapian".
So lets check for the right path and fallback to the wrong one (but
used in old zims).
The double '/' in the path is a bug of zimwriterfs and is specific
to the xapian database.
We must handle this correctly in `hasFulltextIndex` and not (buggly) in
`pathExists`.
(Hopefully, it seems that pathExists were used only by hasFulltextIndex)
The previous API suffer different problems:
- It was difficult to handle articles redirecting to other article.
- It was not possible to get few information (title) without getting
the whole content.
The new API introduce the new class `Entry` that act as a proxy to an
article in the zim file.
Methods of `Reader` now return an `Entry` and the user has to call
`Entry`'s methods to get useful information.
No redirection is made explicitly.
If an entry is not found, an exception is raised instead of returning
an invalid `Entry`.
The common pattern to get the content of an entry become :
```
std::string content;
try {
auto entry = reader.getEntryFromPath(path);
entry = entry.getFinalEntry();
content = entry.getContent();
} catch (NoEntry& e) {
...
}
```
Older methods are keep (with the same behavior) but are marked as
deprecated.
The library's books are created in the metadata in the opds.
As the opds stream is by definition a distant "library", there is no
zim to read to complete missing information.
This can lead to incomplete `library.xml`.
The downloader is using libaria2.
For now, only one download can be run a the time.
A download will start only if (and as soon as) no download is running.
If we check the later the `loopCounter` with 42, we must pre-increment the
content. Else, in case of infinite loop, the `loopCounter` will be 43.
Related to kiwix/kiwix-tools#168
`libzim` will not search in zim file without embedded fulltext index.
If we don't want to mess up with result index, we must not store "wrong"
reader.
Fix#111
If the `nativeHandle` is null, the JNIKiwixReader is invalid and we must
not use it.
Throwing an exception for the caller code to handle this properly.
And previously, user code has no way to detect something went wrong :/
For binary content (not compressed), it could be interesting to
directly read the content in the zim file instead of using `kiwix-lib`.
This method returns the needed information to do so (if possible).
Even if we use the add_reader method to search into embedded full text
index, we need to specify the global `contentHumanReadableId` as it will
be used to generate "page links".
Add the jni method `getContentPart` to get only a part of the artcicle
content.
The method can be used to get a part of the content or to know the size
of the full content.
This is a major API break. User code will have to be rewritten.
Before this commit, API was a unique object wrapping the library and
handle a global state with one `Reader` and one `Writer` at the time.
Now, the API is axed around three main objects :
- The `JNIKiwixReader`, a wrapper around a `kiwix::Reader` (who allow to
read one zim)
- The `JNIKiwixSearcher`, a wrapper around a `kiwix::Searcher` (who allow
to search through one or more reader(s))
- The `JNIKiwixSearcher.Result` a result of a search. Allowing to get all
information about a result (title, url, content, snippet, ...)
This is a small quick and dirty API to do geo query.
It is not possible with this API to do a query search and a geo search.
It's either one or the other.
We should think about a better global API to do searching and provide
both of them in the same time (libzim does it).
70 is a too small limit for the number of results.
Users need at least 100.
As the html rendering will fails with more than 144 results,
explicitly limits the number of search to 140.
Fixeskiwix/kiwix-tools#92
Ctpp2 templates have a limit step number. If the template to render is
too big, the rendering fails, throwing an exception.
From our tests, it seems that, with the template we have, the default
step limit allow us to render 68 results only.
By doubling the limit, we can render up to 144 results.
ctpp2c is used to pre-compile the template resource.
However, on OSX, ctpp2c seems to be difficult to compile, as we don't need
ctpp2 at all on OSX/iOS, lets just stop to force the use of ctpp2c.
We always need a humanReadableName associated with a content to search in.
Do not separate the two values (human readable name and zim) in two
different functions.
This way, we avoid miss-use of the Searcher who could lead to segfault.
Add a `title` write argument to `getContent*` methods.
This argument is filled with the title of the content get.
Also update the JNI accordingly.
Related to kiwix/kiwix-android#214
This was not necessary when searching in only one zim file as `url` was
enough to get the article (and so the content).
If we want to search in several zim in the same time, we need a way to get
the content directly.
If a embedded fulltext database is present, suggestion will search in it :
- insensitive case search.
- search for terms in the middle of the title.
- xapian will try to complete the last word of the query (as if a '*'
were added at the end)
Compilation fails on clang 3.4.1 (and presumably later, tho I haven't tested) with
```
src/reader.cpp:131:59: error: no viable conversion from 'iterator' (aka '__map_iterator<typename __base::iterator>') to 'std::map<std::string, unsigned int>::const_iterator' (aka '__map_const_iterator<typename __base::const_iterator>')
std::map<std::string, unsigned int>::const_iterator it = counterMap.find("text/html");
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/v1/map:713:29: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'iterator' (aka '__map_iterator<typename __base::iterator>') to 'const std::__1::__map_const_iterator<std::__1::__tree_const_iterator<std::__1::__value_type<std::__1::basic_string<char>, unsigned int>, std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char>, unsigned int>, void *> *, long> > &' for 1st argument
class _LIBCPP_TYPE_VIS_ONLY __map_const_iterator
^
```
because we are not using the right type for the map iterator. As we are using
C++11, let's use `auto` and make compiler set the right type for us.
Android will look in specific repository to find native libs.
We need to use the `android_abi` name defined in the cross_compilation
file instead of `cpu_family`.
libzim only know how to read embedded full text index in a zim file.
This is nice as we want to embedded the full text index in zim file and
not have separated full text index.
However, we still have some zim+separated index we have to read.
So we have to support the search in separated index for a while.
`kiwix-android` is using `kiwix-lib` as an external java application now.
So we need `kiwix-lib` build system to also install application files
(manifest, resources, ..).
In C++11 `linux` is not a reserved word, so compilators do not define it.
A correct way to check if we are compiling for linux is to check for
`__linux__`.
Unicode headers ends by defining the DONE symbol in a enum.
It can clash with other includes.
(For instance the httpd.h from apache who use `#define DONE -2`).
Both project should not declare such common symbols publicly but we have
to do with them anyway.
Commit b8d950c removes this symbol.
The indexer is not used anymore and will be soon removed.
So for now, just remove the call to readStopWords until we totally
remove the indexer code.
Matthieu Gautier
mhutti1
kelson42
Chris Li
Steve Wills