Commit Graph

40 Commits

Author SHA1 Message Date
Maneesh P M 8a4080baba Update libkiwix with new libzim api 2021-08-14 22:26:39 +05:30
Maneesh P M 9addd82d2d Fix usage of zim::Searcher::getResults() in libkiwix
The correct usage does not require the user to calculate an `end` using
the `pageLength`. We can directly use getResults(start, pageLength)
2021-08-04 19:20:50 +05:30
Matthieu Gautier fa83a61a54 Move all public *Tools.h in src.
This by definition remove all the tool functions from the public API.
2021-07-07 14:43:13 +02:00
Maneesh P M bcece66960 Add SearchRenderer handles for libzim structures
Introduces a new member mp_search that houses the zim::Search object,
adds a new constructor for this purpose. This commit also add an
overload for getHtml that takes start and end integers as arguments
since they are not part of the search object we include.
2021-07-03 14:05:50 +05:30
Maneesh P M e2f6d91d51 Remove get_readerIndex in favor of get_zimId
The function get_readerIndex was used to get the zimId using an ordered
vector of readers. Now we can use get_zimId directly.
2021-05-26 14:45:25 +02:00
Maneesh P M c35f6f9142 Add `get_zimId` method to Result
get_zimId method allows the user to get the uuid of the archive from
which a result is retrieved directly from the search result itself.
2021-05-26 14:45:25 +02:00
Maneesh P M e5fac30cee Update libkiwix with search iterator rename in libzim
Search iterator API in libzim has been shifted to use camel case naming.
This has to be accomodated in libkiwix as well.
2021-05-26 08:39:13 +02:00
Veloman Yunkan cd02b4de3b Dummy application of new libzim search API
Didn't take any advantage of the new libzim search API. Just fixed the
libkiwix build in the most straightforward way.
2021-05-15 23:34:51 +04:00
Veloman Yunkan 9d4370403b get_url() was renamed in zim::search_iterator 2021-04-16 13:30:36 +04:00
Matthieu Gautier 1a5a2e7a8e Adapt kiwix-lib to the new libzim api. 2020-12-02 12:16:48 +01:00
Matthieu Gautier 064d5f3fa6 Make the search argument constant. 2020-03-06 12:08:05 +01:00
Matthieu Gautier 87dc145dc7 Correctly set searcher information even if resultStart equals resultEnd. 2019-09-09 14:43:51 +02:00
Matthieu Gautier 64dfea2547 Move the search html renderer in a different class than the searcher.
This is two different functionnalies, we don't need to polute the searcher
api with things to render the html.
2019-08-11 10:19:48 +02:00
Matthieu Gautier cca5980b27 Remove limitation of the search len in the Searcher.
The limitation should be made elsewhere (the code using the searcher).
2019-08-11 10:19:48 +02:00
Matthieu Gautier 654a8e304c Set the verbosity of the underlying zim::Search. 2019-07-30 17:03:44 +02:00
Matthieu Gautier 92c9a47a0d Move the templating system to mustache instead of ctpp2.
Mustache templating system is a bit simpler than ctpp2 and ctpp2 is no
more maintained (see #189).
We are moving to the kainjow's Mustache project
(https://github.com/kainjow/Mustache).

It simplify a lot our system has it is header only and we don't have to
precompile the template.

Fix #21
2019-01-09 18:28:48 +01:00
Matthieu Gautier 5159d985c6 Remove support for external index.
This feature is considered obsolete for a while.
In fact, it was already not supported since June 2018 as we were compiling
xapian without the chert backend support.

Assume that we don't support it and remove it from the code.
See kiwix/kiwix-tools#245

This is a API break. library.xml files will still work but the indexPath
and indexType will be dropped silently from the file.
2019-01-07 16:47:08 +01:00
Matthieu Gautier 622d2fc23d Remove unused static resources.
Stop words are not use anymore since a long time now that indexing has
been moved to libzim. No need to embedded them in kiwix-lib.
2018-05-15 11:30:30 +02:00
Matthieu Gautier ec097ab267 Try to compile kiwix-lib without warning. 2018-04-18 16:57:27 +02:00
Matthieu Gautier ee6831d665 Check `internal->_search` before using it.
If a search has been set and a user try to get the nextResult or
restart the search, `internal->_search` will be NULL.
2018-03-12 17:45:18 +01:00
Matthieu Gautier f8a2e4c503 Only add a reader to the searcher if the reader as fulltext index.
`libzim` will not search in zim file without embedded fulltext index.
If we don't want to mess up with result index, we must not store "wrong"
reader.

Fix #111
2018-03-12 17:34:45 +01:00
Matthieu Gautier 41c92cfc3c Better calculate the start of the last search page.
The increment between pages should always be a multiple of
`resultCountPerPage`.
2017-11-27 12:39:04 +00:00
Matthieu Gautier 64dc5131c0 Be able to specify the global contentHumanReadableId without a index.
Even if we use the add_reader method to search into embedded full text
index, we need to specify the global `contentHumanReadableId` as it will
be used to generate "page links".
2017-11-27 12:37:13 +00:00
Matthieu Gautier b9ac7084ac Add small API to do geo query.
This is a small quick and dirty API to do geo query.

It is not possible with this API to do a query search and a geo search.
It's either one or the other.

We should think about a better global API to do searching and provide
both of them in the same time (libzim does it).
2017-11-14 17:32:06 +01:00
Matthieu Gautier 0e8c8f68c5 Extend search limits to 140.
70 is a too small limit for the number of results.
Users need at least 100.

As the html rendering will fails with more than 144 results,
explicitly limits the number of search to 140.

Fixes kiwix/kiwix-tools#92
2017-11-06 12:23:13 +01:00
Matthieu Gautier 382655d83c Explicitly set ctpp2 iIMaxSteps to extends search beyond 68 results.
Ctpp2 templates have a limit step number. If the template to render is
too big, the rendering fails, throwing an exception.

From our tests, it seems that, with the template we have, the default
step limit allow us to render 68 results only.

By doubling the limit, we can render up to 144 results.
2017-11-06 12:10:48 +01:00
Chris Li cf93c8719f Allow kiwix-lib to compile without ctpp2c.
ctpp2c is used to pre-compile the template resource.
However, on OSX, ctpp2c seems to be difficult to compile, as we don't need
ctpp2 at all on OSX/iOS, lets just stop to force the use of ctpp2c.
2017-08-14 10:42:16 -04:00
Matthieu Gautier 1ff1bf6168 Always set the humanReadableName with the readable in kiwix-search.
We always need a humanReadableName associated with a content to search in.
Do not separate the two values (human readable name and zim) in two
different functions.

This way, we avoid miss-use of the Searcher who could lead to segfault.
2017-08-10 09:20:11 -04:00
Matthieu Gautier bc5f4f5de4 Use right contentId to generate the article url in search template.
As we do multisearch, we must use the associated contentID of the result
to generate the url.
2017-07-18 10:04:40 +02:00
Matthieu Gautier 9cc329dbd2 Support multi-zims search in kiwix-lib.
All the code was already in zimlib.
It is mainly a update of the code using zimlib.

No JNI change for now to not break the API.
2017-07-18 10:04:40 +02:00
Matthieu Gautier 3991e648ed Be able to get the reader index from a search result. 2017-07-17 18:16:11 +02:00
Matthieu Gautier 8d39b0b343 Search result objects now have a get_content method.
This was not necessary when searching in only one zim file as `url` was
enough to get the article (and so the content).

If we want to search in several zim in the same time, we need a way to get
the content directly.
2017-07-17 18:16:11 +02:00
Matthieu Gautier d0371cd133 Suggestions now use xapian database when available.
If a embedded fulltext database is present, suggestion will search in it :
 - insensitive case search.
 - search for terms in the middle of the title.
 - xapian will try to complete the last word of the query (as if a '*'
   were added at the end)
2017-07-17 17:17:13 +02:00
Matthieu Gautier f76e9d2dbf Format all the code using clang-format.
Add a script `format_code.sh` to easily format the code.
2017-07-05 15:22:34 +02:00
Matthieu Gautier c44b2acb56 Re-add xapian searcher in kiwix-lib.
libzim only know how to read embedded full text index in a zim file.
This is nice as we want to embedded the full text index in zim file and
not have separated full text index.

However, we still have some zim+separated index we have to read.
So we have to support the search in separated index for a while.
2017-05-24 16:08:00 +02:00
Matthieu Gautier 5ca419bee7 Use the new search API in zimlib.
We do not use xapian anymore. This is all handled by zimlib.
2017-04-10 14:28:25 +02:00
Matthieu Gautier 83d27255cf Do not create all the results at once. Be a bit lazy.
We don't need to generate a vector of result when we do a search.
We better to just keep the handle to the current MSetIterator and
generate the wanted values when needed.
2017-03-21 16:20:17 +01:00
Matthieu Gautier 84796abb4c Make use of ctpp2 optional. 2017-02-22 12:05:44 +01:00
Matthieu Gautier cba71b4e75 Add a new script to compile resources.
- No more dependency to reswrap binary (everything is done in python)
- Resource strings can be directly accessed.
  As side effect, it add a check at compilation if the resource is
  declared and compiled in the binary.
- The resource content can be overwritten at runtime with a env variable.

There is also few clean in the static as some files shoul be in the tools
directory.

The compile_resource script is install to let other project use it.
2016-12-23 13:06:12 +01:00
Matthieu Gautier 8ce1fb0ba8 Switch build system to mesonbuild.
There is no more integrated build of dependencies in the build system.
Dependencies are discovered using pkg-config except for ctpp2 where there
is no pkg-config file.
2016-12-22 12:01:20 +01:00