smallsolar
/
libkiwix
mirror of https://github.com/kiwix/libkiwix.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,427 Commits 23 Branches 58 Tags 89 MiB
Commit Graph

397 Commits

Author SHA1 Message Date
Veloman Yunkan f751aff2fb Full case/diacritics insensitivity in catalog filtering 
Catalog filtering should now be case/diacritics insensitive for all
fields. However it is not validated for language, name and category
fields, and is validated for tags, creator & publisher only for text
supplied in the filter (but not for values read from the book).
 2021-04-27 16:59:21 +04:00
Veloman Yunkan 87dc9d2723 Made catalog filtering by query diacritics insensitive 
Catalog filtering by titles/description was sensitive to diacritics
present in the query string. Fixed that.

Also enhanced the unit test to validate the insensitivity to diacritics
present in either the title/description or the query string.
 2021-04-27 16:59:21 +04:00
Veloman Yunkan 3d5fd8f585 Catalog filtering by creator works via Xapian 2021-04-27 16:59:21 +04:00
Veloman Yunkan e805f68994 Enhanced & broke LibraryTest.filterByPublisher 2021-04-27 16:59:21 +04:00
Veloman Yunkan a759ab989f Catalog filtering by publisher works via Xapian 2021-04-27 16:59:21 +04:00
Veloman Yunkan 7ccd9ffcce Catalog filtering by language works via Xapian 2021-04-27 16:59:21 +04:00
Veloman Yunkan 0c0a37073b Catalog filtering by category works via Xapian 2021-04-27 16:59:21 +04:00
Veloman Yunkan 415c65cf03 Catalog filtering by book name works via Xapian 2021-04-27 16:59:21 +04:00
Veloman Yunkan 2f3f1a4859 Improved LibraryTest.filterByMultipleCriteria 2021-04-27 16:59:21 +04:00
Veloman Yunkan b9be742085 LibraryTest.filterByMaxSize 2021-04-27 16:59:21 +04:00
Veloman Yunkan 95c354a5fa LibraryTest.filterByCategory 2021-04-27 16:59:21 +04:00
Veloman Yunkan cdd272fc5a LibraryTest.filterByName 2021-04-27 16:59:21 +04:00
Veloman Yunkan ef962a9174 LibraryTest.filterByPublisher 2021-04-27 16:59:21 +04:00
Veloman Yunkan f063d350c6 LibraryTest.{filterLocal,filterRemote} 2021-04-27 16:59:21 +04:00
Veloman Yunkan d8fe593f59 Extended the unit-test library with 2 XML entries 2021-04-27 16:59:21 +04:00
Veloman Yunkan 22b8625033 Enter EXPECT_FILTER_RESULTS() 
This diff is easier to view if whitespace change is ignored.
 2021-04-27 16:59:21 +04:00
Veloman Yunkan 0f277ffa34 Enhanced the LibraryTest.filterByTags unit-test 2021-04-27 16:59:21 +04:00
Veloman Yunkan 068f7e5e95 New unit-test LibraryTest.filterByCreator 2021-04-27 16:59:21 +04:00
Veloman Yunkan 8c810d2d2f Enhanced the LibraryTest.filterByQuery unit-test 2021-04-27 16:59:21 +04:00
Veloman Yunkan 8c18a37961 Split LibraryTest.filterCheck into several tests 2021-04-27 16:59:21 +04:00
Veloman Yunkan db3e0d7f72 Enhanced the LibraryTest.filterCheck unit-test 
Now the LibraryTest.filterCheck unit-test validates the actual entries
returned by `Library::filter` (previously only the count of the results
was checked).
 2021-04-27 16:59:21 +04:00
Manan Jethwani 965b9622c2 removed redirect to articles in search 2021-04-20 20:23:42 +05:30
Veloman Yunkan 0c7d19ab45 Testing of Manager.readXml() 2021-04-12 15:14:12 +02:00
Veloman Yunkan 49940a30d0 XmlLibraryTest.removeBookByIdUpdatesTheSearchDB 
The new unit-test fails with a reason not expected before it was
written. The `Library::filter()` operation returns a correct result
after the call to `removeBookById()` (this was a surprise!) but it has
a side-effect of re-adding an empty book with the id still surviving
in the search DB (the emptiness of this re-created book doesn't allow
it to pass the other filtering criteria, which explains why the result
of `Library::filter()` is correct). Had to add a special check
to the new unit-test against that hidden side-effect of
`Library::removeBookById()` + `Library::filter()` combination.
 2021-04-09 17:06:45 +04:00
Veloman Yunkan ccdc316217 Two unit-tests for Library::removeBookById 
The `XmlLibraryTest.removeBookByIdDropsTheReader` unit-test fails,
demonstrating a bug in `kiwix::Library::removeBookById()`.
 2021-04-09 16:59:55 +04:00
Veloman Yunkan 09233bf4f3 Support for partial queries in catalog search 
The search text in the catalog query is interpreted as partial by
default, but partial query mode can be disabled in C++. The latter
possibility is not exposed via the /catalog/search kiwix-serve endpoint,
though.
 2021-03-17 14:32:03 +01:00
Veloman Yunkan 47c67a4202 LibraryServerTest.catalog_search_with_word_exclusion 2021-03-17 14:32:03 +01:00
Veloman Yunkan 6b600a18eb LibraryServerTest.catalog_prefix_search 2021-03-17 14:32:03 +01:00
Veloman Yunkan 9e887cadf1 Added some diversity to test/data/library.xml 2021-03-17 14:32:03 +01:00
Veloman Yunkan a599fb3892 Initial version of Xapian-based catalog search 2021-03-17 14:32:03 +01:00
Veloman Yunkan b7b0bdbdd8 Both Book::update() methods update the category 2021-03-17 14:10:57 +04:00
Veloman Yunkan a870e05621 Slight enhancement of BookTest.updateTest 2021-03-17 14:10:57 +04:00
Veloman Yunkan 4abc4f8518 Support for book category attribute in library.xml 2021-03-17 14:10:57 +04:00
Veloman Yunkan 6b2067c236 Reading category element from OPDS stream 2021-03-17 14:10:57 +04:00
Veloman Yunkan e55bf514e8 Dedicated 'category' parameter in catalog search 2021-03-17 14:10:57 +04:00
Veloman Yunkan f270724b1f Testing of a library entry without a category 2021-03-17 14:10:44 +04:00
Veloman Yunkan 58186ffb26 kiwix::Book::getCategory() 2021-03-17 14:09:48 +04:00
Veloman Yunkan 6d43fd065f Less boilerplate in LibraryServerTest unit-tests 2021-03-17 14:02:27 +04:00
Veloman Yunkan 071d2bedd3 LibraryServerTest.catalog_search_by_text 2021-03-17 14:02:27 +04:00
Veloman Yunkan 0b1740e6c5 LibraryServerTest.catalog_search_by_tag 2021-03-17 14:02:27 +04:00
Veloman Yunkan 9913f748e2 LibraryServerTest.catalog_searchdescription_xml 2021-03-17 14:02:27 +04:00
Veloman Yunkan c5c40cb189 New unit-test LibraryServerTest.catalog_root_xml 2021-03-17 14:02:27 +04:00
Matthieu Gautier 17bc1a3e1a [TEST] Do not try to use the bookId if it is wrong. 2021-01-26 17:53:25 +01:00
Matthieu Gautier 1a5a2e7a8e Adapt kiwix-lib to the new libzim api. 2020-12-02 12:16:48 +01:00
Veloman Yunkan 0f8fe1f63f Alternative implementation of parseMimetypeCounter() 2020-10-29 14:11:27 +04:00
Matthieu Gautier ed32e16db2 Use a reference in `test/server.cpp` loop. 
This is mainly to make the macos CI pass.
 2020-10-28 16:08:37 +01:00
Matthieu Gautier 08464f23bc Better parsing of `M/Counter` 
Mimetype may contain a parameters.
Then, the mimetype would be something like "text/html;foo=bar;foz=baz"

It will contains a `;` and `=` and it conflicts with the same operators
we use to separate the items in our list.

We have to use a more advanced algorithm which takes the context into
account.

Fix #416
 2020-10-28 16:03:18 +01:00
Matthieu Gautier ef42abea4b Add some tests of `parseMimetypeCounter` 2020-10-28 14:44:23 +01:00
Veloman Yunkan 18e46969b7 Workaround for configure_file(copy:true) 
The `copy` keyword argument of `configure_file()` first appears in
meson 0.47. Now performing file copy via python.
 2020-08-11 18:17:18 +02:00
Veloman Yunkan e8cc6e4205 Copying data files via a loop 2020-08-11 18:17:18 +02:00
Veloman Yunkan 5f0bcd2bfa Renamed wikipedia_en_ray_charles_mini_2020-03.zim 2020-08-11 18:17:18 +02:00
Veloman Yunkan a3bef76083 example.zim is copied only if gtest is available 2020-08-11 18:17:18 +02:00
MananJethwani 599aaa4c1b added code for status code 204 for empty return of search. 2020-08-01 01:45:42 +05:30
Veloman Yunkan 3d425f44de Request header case is ignored 
Originally reported against case sensitivity of the Range header
(see issue #387), this fix applies to all request headers (since
according to RFC 7230 all header fields are case-insensitive, see
https://tools.ietf.org/html/rfc7230#section-3.2). However, a
corresponding unit-test was added only for the Range header.
 2020-07-30 16:01:51 +02:00
Matthieu Gautier 7ece383004 Add support for samba path on windows. 
Fix kiwix/kiwix-desktop#429
 2020-07-15 11:40:02 +02:00
Matthieu Gautier 4ca9558e30 Fix library test on windows. 2020-07-15 11:40:02 +02:00
Kelson ee204a9b5e Increase test timeout to 160s 2020-07-09 10:02:36 +02:00
Matthieu Gautier 157d1664cf Fix test compilation on bionic 2020-06-09 12:10:05 +02:00
Veloman Yunkan 1cdf830217 Testing of byte-range requests of 0-sized entries 2020-06-03 14:18:22 +04:00
Veloman Yunkan 0b48ab20bb Enhanced the server unit-test with corner cases 2020-06-03 13:45:31 +04:00
Veloman Yunkan a9b6d481cc ServerTest.RangeHasPrecedenceOverCompression 2020-05-26 13:58:20 +04:00
Veloman Yunkan c2ebdefe8d Handling of unsatisfiable ranges 2020-05-26 02:11:26 +04:00
Veloman Yunkan 931e95f391 Invalid byte ranges result in 416 responses 2020-05-26 01:40:07 +04:00
Veloman Yunkan f7571b5b69 Content-Range header is set only for partial content 2020-05-25 17:42:18 +04:00
Veloman Yunkan 52f207eaa6 Support for single-ended byte ranges 2020-05-25 16:37:01 +04:00
Veloman Yunkan bd2d0bc489 Unit-test for valid single range requests 2020-05-22 17:39:00 +04:00
Matthieu Gautier d8498fd655 Do not build `server` test on windows. 
On the appveyor CI, the link of the `server` test fails with
`libmicrohttpd`.

Do not build the test on windows to not break the CI.
 2020-05-18 13:13:17 +02:00
Matthieu Gautier 7ec8e33b83 Fix include of `httplib.h` on windows. 
On windows, `httplib.h` must be included before `windows.h`
We do not include directly `windows.h` in the test but it is included
indirectly by other headers.

Let's include httplib first.
 2020-05-18 11:16:25 +02:00
Kelson 94c2ab4395 Add two OPDS related mime-types to compress for HTTP 2020-05-18 08:19:51 +02:00
Kelson 2df74d9755 Fix small compilation error in tests 2020-05-17 16:47:21 +02:00
Veloman Yunkan 54f5dbbd35 Handling of If-None-Match conditional requests 2020-05-14 17:01:22 +04:00
Veloman Yunkan 95a5cde359 ETags are set in the response as needed 
Also added server-unit tests related to ETags in the response.
 2020-05-14 17:01:22 +04:00
Veloman Yunkan 3d08ef43f2 HEAD request is not rejected 
libmicrohttpd handles HEAD requests by dropping the body of the response
(if any). Hence letting a HEAD request through into the code that
processes GET requests is safe.

Also added server unit-tests related to the handling of HEAD requests.
 2020-05-14 17:01:22 +04:00
Veloman Yunkan 9a893a854e Revert "Server can be started on a random free port" 
This change failed to build under the following platforms
due to an older version of libmicrohttpd missing support for
the MHD_DAEMON_INFO_BIND_PORT query:
- Linux (native_dyn)
- Linux (win32_dyn)
 2020-04-29 15:40:02 +04:00
Veloman Yunkan b0f65a02f2 Server can be started on a random free port 
If the server is started with a port value of 0, it binds to a random
free port. The bound port can be obtained with a `getPort()` method.
 2020-04-29 15:40:02 +04:00
Veloman Yunkan 9bf6d0621f Introducing 1st real unit test of the server 
Added a new unit test 'test/server.cpp'. The pre-existing unit test
test/kiwixserve.cpp tests kiwixlib indirectly by running the kiwix-serve
executable which may be built with another version of the library.

Included with this commit is the cpp-httplib C++ HTTP/HTTPS library
(https://github.com/yhirose/cpp-httplib, v0.5.11).
The new file test/httplib.h was copied from
https://raw.githubusercontent.com/yhirose/cpp-httplib/v0.5.11/httplib.h
 2020-04-29 15:39:18 +04:00
luddens 0630298fb9 Manager::addBookFromPathAndGetId unit test 2020-04-28 10:56:12 +02:00
Matthieu Gautier 46626a3f98 Add the method get bookByPath in library. 2020-03-06 12:08:05 +01:00
Matthieu Gautier 7e26f3502d Fix opds dumper/parser. 
Add missing attributes.
 2020-01-30 15:42:54 +01:00
luddens ff21a095cb update book even if the members aren't empty 
remove the conditions to always update the book
 2020-01-27 17:50:44 +01:00
Matthieu Gautier 11118efd84 Do not build tests if we are cross-compiling 2020-01-21 10:58:43 +01:00
Matthieu Gautier 4fe31a20e3 Fix tests. 
As we use the main library of gtest (if already installed)
we don't need to (and must not) define a main function for the tests.

Does the same (use main library) if we use the subproject.
 2020-01-20 17:00:18 +01:00
Kelson b7c5e5f339 Remove trailing spaces 2019-12-08 11:52:16 +01:00
luddens bd6797143c add getBookmarks test 2019-10-31 15:01:52 +02:00
luddens a2c2955f41 add kiwixserve unit test 2019-10-28 16:00:26 +01:00
Matthieu Gautier 67170709bb Convert path get from windows environment to utf8. 
Fix kiwix/kiwix-desktop#203
 2019-09-25 18:07:42 +02:00
Matthieu Gautier 598dd3c175 [API Break] Fix pathTools (and a bit stringTools). 
Api changes :
 - removeLastPathElement do not takes extra arguments
   `removePreSeparator` and `removePostSeparator`.
   This is not needed as path do not need special tailing separator.
 - Only one function `split`. Arguments can be implicitly convert to
   string. No need for overloading functions to explicitly cast them.
 - `split` function takes another argument `trimEmpty`. If true, empty
   element are removed.

Path manipulation now almost pass trough a vector<string> to store each
path's part.

Most of the complex works is now made in the normalizeParts function.
 2019-09-19 18:16:06 +02:00
Matthieu Gautier 2f4636e2df Fix stringTools join function. 2019-09-17 16:22:28 +02:00
Matthieu Gautier 6ee174b546 Add a method to get the value of a specific tag. 
Fix #258
 2019-09-17 10:37:53 +02:00
Matthieu Gautier 2a6772b76d [API Change] Convert tags to the new convention. 
Use the new convention describe here : https://wiki.openzim.org/wiki/Tags
 2019-09-17 10:30:24 +02:00
Matthieu Gautier 3e3e1683f5 Make the gtest dependency optional. 
This is needed to build in flatpak.
 2019-08-12 15:31:16 +02:00
Matthieu Gautier ca27ddf41a Do not embed the gtest dependency. 
We have a special wrap file for that.
 2019-08-12 14:55:39 +02:00
Matthieu Gautier fee2da57e6 Add tests of regex tools. 2019-08-12 12:05:51 +02:00
Matthieu Gautier 31c9375a3a Better API to filter books in a library. 
Instead of having a single method `listBooksIds` that tries to be
exhaustive about all the filter and sort option, split the method in
two separated methods `filter` and `sort`.

The `filter` method takes a `Filter` object that represent on what we are
filtering. This object has to be construct before calling `filter`.

```cpp
Filter filter;
filter.query("Astring");
filter.acceptTags({"nopic"});
// return all book in eng and with "Astring" in the tile or description".
library.filter(filter);
//equivalent to
library.listBooksIds(ALL, UNSORTED, "Astring", "", "", "", {"nopic"});
// or better
library.filter(Filter().query("Astring").acceptTags({"nopic"}));
```

The method `listBooksIds` has been marked as deprecated.

Add a small test on the library.
 2019-06-26 16:41:01 +02:00
Matthieu Gautier bf2188af14 [Windows] Add extra link arguments to build test on windows. 2018-08-03 19:22:39 +02:00
Matthieu Gautier c0b1c6013e Fix parsing of url 
Fix kiwix/kiwix-tools#193
 2018-05-14 17:41:05 +02:00
Matthieu Gautier 433a47c3fe Add unittest structure. 
No tests, just everything to add tests later.
 2018-05-14 17:40:43 +02:00