Commit Graph

1645 Commits

Author SHA1 Message Date
9c7366890d Catalog filtering by tags works via Xapian 2021-04-27 16:59:21 +04:00
19e195cb7d Filter::Tags typedef 2021-04-27 16:59:21 +04:00
3d5fd8f585 Catalog filtering by creator works via Xapian 2021-04-27 16:59:21 +04:00
d3d5abe14d Handling of non-words in publisher query
This change fixes the failure of the LibraryTest.filterByPublisher
unit-test broken by the previous commit.

The previous approach used in `publisherQuery()` for building a phrase
query enforcing the specified prefix for all terms fails if

1. the input phrase contains a non-word term that Xapian's query parser
   doesn't like (e.g. a standalone ampersand character, 1/2, a#1, etc);
2. the input phrase contains at least three terms that Xapian's query
   parser has no issue with.

Using the `quest` tool (coming with xapian-tools under Ubuntu) the
issue can be demonstrated as follows:

```
$ quest -o phrase -d some_xapian_db "Energy & security"
Parsed Query: Query((energy@1 PHRASE 11 Zsecur@2))
Exactly 0 matches
MSet:

$ quest -o phrase -d some_xapian_db "Energy & security act"
UnimplementedError: OP_NEAR and OP_PHRASE only currently support leaf subqueries

$ quest -o phrase -d some_xapian_db 'Energy 1/2 security act'
UnimplementedError: OP_NEAR and OP_PHRASE only currently support leaf subqueries

$ quest -o phrase -d some_xapian_db "Energy a#1 security act"
UnimplementedError: OP_NEAR and OP_PHRASE only currently support leaf subqueries
```

The problem comes from parsing the query with the default operation set
to `OP_PHRASE` (exemplified by the `-o phrase` option in above
invocations of `quest`). A workaround is to parse the phrase with a
default operation of `OP_OR` and then combine all the terms with
`OP_PHRASE`.

Besides stemming should be disabled in order to target an exact phrase
match (save for the non-word terms, if any, that are ignored by the
query parser).
2021-04-27 16:59:21 +04:00
e805f68994 Enhanced & broke LibraryTest.filterByPublisher 2021-04-27 16:59:21 +04:00
a759ab989f Catalog filtering by publisher works via Xapian 2021-04-27 16:59:21 +04:00
7ccd9ffcce Catalog filtering by language works via Xapian 2021-04-27 16:59:21 +04:00
0c0a37073b Catalog filtering by category works via Xapian 2021-04-27 16:59:21 +04:00
415c65cf03 Catalog filtering by book name works via Xapian 2021-04-27 16:59:21 +04:00
8287f351e7 Final logic of Library::filterViaBookDB()
Moved the `filter.hasQuery()` check inside `buildXapianQuery()`.
`Library::filterViaBookDB()` only cares if the query that is going to be
run on the book DB would match all documents. The rest of changes
related to enhancing the usage of Xapian for the catalog search will
happen inside `buildXapianQuery()` and `updateBookDB()`.
2021-04-27 16:59:21 +04:00
ea779ac200 Extracted buildXapianQuery() 2021-04-27 16:59:21 +04:00
80cd1fc989 Renamed 2 functions in Filter and Library 2021-04-27 16:59:21 +04:00
2d76f8395e Dropped unused functions from Filter's private API
This should have been done back in PR #460
2021-04-27 16:59:21 +04:00
29a6a34ecf Delimited kiwix::Filter's public and private APIs 2021-04-27 16:59:21 +04:00
2f3f1a4859 Improved LibraryTest.filterByMultipleCriteria 2021-04-27 16:59:21 +04:00
b9be742085 LibraryTest.filterByMaxSize 2021-04-27 16:59:21 +04:00
95c354a5fa LibraryTest.filterByCategory 2021-04-27 16:59:21 +04:00
cdd272fc5a LibraryTest.filterByName 2021-04-27 16:59:21 +04:00
ef962a9174 LibraryTest.filterByPublisher 2021-04-27 16:59:21 +04:00
f063d350c6 LibraryTest.{filterLocal,filterRemote} 2021-04-27 16:59:21 +04:00
d8fe593f59 Extended the unit-test library with 2 XML entries 2021-04-27 16:59:21 +04:00
22b8625033 Enter EXPECT_FILTER_RESULTS()
This diff is easier to view if whitespace change is ignored.
2021-04-27 16:59:21 +04:00
0f277ffa34 Enhanced the LibraryTest.filterByTags unit-test 2021-04-27 16:59:21 +04:00
068f7e5e95 New unit-test LibraryTest.filterByCreator 2021-04-27 16:59:21 +04:00
8c810d2d2f Enhanced the LibraryTest.filterByQuery unit-test 2021-04-27 16:59:21 +04:00
8c18a37961 Split LibraryTest.filterCheck into several tests 2021-04-27 16:59:21 +04:00
db3e0d7f72 Enhanced the LibraryTest.filterCheck unit-test
Now the LibraryTest.filterCheck unit-test validates the actual entries
returned by `Library::filter` (previously only the count of the results
was checked).
2021-04-27 16:59:21 +04:00
d134ad417f Merge pull request #497 from MananJethwani/issue/481
removed redirect to articles in search
2021-04-20 17:13:01 +02:00
965b9622c2 removed redirect to articles in search 2021-04-20 20:23:42 +05:30
11db5dec4e Merge pull request #494 from kiwix/ripple_effect_of_libzim_pr524
get_url() was renamed in zim::search_iterator
2021-04-16 12:38:15 +02:00
9d4370403b get_url() was renamed in zim::search_iterator 2021-04-16 13:30:36 +04:00
cb57178c23 Merge pull request #491 from kiwix/fix_macos_ci
Update brew before installing packages
2021-04-12 18:41:00 +02:00
9ba5ab4678 Update brew before installing packages
brew changed his backend repository, we must update brew itself first.
2021-04-12 18:31:29 +02:00
a597870025 Merge pull request #465 from soumyankar/master 2021-04-12 18:17:57 +02:00
611146aa37 Added Search Link for bad bookName/articleName on 404 2021-04-12 21:31:47 +05:30
6d2f227c42 Merge pull request #486 from kiwix/fix_for_issue462 2021-04-12 15:18:57 +02:00
0c7d19ab45 Testing of Manager.readXml() 2021-04-12 15:14:12 +02:00
b54215f146 Manager::readOpds() doesn't modify its input 2021-04-12 15:14:12 +02:00
9033f2f28e Manager::readXml() doesn't modify its input 2021-04-12 15:14:12 +02:00
5c289abd0e Merge pull request #485 from kiwix/fix_for_issue478 2021-04-12 15:05:13 +02:00
ec9186b174 Library::removeBookById() updates the search DB
This fix makes the `XmlLibraryTest.removeBookByIdUpdatesTheSearchDB`
unit-test pass.
2021-04-09 17:06:45 +04:00
aaaa5a637e Library::filter() doesn't create empty books
This changes how the `XmlLibraryTest.removeBookByIdUpdatesTheSearchDB`
unit-test fails.
2021-04-09 17:06:45 +04:00
49940a30d0 XmlLibraryTest.removeBookByIdUpdatesTheSearchDB
The new unit-test fails with a reason not expected before it was
written. The `Library::filter()` operation returns a correct result
after the call to `removeBookById()` (this was a surprise!) but it has
a side-effect of re-adding an empty book with the id still surviving
in the search DB (the emptiness of this re-created book doesn't allow
it to pass the other filtering criteria, which explains why the result
of `Library::filter()` is correct). Had to add a special check
to the new unit-test against that hidden side-effect of
`Library::removeBookById()` + `Library::filter()` combination.
2021-04-09 17:06:45 +04:00
24ed96a38c Library.removeBookById() drops the reader too
This fix makes the `XmlLibraryTest.removeBookByIdDropsTheReader`
unit-test pass.
2021-04-09 17:05:56 +04:00
ccdc316217 Two unit-tests for Library::removeBookById
The `XmlLibraryTest.removeBookByIdDropsTheReader` unit-test fails,
demonstrating a bug in `kiwix::Library::removeBookById()`.
2021-04-09 16:59:55 +04:00
ba44033273 Merge pull request #464 from MananJethwani/issue/kiwix-tools/205
adding kind and path attributes to suggest response object and using it in autocomplete
2021-04-07 18:08:29 +02:00
5cb276a933 adding kind and path attributes to suggest response object and using it in autocomplete 2021-04-07 21:04:33 +05:30
e4be97a032 Merge pull request #470 from kiwix/xapian_should_not_be_exposed 2021-04-07 16:55:24 +02:00
aa2a031ba4 Xapian headers are not exposed through libkiwix 2021-04-07 18:24:33 +04:00
803cb1c2c5 Merge pull request #476 from MananJethwani/correcting#474
changed method of injecting root link
2021-03-24 10:20:54 +01:00