Commit Graph

157 Commits

Author SHA1 Message Date
Veloman Yunkan 60148717e1 Fixed search results for kiwix-desktop 2022-09-26 13:11:25 +04:00
Veloman Yunkan cac2d212c6 Respecting the --nosearchbar option of kiwix-serve
If `kiwix-serve` is run with the `--nosearchbar` option the toolbar is
disabled (hidden) in its viewer.

Note however that certain actions performed by the viewer merely with
the purpose of keeping the toolbar up-to-date are still carried out.
2022-09-21 15:41:40 +04:00
Veloman Yunkan da23e4eca4 Revert "Partly respecting the kiwix-serve --nosearchbar option"
This reverts commit 436d890893713c5eb98df6893d0e0b41b22e2472.
2022-09-21 15:41:40 +04:00
Veloman Yunkan 2be9ac342f Partly respecting the kiwix-serve --nosearchbar option
`--nosearchbar` option of `kiwix-serve` (despite its misleading name)
was used to disable the entire taskbar. This commit accounts for the
existence of that option only partially:

1. Links to books on the welcome/library page are affected - by default
   books are displayed in the viewer, but in a kiwix-serve instance run
   with --nosearchbar books are loaded in the top window.

2. The `/viewer` endpoint is enabled unconditionally, so if anyone
   enters the viewer URL in the address bar they will see books in the
   viewer.
2022-09-21 15:41:40 +04:00
Veloman Yunkan 369406fb5d Viewer settings
Made the viewer respect the `--blockexternal` and `--nolibrarybutton`
options of `kiwix-serve`. Those options are passed to the viewer
via the dynamically generated resource `/viewer_settings.js`.
2022-09-21 15:41:40 +04:00
Veloman Yunkan b81cb3a8e9 Got rid of raw mode in response generation 2022-09-21 15:41:40 +04:00
Veloman Yunkan 0ce36e6246 Got rid of isHomePage in ContentResponse::build() 2022-09-21 15:41:40 +04:00
Veloman Yunkan eb0a45b13e Undefaulted bool params of ContentResponse::build()
This resulted in compiler aided discovery of all call sites where the
default values were used. For OPDS/catalog requests now passing true for the
`raw` parameter, since XML content isn't supposed to undergo any
transformations.
2022-09-21 15:41:40 +04:00
Veloman Yunkan c988511561 Removed unused param from ContentResponse::build()
Removed the isHomePage param from one of the variants of
`ContentResponse::build()`. The other overload is dangerous since
failing to review&update all of its call site may result in changed
semantics. Will do it in a couple of separate commits.
2022-09-21 15:41:40 +04:00
Veloman Yunkan 0cf4850a9b Dropped TaskbarInfo 2022-09-21 15:41:40 +04:00
Veloman Yunkan 4db443eca6 Embryo of iframe-based viewer 2022-09-21 15:41:40 +04:00
Emmanuel Engelhart 1062bd73a3
It's libkiwix, not kiwixlib 2022-09-11 16:05:25 +02:00
Veloman Yunkan e323dcf6c9 Redirecting /nonendpoint URLs to /content/nonendpoint 2022-08-11 18:04:05 +04:00
Veloman Yunkan 3b98987cb3 More robust handling of endpoint URLs
The next goal is to redirect old-style /book/path/to/entry URLs to
/content/book/path/to/entry, which seemed pretty trivial.

However, given the current handling of some endpoint URLs, more work was
required to ensure that invalid endpoint URLs (e.g.  "/random/number" or
"/suggest/fr") are not interpreted as content URLs. Previously, that was
not a user-observable issue, since the result would be an immediate 404
error (except in certain edge cases, like handling the request for
"/random/number" when there is a book with name "random" containing an
article at path "/number"). With redirection of URLs that were assumed
to refer to content a 404 error would be issued for the
transformed URL ("/content/random/number") which may be confusing.

Therefore this change is to ensure the correct routing of endpoint URL
handling.
2022-08-11 18:04:05 +04:00
Veloman Yunkan 1b1c1e352e Introduced /content endpoint
Book content is now served under /content/book/...

The old access to book content via a top-level URL /book/... is so far
preserved for backward compatibility.

Redirects were changed to use the new URL scheme. Links in the search results
still use the old scheme.
2022-08-11 18:04:05 +04:00
Veloman Yunkan a4b18893aa Moved handling of the "/" URL 2022-08-11 18:04:05 +04:00
Matthieu Gautier 69931fb347
Remove libzim's wrapper.
It is time to remove them. They are deprecated since 10.0.0
2022-07-02 16:33:32 +02:00
Veloman Yunkan e3e4bfa533 Support for serving customized resources
During work on the kiwix-serve front-end, the edit-save-test cycle is
a multistep procedure:

1. build and install libkiwix
2. build kiwix-tools
3. run kiwix-serve
4. reload the web-page in the browser

When making changes in static resources that are served by kiwix-serve
unmodified, the steps 1-3 can be eliminated if kiwix-serve is capable of
serving resources from the file-system. This commit adds such a
functionality to kiwix-serve. Now, if during startup of kiwix-serve the
environment variable `KIWIX_SERVE_CUSTOMIZED_RESOURCES` is defined it is
assumed to point to a file where every line has the following format:

URL MIMETYPE RESOURCE_FILE_PATH

When a request is received by kiwix-serve and its URL matches any of the
URLs read from the customized resource file, then the resource data is
read from the respective file RESOURCE_FILE_PATH and served with
mime-type MIMETYPE.

Though this feature was introduced in order to facilitate the
development of the iframe-based content viewer, it can also be useful to
users who would like to customize the kiwix-serve front-end on their own
(without re-building all of kiwix-serve).

There is some overlap with a feature of the kiwix-compile-resources
script that also allows to override resources. The differences are:

1. The new way of customizing front-end resources has all such resources
   listed in a text file and there is a single environment variable
   from which the path of that file is read. kiwix-compile-resources
   associates a separate environment variable with each resource.

2. The new way uses regular paths to identify a resource. The
   kiwix-compile-resources method encodes the resource path by replacing
   any non-alphanumeric characters (including the path separator) with
   underscores (so that the resulting resource identifier can be used
   to construct the name of the environment variable controlling that
   resource).

3. The new method allows adding new front-end resources. The old method
   only allows to modify existing resources.

4. The new method allows (actually requires) to specify the URL at which
   the overriden resource should be served (similarly, the MIME-type can/must
   be specified, too). The old method only allows to override the contents of
   a resource.

5. The new method only allows to override front-end resources that are
   served without any preprocessing by kiwix-serve at runtime. The old
   method allows to override template resources as well (note that
   internationalization/translation resources cannot be overriden using the
   old method, either).
2022-06-22 10:59:41 +02:00
Matthieu Gautier 7cb98f7f4e Make opensearch start parameter 1 indexed. 2022-06-03 15:46:41 +02:00
Matthieu Gautier cadd2a5cbb Make the HTTPErrorHtmlResponse not Html only. 2022-06-03 15:46:41 +02:00
Matthieu Gautier e51a5b9ebc Introduce `get_requested_format` helper 2022-06-03 15:46:41 +02:00
Matthieu Gautier 5d6b0ea96a Add searchdescription.xml endpoint 2022-06-03 15:46:41 +02:00
Matthieu Gautier e5df5e936f Render the search result using (opensearch/atom) xml format. 2022-06-03 15:46:41 +02:00
Matthieu Gautier a7651d0e9b Check early that provided bookIds are valid 2022-06-02 12:37:52 +02:00
Matthieu Gautier 3bca43344f Correctly url encode querystring
Fix tests with querystring needed url encoding
(pattern=jazz&books.query.title=Ray%20Charles)
2022-06-02 12:37:52 +02:00
Matthieu Gautier b857293cfd Build the bookSelection query string when we parse the query.
We have to reuse the query the user give us to generate the
pagination links.
At search result rendering step we don't have access to the query object.
The best place to know which arguments are used to select books
(and so which arguments to keep in the pagination links) is when we
parse the query to select books.

Fix tests (pagination links) with book selector other than "books.id="
(pattern=jazz&books.query.lang=eng)
2022-06-02 12:37:52 +02:00
Matthieu Gautier e2ab7fd62e Add some more testing.
Note that some tests are failing and will be fixed in next commits.
2022-06-02 12:37:52 +02:00
Matthieu Gautier 1514661c26 Protect search from multi threading race condition.
libzim's search is not thread safe (mainly because xapian is not).
So we must protect our search objects from multi thread calls.

The best way to do this is to associate a mutex to the `zim::Searcher`
and lock the searcher each time we access object derivated from the
searcher (search, results, iterator, ...)
2022-06-02 12:37:52 +02:00
Matthieu Gautier 0081b4d8e7 Make the limit of zim files per search configurable.
The default value is 0, which means no limit.
2022-06-02 12:37:52 +02:00
Matthieu Gautier b74910b2af Limit the number of zim in multizim fulltext search.
We are currently limiting to 5 but it will be changed in next commit.
2022-06-02 12:37:50 +02:00
Matthieu Gautier cf30233358 Prefix env variable name with `KIWIX_` 2022-06-02 12:23:43 +02:00
Matthieu Gautier f0065fdd6f Introduce Error exception to do i18n 2022-06-02 12:23:42 +02:00
Matthieu Gautier c72132054d Move i18n helper functions 2022-06-02 12:22:28 +02:00
Matthieu Gautier 077ceac5a5 Make the search_rendered handle multizim search.
This introduce a intermediate mustache object to store information
about the request made by the user.
2022-06-02 12:22:28 +02:00
Matthieu Gautier 39d0a56be8 Use selectBooks in handle_search 2022-06-02 12:22:28 +02:00
Matthieu Gautier 76d5fafb72 Introduce `selectBooks`
`selectBooks` allow us to parse a query in a "standard" way to get
the book(s) on which the user want to work.
2022-06-02 12:22:28 +02:00
Matthieu Gautier 4438106c2f Add a prefix in get_search_filter
The prefix will be used to parse a "query to select book" in different context.
For now we have only one context : selecting books for the catalog search.
But we will want to select books to do fulltext search on them
(will be done in later commit)
2022-06-02 12:22:28 +02:00
Matthieu Gautier 76ebfd7ea4 Move get_search_filter and subrange. 2022-06-02 12:22:27 +02:00
Matthieu Gautier 22996e4a6b Allow user to select multiple books when doing search. 2022-06-02 12:22:27 +02:00
Matthieu Gautier 854623618c Use the newly introduced searcherCache for multizim searcher. 2022-06-02 12:22:25 +02:00
Matthieu Gautier f5af0633ec Move the searcher cache into the Library 2022-05-24 14:55:48 +02:00
Matthieu Gautier 7c688a4acc Move `getCacheLength` to a generic helper function `getEnvVar` 2022-05-24 14:55:48 +02:00
Matthieu Gautier 66b2449800 Remove unnecessary catch
Catch of std::exception is already made in `handle_request`
2022-05-23 19:17:28 +02:00
Matthieu Gautier f0dd34b6db Introduce buildQueryData helper in SearchRenderer 2022-05-23 19:13:25 +02:00
Matthieu Gautier 4695f47dd2 Introduce operator+= to simplify response creation. 2022-05-23 10:56:39 +02:00
Veloman Yunkan 3b9f28b2b5 Applied cache-id to search_results.css
The story of search_results.css

static/skin/search_results.css was extracted from
static/templates/no_search_result.html before the latter was dropped.

static/templates/no_search_result.html in turn seems to be a copied and
edited version of static/templates/search_result.html.

In the context of exploratory work on the internationalization of
kiwix-serve (PR #679) I noticed duplication of inline CSS across those
two templates and intended to eliminated it. That goal was not fully
accomplished (static/templates/search_result.html remained untouched)
because by that time PR #679 grew too big and the efforts were diverted
into splitting it into smaller ones. Thus search_results.css slipped
into one of those small PRs, without making much sense because nothing
really justifies preserving custom CSS in the "Fulltext search unavailable"
error page.

At the same time, it served as the only case where a link to a cacheable
resource is generated in C++ code (rather than found in a template).
This poses certain problems to the handling of cache-ids. A workaround
is to expel the URL into a template so that it is processed by
`kiwix-resources`. This commit merely demonstrates that solution. But
whether it should be preserved (or rather the "Fulltext search
unavailable" page should be deprived of CSS) is questionable.
2022-05-02 20:37:22 +04:00
Matthieu Gautier 0d294c50a5 [SERVER] Support gzip encoding instead of deflate.
The `compress` function is copied from httplib
2022-04-27 18:23:38 +02:00
Veloman Yunkan 9987fbd488 Fixed CI build failure under android_arm* 2022-04-13 16:40:20 +02:00
Veloman Yunkan 6f3db20078 Internationalized "Fulltext search unavailable" page 2022-04-13 16:40:20 +02:00
Veloman Yunkan fbd23a8329 Fully internationalized 400, 404 & 500 error pages 2022-04-13 16:40:20 +02:00