Commit Graph

1359 Commits

Author SHA1 Message Date
Veloman Yunkan 5f27b4b651 Taking advantage of std::make_unique() 2023-11-29 21:32:16 +04:00
Veloman Yunkan 7a85c92025 Dropped root from HTTPErrorResponse & friends 2023-11-29 21:32:16 +04:00
Veloman Yunkan 6e2be481fd Dropped the root param from ItemResponse::build() 2023-11-29 21:32:16 +04:00
Veloman Yunkan db3b76247f Last step of removing root from ContentResponse 2023-11-29 21:32:16 +04:00
Veloman Yunkan 6a651e04e5 1st step in removing root from ContentResponse
It turned out that ContentResponse::m_root is no longer used.

At this point, the root parameter is dropped only from the 3-ary variant
of ContentResponse::build(), so that its all call sites are
automatically discovered by the compiler (and updated manually).
Including the other (4-ary) variant of ContentResponse::build() in this
change might result in the semantic change of expressions like
`ContentResponse::build(x, y, z)` and failure to update them.
2023-11-29 21:32:16 +04:00
Veloman Yunkan 22ea3106c5 Passing only root location instead of the entire server 2023-11-29 21:32:16 +04:00
Veloman Yunkan 2d132d701e Dropped the server param from Response::build*() 2023-11-29 21:32:16 +04:00
Veloman Yunkan f81a5a1a4b Moved verbosity control to Response::send()
It makes little sense to pass the verbosity control to the `Response`
constructor if it is used only in `Response::send()`.
2023-11-29 21:32:12 +04:00
Veloman Yunkan 3dce025f47 Deleted an unused function 2023-11-29 17:16:23 +04:00
Veloman Yunkan e470c97f74 Got rid of InvalidUrlMsg 2023-11-29 15:42:21 +04:00
Veloman Yunkan a7ea908bcd HTTPErrorResponse no longer accepts std::strings 2023-11-29 15:35:53 +04:00
Veloman Yunkan 41f25083da Replaced UrlNotFoundMsg with UrlNotFoundResponse 2023-11-29 14:31:38 +04:00
Veloman Yunkan 3188b0afe6 Translated a hard-coded error message 2023-11-29 14:18:06 +04:00
Veloman Yunkan 571c09e00a Control characters are escaped in suggestions JSON
According to the JSON spec, control characters from U+0000 through U+001F
must NOT appear in strings unescaped.
2023-11-17 14:55:01 +01:00
Matthieu Gautier 07ff4eab43 Do not index book's name as a phrase.
Fix #1004
2023-11-08 10:29:31 +01:00
Emmanuel Engelhart 733b027c2f
Add support of Fon language (no supported in libicu) 2023-11-04 15:34:42 +01:00
Matthieu Gautier 9166b67c47 Do not allow SearchRendered to work on a delete nameMapper/Library.
By moving the nameMapper/library arguments in `getHtml`/`getXml` we avoid
any potential "use after free" of name mapper or library as they are not
stored.
2023-10-05 16:37:22 +02:00
Matthieu Gautier 1dc9705597 Introduce LibraryPtr and ConstLibraryPtr.
As we enforce the use of Library through a shared_ptr, let's simplify
user life (and code) with new "type".
2023-10-05 16:37:22 +02:00
Matthieu Gautier 5292f06fff Move back `Library::Impl` in `Library`.
As we now always use Library through a shared_ptr, we can make `Library`
not copiable and not movable.
So we can move back the data in `Library`, we don't care about move
constructor.
2023-10-05 16:37:22 +02:00
Matthieu Gautier f8e7c3d476 Move the `Library` mutex in `Library::Impl`.
The why of this mutex is in `Library` is a bit complex.
It has been introduced in c2927ce when there was only `Library` and no
`std::unique_ptr<Impl>`.
As introducing the mutex imply implementing the move constructor, we have
split all data in `LibraryBase` (and keep a default move constructor here)
and add the mutex in `Library` (and implement a simple move constructor).

Later, in 090c2fd, we have move the `LibraryBase` to `Library::Impl`
(which should have been `Library::Data`).

So at the end, `Library::Impl` is never moved. We can move the `mutex` in
it and still simply implement move constructor for `Library`.
2023-10-05 16:37:22 +02:00
Matthieu Gautier ead1474ead Make `SearchRendered` taking a const pointer. 2023-10-05 16:37:22 +02:00
Matthieu Gautier 1316dec37c Make the `Server` keep a `shared_ptr` instead of a raw NameMapper pointer.
Same as for `Library`, we want to be sure that the `NameMapper`
actually exists when the server is using it.
2023-10-05 16:37:22 +02:00
Matthieu Gautier a5557eeb25 Make the `Server` keep a `shared_ptr` instead of a raw Library pointer.
We want to be sure that `Library` actually exists when we use it.
While it is not a silver bullet (user can still create a shared_ptr on
a raw pointer), making the `Server` keep `shared_ptr` on the library
help us a lot here.
2023-10-05 16:36:18 +02:00
Matthieu Gautier efcbf6ef1e Make the `UpdatableNameMapper` keep a `shared_ptr`.
Same as `Manager`, we want to be sure that `Library` actually exists
when we use it.
2023-09-25 16:31:55 +02:00
Matthieu Gautier 139b561253 Make the `Manager` keep a `shared_ptr` instead of a raw Library reference.
We want to be sure that `Library` actually exists when we modify it.
While it is not a silver bullet (user can still create a shared_ptr on
a raw pointer), making the `Manager` keep `shared_ptr` on the library
help us a lot here.
2023-09-25 16:30:56 +02:00
Matthieu Gautier 49e99e7c22 Remove dumpers from the public API.
All those dumper were not used by any of our other projects.
They are only used internally, either by `Library::writeToFile` or the
server.
2023-09-19 16:46:58 +02:00
Veloman Yunkan f8c3a1fd2e Added default user language to viewer_settings.js
The default user language determined from the value of "Accept-Language"
header is communicated to the client via the /viewer_settings.js
endpoint.
2023-09-09 19:37:49 +04:00
Veloman Yunkan b5b98e7a61 RIP userlang cookie
This commit drops the usage of the userlang cookie in the backend but
not in the frontend. UI language control should be broken at this point
and will be fixed in the next few commits.
2023-09-09 19:37:49 +04:00
Nikhil Tanwar c938101c70 Allow multiple category support
Created a generic function multipleQuery which takes:
1. string (representing a comma separated list)
2. param (the value to query on using Xapian).

Category and language query will use this function.
2023-07-26 18:15:45 +05:30
Nikhil Tanwar 385931f229 Move getLanguageSelfName to tools.h
This is a general utility which other ports can get use of.
Added tests
2023-07-26 16:02:32 +05:30
Nikhil Tanwar 94d6bef402 Introduce readCategoriesFromFeed()
Added a function to load categories stored in an OPDS stream
2023-07-26 16:02:32 +05:30
Nikhil Tanwar a28c2973e9 Introduce readLanguagesFromFeed()
Added a new function to read languages stored in an OPDS feed
2023-07-26 16:02:32 +05:30
Nikhil Tanwar 5c3a997de4 make beautifyFileSize public
This general function will be useful in other kiwix apps
2023-07-21 22:19:45 +05:30
Veloman Yunkan e9d9d85427 Added Gungbe language code 2023-07-10 23:28:04 +04:00
Veloman Yunkan f41007989b Fool-proof checking of book illustration presence
Now (in a library.xml flow) a book is considered to contain an
illustration only if both "faviconMimeType" and "favicon" attributes
are set to non-empty values.
2023-06-24 20:10:41 +04:00
Anthony J. Bentley df164aefe5 Include netinet/in.h everywhere except Windows
According to POSIX, sockaddr_in is declared in netinet/in.h.
Some POSIX systems (notably OpenBSD and FreeBSD) declare it in
only this header, so including it is required. Others, like Linux,
are are more lax in exposing symbols to the namespace, providing
sockaddr_in via additional headers, but it does no harm to include
the standard header on such systems.
2023-05-27 12:42:29 +03:00
Veloman Yunkan dc58e278c7 git mv src/server/internalServer_catalog{_v2,}.cpp 2023-04-25 12:48:49 +04:00
Veloman Yunkan 9994302312 Explicit charset in OPDS response MIME types 2023-04-25 12:48:29 +04:00
Veloman Yunkan 8c190cf34f Moved InternalServer::handle_catalog() 2023-04-25 12:48:10 +04:00
Veloman Yunkan 1273570e01 Deduplication of OPDS MIME type strings 2023-04-25 12:47:02 +04:00
Veloman Yunkan 6f0e55d603 A slight simplification of the mustache template
Got rid of the partial vs full entries logic in the mustache template -
now it is entirely contained in `OPDSDumper::dumpOPDSFeedV2()`.
2023-04-18 14:45:51 +04:00
Veloman Yunkan 9fd7f7da34 Really enable Chromium to display PDFs in the viewer iframe
The previous "fix" (merged PR #924) was buggy. It not only didn't work
in Chromium v90, but in more recent versions too.

I verified that this fix works in Firefox (v111) and Chromium (v90):

- Attempts by the ZIM content to break out of the viewer iframe are
blocked.
- PDFs are displayed in the viewer.
2023-04-10 16:42:21 +04:00
Veloman Yunkan a6659cbe96 Enable Chromium to display PDFs in the viewer iframe
This fix requires Chromium version above 90.
2023-04-05 15:08:29 +02:00
Nikhil Tanwar 951e15c665 No results display in /nojs
Shows a link to reset filter if there are no books.
2023-03-29 19:03:30 +05:30
Nikhil Tanwar cc35fe503f Translations for /nojs endpoint
Uses the string from #846 for translations.
A couple new translations are also added for <title> tag.
2023-03-29 19:03:29 +05:30
Nikhil Tanwar 37aadb86fb language/category filtering in /nojs endpoint
Adds language and category filter in /nojs.
Unlike the main page, the filtering is only done after user submits the form.
2023-03-29 19:02:58 +05:30
Nikhil Tanwar f843ea48f0 Add Results label
Shows "x results" label where x = number of books based on filters
2023-03-29 19:02:58 +05:30
Nikhil Tanwar a48e2e6f06 Add search form for /nojs endpoint
Adds an html form to search books by the q= parameter
2023-03-29 19:02:58 +05:30
Nikhil Tanwar 0f7e11bd86 Add download-links to tiles in /nojs
The download-link links to /nojs/download/<bookname> for all 4 types of downloads.
2023-03-29 19:02:56 +05:30
Nikhil Tanwar dbded6eee2 Add links to content for tiles in /nojs
If the tiles are now clicked, they redirect to main page of book.
2023-03-28 21:50:47 +05:30
Nikhil Tanwar c1d7cc37fd Add tags in tiles for /nojs endpoint
Adds span elements for tags
2023-03-28 21:49:31 +05:30
Nikhil Tanwar 6071b98fb7 Import book tiles
Tries to copy the same design of tiles as main page with javascript enabled
2023-03-28 21:49:31 +05:30
Nikhil Tanwar dca47d35f7 Introduce /nojs endpoint
Adds /nojs endpoint for fallback.
Currently, it serves an HTML with book names in library
2023-03-28 20:25:44 +05:30
Nikhil Tanwar d8656ec149 Introduce HTMLDumper
HTMLDumper class will be used to dump library in HTML format. It inherits from LibraryDumper
2023-03-28 20:25:44 +05:30
Nikhil Tanwar f1873876b2 Extract LibraryDumper from OPDSDumper
This change creates a new common class for dumping the library into various formats: LibraryDumper
2023-03-28 20:25:44 +05:30
Veloman Yunkan eb002ae306 Deprecated Book::getLanguage()
Introduced `Book::getCommaSeparatedLanguages()` instead.
2023-03-08 15:24:53 +01:00
Veloman Yunkan 2550306052 One more usage of Book::getLanguages()
`Book::getLanguages()` is used instead of `Book::getLanguage()` when
determining the set of languages for a collection of books.
2023-03-08 15:24:53 +01:00
Veloman Yunkan 51fcb90dc0 Library::updateBookDB() uses Book::getLanguages() 2023-03-08 15:24:53 +01:00
Veloman Yunkan b1ad319d52 Enter Book::getLanguages() 2023-03-08 15:24:53 +01:00
Veloman Yunkan 5bda7fd45c Support for multilang ZIMs 2023-03-08 15:24:53 +01:00
Veloman Yunkan ac742e9da2 Redirection of slashless root URL
With non-empty root location, the canonic form of the root URL for a
kiwix server is now required to end with a slash (to match the situation
for an empty root location). This requirement enables usage of relative
URLs on the welcome page and resources/scripts loaded through that page.

A slashless root URL is redirected to the slashful version.
2023-02-22 17:54:20 +04:00
Veloman Yunkan 2e0124710a `?count=0` OPDS catalog queries return 0 results
... which is a useful way of finding out the total number of results
with the least consumption of resources.
2023-02-10 19:15:29 +01:00
Veloman Yunkan e8c8a297b5 Registered MIME-types for .ico and .json
As a result, favicon.ico stopped being considered a compressible resource.
2023-02-10 15:07:00 +01:00
Nikhil Tanwar 12f0614350 Add Serbo-croate language name
Adds "srpskohrvatski" as name for "hbs" language tag.
2023-02-10 09:20:23 +05:30
Veloman Yunkan c2fffacbbd Renamed a data member 2023-02-09 10:40:23 +01:00
Veloman Yunkan 02f631fdb6 Got rid of RequestContext::full_url 2023-02-09 10:40:23 +01:00
Veloman Yunkan 05a66ead6e URI-encoding of the root location part
Now the root location is URI-encoded too.

In order to properly test this change the root location in the tests was
changed from "/ROOT" to "/ROOT#?" (or "/ROOT%23%3F" in URI-encoded form),
which is why this commit is so big.
2023-02-09 10:40:07 +01:00
Veloman Yunkan 97f0314fe6 Saving a few CPU cycles
This silly optimization in fact helps to avoid a somewhat more serious
waste of CPU cycles that would otherwise result in the next commit.
2023-02-08 22:16:27 +01:00
Veloman Yunkan a7fe4193e3 Preparing to save a few CPU cycles 2023-02-08 22:16:27 +01:00
Veloman Yunkan 2c5e84b6b3 Simpler fullURL2LocalURL() 2023-02-08 22:16:27 +01:00
Veloman Yunkan 71a66e0528 Passing of unrooted URL into RequestContext()
This change doesn't make much sense on its own - the real goal is to
prepare some ground for easier implementation of URI-encoding of the root
location.
2023-02-08 22:16:27 +01:00
Veloman Yunkan a807ce27f1 URI-encoding when redirecting legacy URLs to /content
Testing of this functionality revealed that the query part containing +
symbols (as replacement for spaces in the parameter values) isn't
forwarded properly as the + symbols are URI-encoded (this is a bug on
the part of the `RequestContext::get_query()` the result of which
already contains URI-encoded +'s).
2023-02-08 22:16:27 +01:00
Veloman Yunkan 2e9bec95b0 Proper URI-encoding in InternalServer::build_redirect()
- Before this change `InternalServer::build_redirect()` only URI-encoded the
  article path, ignoring the book name and/or the root location components of
  the URL.

- In order to be able to test this fix, corner_cases.zim was renamed to
  contain a couple of special URL symbols in its filename. The
  `create_corner_cases_zim_file` script was updated accordingly.
2023-02-08 22:16:09 +01:00
Matthieu Gautier 1ba588272c Get `Waiting` downloads before `Active` ones.
`Waiting` can become `Active` while we are getting the downloads.
We may have rare case where we miss a download if we get `Active` before
`Waiting`.
2023-02-08 15:42:17 +01:00
Matthieu Gautier 2c3b7409aa Remove the default value of follow parameter in `updateStatus`.
`false` is a pretty bad default value as most user want to track
the real download.

By removing the default value, we force user to make a choice.
We could have change the default value to true but it would have been
a silent API change and we don't want that.
2023-02-08 15:42:17 +01:00
Matthieu Gautier 18b7b5f277 Mark constant methods as const. 2023-02-08 15:42:17 +01:00
Matthieu Gautier 0e612de4d1 Make `Downloader` return shared_ptr instead of raw pointer.
This is dangerous by nature to return raw pointer on internal data.
2023-02-08 15:42:17 +01:00
Matthieu Gautier 52ae5c3a5f Make Downloader thread safe. 2023-02-08 15:42:17 +01:00
Matthieu Gautier d1fe1b89ae Do not automatically update the status of existing Download.
User may already have a pointer to the `Download` and it is not protected
against concurrent access.

We could update the status of new created `Download` as by definition,
no one have a pointer on it.
But it better to not do it neither :
- For consistency
- Because the first call on update status may be long on windows (because
  of file preallocation). It is better to not block the downloader for that.
2023-02-08 15:42:17 +01:00
Matthieu Gautier 1aa8521e15 Remove the lock.
As we now build a new request handle for every request, we don't need
a lock.

libcurl itself is thread safe as long as we don't share a handle.
2023-02-08 15:42:17 +01:00
Matthieu Gautier 95ebb6a492 Build a new curl "handle" at everyrequest instead of reusing the same one. 2023-02-08 15:42:17 +01:00
Veloman Yunkan ca079a72cc Some clean-up 2023-01-25 19:15:12 +04:00
Veloman Yunkan 471c5b89f4 Dropped the 2nd param of urlEncode()
`urlEncode(str)` is now equivalent to the previous `urlEncode(str, true)`.
2023-01-25 19:15:12 +04:00
Veloman Yunkan 3bf8211b70 Made 2nd param of urlEncode() mandatory
This is a precautionary step before dropping the said parameter.
2023-01-25 19:15:12 +04:00
Veloman Yunkan ec81d5904d Proper URI-encoding in kiwix::getSearchUrl() 2023-01-25 19:15:12 +04:00
Veloman Yunkan 63e0d5c7c2 RequestContext::get_query() is fully URI-encoded 2023-01-25 19:15:12 +04:00
Veloman Yunkan 772243e832 Category name is fully URI-encoded 2023-01-25 19:15:12 +04:00
Veloman Yunkan bad13d76b4 Removed unused code 2023-01-25 19:15:12 +04:00
Veloman Yunkan 0bde4d9412 Properly URI-encoded links in search results
Special URI symbols occurring in the item path part of the search result
link were NOT encoded, because that would also encode the path separator (/)
symbol. Now that `urlEncode()` never encodes the / symbol, it is safe to
encode all other URI-special symbols in the path.
2023-01-25 19:15:12 +04:00
Veloman Yunkan 239b108fa7 / is no longer a reserved char for urlEncode()
This change is a quick hack solving known issues with URI-encoding in
libkiwix.

This change removes the slash character from the list of URL separator
symbols in URL encoding/decoding utilities, and makes it a symbol that
is safe to leave unencoded.

Effects:

- `urlEncode()` never encodes the '/' symbol (even when it is requested
  to encode the URL separator symbols too).

- `urlDecode(str)`/`urlDecode(..., false)` will now decode %2F to '/';
  other encoded URL separator symbols are NOT decoded when the second
  argument of `urlDecode()` is set to false (which is the default).
2023-01-25 19:15:12 +04:00
Veloman Yunkan c5ccbd37e2 Extracted isHarmlessUriChar() 2023-01-25 19:15:12 +04:00
Veloman Yunkan aa2e443eb8 Fixed indentation
Replaced tabs with spaces.
2023-01-25 19:15:11 +04:00
Veloman Yunkan 82d477009d '#' is a URI delimiter symbol 2023-01-25 19:15:11 +04:00
Veloman Yunkan e49081da80 Fixed urlEncode() for chars below 0x10 2023-01-25 19:15:11 +04:00
Veloman Yunkan e35e7585e0 Server sets userlang cookie as global and permanent
Without specifying the "Path" attribute of the cookie in the "Set-Cookie" header
we end up with multiple instances of the cookie for different URLs. We
want a single "global" cookie for kiwix-serve. Besides we want it to be
"permanent" rather than a session cookie, hence the large (1-year-long)
TTL value for the "Max-Age" attribute.
2023-01-24 19:01:32 +01:00
Veloman Yunkan fcb97c3c06 Sparing use of "Set-Cookie: userlang=..." header
Server adds the "Set-Cookie: userlang=..." header to the response only
if the "userlang" cookie is not already present with the same value.
2023-01-24 19:01:32 +01:00
Veloman Yunkan 8eb527389e URI-encoding of redirections to URLs with special symbols 2023-01-10 17:41:59 +04:00
Veloman Yunkan 28e9fb48b6 Properly implemented parseUserLanguagePreferences() 2022-12-14 15:34:46 +01:00
Veloman Yunkan 634f3fcf14 Properly implemented selectMostSuitableLanguage() 2022-12-14 15:34:46 +01:00
Veloman Yunkan 88597e1834 Enter selectMostSuitableLanguage() 2022-12-14 15:34:46 +01:00