smallsolar
/
libkiwix
mirror of https://github.com/kiwix/libkiwix.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,471 Commits 23 Branches 58 Tags 113 MiB
Commit Graph

851 Commits

Author SHA1 Message Date
Matthieu Gautier ba302bed33 Use new libzim method `getFaviconEntry` to get the favicon. 2020-12-09 14:25:02 +01:00
Steve Wills 6900b4e506 fix build on FreeBSD 
With this header, sockaddr_in and INADDR_ANY are not defined
 2020-12-07 09:38:46 -05:00
Matthieu Gautier 1a5a2e7a8e Adapt kiwix-lib to the new libzim api. 2020-12-02 12:16:48 +01:00
Matthieu Gautier d87079ec13 Remove deprecated method in the reader. 2020-11-24 19:00:52 +01:00
Veloman Yunkan 0f8fe1f63f Alternative implementation of parseMimetypeCounter() 2020-10-29 14:11:27 +04:00
Matthieu Gautier 08464f23bc Better parsing of `M/Counter` 
Mimetype may contain a parameters.
Then, the mimetype would be something like "text/html;foo=bar;foz=baz"

It will contains a `;` and `=` and it conflicts with the same operators
we use to separate the items in our list.

We have to use a more advanced algorithm which takes the context into
account.

Fix #416
 2020-10-28 16:03:18 +01:00
Matthieu Gautier ef42abea4b Add some tests of `parseMimetypeCounter` 2020-10-28 14:44:23 +01:00
Matthieu Gautier 4407dd12bd Move mimetypeCounter parsing in its own function. 2020-10-28 14:08:06 +01:00
Matthieu Gautier 632583ede2 Add missing include 2020-10-07 18:43:57 +02:00
Matthieu Gautier 61f9d4ab3a Stop the internal server only if it exists. 2020-10-07 14:36:45 +02:00
Matthieu Gautier 470bfc3f1f Better variable name for outStream. 2020-08-28 15:27:03 +02:00
Matthieu Gautier ea3180cb8c Better error printing. 2020-08-28 15:27:03 +02:00
Matthieu Gautier 72d3f8f8e2 Fix segmentation fault with curl requests. 
Use a heap allocated buffer (with lifetime of Aria2 class) instead of
a stack allocated one.

Original fix made by @ZaWertun. Kudos to him.

Fix #kiwix/kiwix-desktop#123, kiwix/kiwix-desktop#513
and kiwix/kiwix-desktop#423
 2020-08-26 12:42:16 +02:00
Matthieu Gautier af9e03904c Use std::mutex and std::unique_lock instead of pthread mutex/lock. 
It simplify a bit the code and ensure that mutex is correctly unlock
even in case of exception.
 2020-08-26 12:30:56 +02:00
Matthieu Gautier 39611cbd60 Wait for waitingThread to exit before destroying the subprocess memory. 
WaitingThread read some shared memory with the SubProcess
(`mutex`, `m_running`).
When we destroy the SubProcess, we must be sure that WaitingThread has
correctly finished else we may have invalid read/write on freed memory.
 2020-08-26 12:26:04 +02:00
Matthieu Gautier 6f0d3003ac Remove `m_compress` member. 2020-08-13 11:16:41 +02:00
Matthieu Gautier ee17b0739a Fix compilation on CI native dyn. 
On the CI, the native_dyn docker image is setup with a packaged version
on libmicrohttpd for which `MHD_HTTP_RANGE_NOT_SATISFIABLE` is not
defined.

When the CI will be fixed, we can revert this commit.
 2020-08-13 11:16:41 +02:00
Matthieu Gautier 47436f7bdd Move some header setting in response's constructors. 
It make easier to understand what is somehow constant and what depends
of the context.
 2020-08-13 11:16:41 +02:00
Matthieu Gautier 3352c95314 Remove the `RedirectResponse` and use a basic `Response` with header. 2020-08-13 11:16:41 +02:00
Matthieu Gautier 77123ac74c Move the adding of 304 headers in 304 factory. 
This avoid us to create a ContentResponse just to have some correct
headers.
 2020-08-13 11:16:41 +02:00
Matthieu Gautier 9078f0ac6e Remove `ResponseMode`. 2020-08-13 11:16:41 +02:00
Matthieu Gautier 8d6567d067 Create a utility builder for 416 response. 
Also add a map in the response to store specific headers.
 2020-08-13 11:16:41 +02:00
Matthieu Gautier 6d5cddca12 Fix android compilation 
Android clang complains about the fact it cannot move the
`std::unique_ptr<ContentResponse>` into a `std::unique_ptr<Response>&&`
(for the implicit `std::unique_ptr<Response>` constructor).
Let's help him a bit.
 2020-08-13 11:16:41 +02:00
Matthieu Gautier a3939e9a05 Move all the content code in the ContentResponse. 2020-08-13 11:16:41 +02:00
Matthieu Gautier eee621d15b Move small utilities method to create response in Response class. 2020-08-13 11:16:41 +02:00
Matthieu Gautier 7b2ee37437 Move the entry response to its own class. 2020-08-13 11:16:41 +02:00
Matthieu Gautier f014fb2895 Introduce a ContentResponse. 
This is only an "interface" for now as other type of response (entry) may
be "transformed" to a ContentResponse.
We cannot move all the code in the class.
 2020-08-13 11:16:41 +02:00
Matthieu Gautier 1011d1ff0b Move the redirection response in its own class. 
The redirection is the easiest to move, let's start with this one.
 2020-08-13 11:16:41 +02:00
Matthieu Gautier 9e351b279e Remove `get_default_response` in favor of a static Response method. 
We want to build different kind of response depending of the context.
 2020-08-13 11:16:41 +02:00
Matthieu Gautier a0bdc0821c Move internalServer code into its own source files. 2020-08-13 11:16:41 +02:00
Matthieu Gautier a819d9e3e0 Make the server handle pointer to response instead of plain response. 
This is a preparatory work.
We will specialize the response and so we need a pointer to response
instead of plain response.
 2020-08-13 11:16:41 +02:00
manan jethwani c74b935a9b added pageLength for search_pagination 2020-08-12 02:08:02 +05:30
Matthieu Gautier a55d504017 Fix getArticleCount. 
With #403, the article mimetype may be different than "text/html".
It can also be "text/html; raw=true".
(And in fact it already could have any kind of optional argument).
 2020-08-11 18:27:54 +02:00
Matthieu Gautier 87b5adcaf4 Make the response responsible to detect if we must introduce taskbar. 
The response detect if taskbar must be added depending of the mimetype.

Now, `set_taskbar` can be call unconditionally
(no need to check for the mimetype)

And we don't need to call set_taskbar if we have no information to set.
 2020-08-11 18:27:54 +02:00
Veloman Yunkan c4e6313c90 x in a --> a.contains(x) in meson.build files 2020-08-11 18:17:18 +02:00
renaud gaudin 3f25a3d005 Fixed #391: prevent taskbar and blocker at article level 
Some HTML articles are meant to be displayed through a viewer. In this case,
we know we don't want the server to inject the taskbar nor the link blocker
because the content is not a user-ready web page but a partial element of it.

Such articles still need to be `text/html` to be parsed properly by browsers.

This changes the way we decide to display the tasbar or not.
Previously, we were adding it to every article with a MIME __starting with__ `text/html`.
Now, we're additionally preventing it on `text/html` MIME if there is a `;raw=true` string inside.

This leaves articles with MIME `text/html;raw=true` (warc2zim convention) outside
of the taskbar target.

For similar reasons, the external-link blocker is set to apply to the same set of articles.
Previously, it was applied to all articles which was an (unoticable) mistake.
 2020-08-07 09:26:24 +02:00
MananJethwani 599aaa4c1b added code for status code 204 for empty return of search. 2020-08-01 01:45:42 +05:30
Veloman Yunkan 3d425f44de Request header case is ignored 
Originally reported against case sensitivity of the Range header
(see issue #387), this fix applies to all request headers (since
according to RFC 7230 all header fields are case-insensitive, see
https://tools.ietf.org/html/rfc7230#section-3.2). However, a
corresponding unit-test was added only for the Range header.
 2020-07-30 16:01:51 +02:00
Matthieu Gautier 7ece383004 Add support for samba path on windows. 
Fix kiwix/kiwix-desktop#429
 2020-07-15 11:40:02 +02:00
Kelson cf8e8b94eb Fix compilation with libmicrohttpd v0.97.1 2020-07-08 14:42:46 +02:00
Matthieu Gautier 4d307e18eb Add new thread safe suggestion API. 
Previous API were using an internal vector to store the suggestions search
results.

The new API takes a vector as out argument. So user can call the functions
without having to protect the search.

We should change the android API to reflect the change but it is a bit
more complex to do at JNI level. As android do not call it multithreaded
we are safe for now. And we need the new API asap for kiwix-desktop.

So we keep the same API on android for now, the new api will be made
in next version.
 2020-07-01 17:16:13 +02:00
Kunal Mehta fb79cde729 Pass -latomic for architectures that need it 
Some architectures, specifically armel, mipsel, m68k & powerpc in
Debian, need to explicitly link to atomic.

Use meson to see if the target's CPU family is one of those, and if so,
pass -latomic to the linker.

Tested on armel and mipsel machines to verify passing -latomic works, and
on armhf and amd64 to ensure normal builds aren't broken.

Fixes #371.
 2020-06-29 00:18:13 -07:00
Matthieu Gautier ff605873ed Include missing `algorithm` header. 
`min` and `max` functions are defined here.
 2020-06-10 15:27:51 +02:00
Veloman Yunkan 05ef5d5f51
Assertion in ByteRange allows 0-sized content 
The assertion in the ByteRange constructor was written under the assumption that the content must have non-zero size. Now it allows that corner case.
 2020-06-02 21:53:47 +04:00
Veloman Yunkan f52b220d01 Dropped RequestContext::has_range() 2020-05-26 14:10:26 +04:00
Veloman Yunkan 50a850f3a9 Fixed a comment 2020-05-26 14:04:18 +04:00
Veloman Yunkan 886ae17274 Fixed a CodeFactor issue 2020-05-26 13:59:47 +04:00
Veloman Yunkan 85d6daabac Rolled back minor unneeded changes 2020-05-26 13:10:50 +04:00
Veloman Yunkan 5f1918d005 Split a long line 2020-05-26 13:04:03 +04:00
Veloman Yunkan 16bd79fa1b Final clean-up of byte_range.{h,cpp} 2020-05-26 12:50:08 +04:00
Veloman Yunkan c2ebdefe8d Handling of unsatisfiable ranges 2020-05-26 02:11:26 +04:00
Veloman Yunkan 37032892a4 Fixed compilation error under win32_* 
ERROR is a macro under Windows
 2020-05-26 01:58:17 +04:00
Veloman Yunkan 6b43438b74 Fixed compilation error under native_dyn 
MHD_HTTP_RANGE_NOT_SATISFIABLE is not defined in the older version of
libmicrohttpd (that is used under CI/Linux native_dyn).
 2020-05-26 01:54:36 +04:00
Veloman Yunkan 7301bf89bb Some refactoring of byte-range parsing 2020-05-26 01:50:29 +04:00
Veloman Yunkan ff23b28e7c Removed unnecessary qualifier 2020-05-26 01:41:37 +04:00
Veloman Yunkan 931e95f391 Invalid byte ranges result in 416 responses 2020-05-26 01:40:07 +04:00
Veloman Yunkan f7571b5b69 Content-Range header is set only for partial content 2020-05-25 17:42:18 +04:00
Veloman Yunkan 801ad18a89 ByteRange::resolve() 2020-05-25 17:27:35 +04:00
Veloman Yunkan 67a347c0c4 Moved byte-range parsing to byte_range.cpp 2020-05-25 17:21:10 +04:00
Veloman Yunkan 693905eb68 Default constructed ByteRange is a full range 2020-05-25 17:17:56 +04:00
Veloman Yunkan f3e79c6b4c Introduced src/server/byte_range.cpp 2020-05-25 16:43:44 +04:00
Veloman Yunkan 52f207eaa6 Support for single-ended byte ranges 2020-05-25 16:37:01 +04:00
Veloman Yunkan 67294217a8 ByteRange::Kind 2020-05-25 16:23:44 +04:00
Veloman Yunkan d111a40ce8 Response::m_byteRange 2020-05-23 20:35:22 +04:00
Veloman Yunkan 0c5bb3fcfe Moved ByteRange to a header file of its own 2020-05-23 20:08:53 +04:00
Veloman Yunkan 3fba8c20a0 Converted RequestContext::ByteRange to a class 
Also renamed the `range_pair` data member of `RequestContext` to `byteRange_`
 2020-05-23 19:59:47 +04:00
Veloman Yunkan 54db6049b7 Byte-range parsing not exposed in the header file 2020-05-23 18:58:19 +04:00
Veloman Yunkan 81c38d6b2b parse_byte_range() without side-effects 2020-05-23 18:53:16 +04:00
Veloman Yunkan e6a86c02ae Got rid of RequestContext::accept_range 2020-05-23 17:15:42 +04:00
Veloman Yunkan a0f7f32570 Re-ordered function definitions 2020-05-23 17:11:26 +04:00
Veloman Yunkan c39fce8839 RequestContext::parse_byte_range() 2020-05-23 17:09:51 +04:00
Veloman Yunkan de37489c53 Range header starts with a unit spec 
After this commit valid ranges of the form "bytes=firstbyte-lastbyte" should
be handled correctly.
 2020-05-22 17:17:31 +04:00
Veloman Yunkan 2a35a86de6 Fixed the size value used creating a response 
In case of a partial response the size of the response is different
from the served entry size.
 2020-05-22 16:49:35 +04:00
Veloman Yunkan 0a30a77c08 Handling of out of bound byte ranges 2020-05-22 16:46:38 +04:00
Veloman Yunkan 1a99bacfe3 Byte ranges are inclusive 
The second component of a byte range, if present, designates the
index of the last byte to be included in the partial response.
 2020-05-22 16:30:43 +04:00
Kelson 94c2ab4395 Add two OPDS related mime-types to compress for HTTP 2020-05-18 08:19:51 +02:00
Kelson 0f07cab920 Small HTTP header beautification 2020-05-17 20:19:19 +02:00
Veloman Yunkan 5f0a9d0b08 Added a comment clarifying a non-obvious case 2020-05-15 15:17:04 +04:00
Veloman Yunkan 54f5dbbd35 Handling of If-None-Match conditional requests 2020-05-14 17:01:22 +04:00
Veloman Yunkan 95a5cde359 ETags are set in the response as needed 
Also added server-unit tests related to ETags in the response.
 2020-05-14 17:01:22 +04:00
Veloman Yunkan 3d08ef43f2 HEAD request is not rejected 
libmicrohttpd handles HEAD requests by dropping the body of the response
(if any). Hence letting a HEAD request through into the code that
processes GET requests is safe.

Also added server unit-tests related to the handling of HEAD requests.
 2020-05-14 17:01:22 +04:00
Veloman Yunkan bfa51c2d87 Refactoring: got rid of duplicate get_mime_type() 2020-04-29 18:33:25 +04:00
Veloman Yunkan 81e781133d Refactoring: utilized is_compressible_mime_type() 2020-04-29 18:33:01 +04:00
Veloman Yunkan 9ec7757efe Refactoring: smart Response::set_entry() 
Response::set_entry() was upgraded from a simple setter to a method
performing certain business logic that was previously taken care of by
InternalServer::handle_content().
 2020-04-29 18:22:15 +04:00
Veloman Yunkan 7bd7ec4937 Refactoring: preparing to move some code 2020-04-29 18:22:15 +04:00
Veloman Yunkan 14d8583c83 Refactoring in InternalServer::handle_content() 
Deduplicated common code found in the two branches of the last
if(){}else{} statement in InternalServer::handle_content().
 2020-04-29 18:22:15 +04:00
Veloman Yunkan a004d96cd7 Refactoring: extracted get_range_len() 2020-04-29 18:22:15 +04:00
Veloman Yunkan 21c6de2f80 Refactoring: split Response::create_mhd_response() 
The changes are easier to understand in ignore-white-space mode
(git diff -w, git show -w).
 2020-04-29 18:22:15 +04:00
Veloman Yunkan a8e78f27e1 Refactoring: extracted Response::create_mhd_response() 2020-04-29 18:22:15 +04:00
Veloman Yunkan 6c7ab6ff54 Refactoring: moved local variable declarations 2020-04-29 18:21:40 +04:00
Veloman Yunkan 659ee6ba71 Refactoring: extracted InternalServer::build_redirect() 2020-04-29 16:08:10 +04:00
Veloman Yunkan 83ee8dec15 Made InternalServer::get_default_response() const 2020-04-29 16:08:10 +04:00
Veloman Yunkan 87cbbed9e3 Refactoring: extracted is_compressible_mime_type() 2020-04-29 16:08:10 +04:00
Veloman Yunkan a058520628 Refactoring: extracted get_mime_type() 2020-04-29 16:08:10 +04:00
Veloman Yunkan 1ef5ebfb52 Refactoring: extracted InternalServer::get_reader() 2020-04-29 16:08:10 +04:00
Veloman Yunkan bbc06931ad Refactoring: extracted get_book_name() 2020-04-29 16:08:10 +04:00
Veloman Yunkan 2d3bf9b981 Refactoring: extracted InternalServer::homepage_data() 
Also typedef'ed kainjow::mustache::data as MustacheData
 2020-04-29 16:08:10 +04:00
Veloman Yunkan fd80f2a89f Refactoring: extracted fullURL2LocalURL() 
Also dropped RequestContex::valid_url
 2020-04-29 16:08:10 +04:00
Veloman Yunkan abb3dec700 Refactoring: extracted str2RequestMethod() 2020-04-29 16:08:10 +04:00
luddens 0586ef6d41 fix open external zim 
Check if the parameter `pathToSave` is empty before use it otherwise the
book path is empty too, which causes crash on opening external zim files
 2020-04-20 15:22:36 +02:00