Commit Graph

1339 Commits

Author SHA1 Message Date
Manan Jethwani 93264f7409 added root functionality for block external link feature 2021-03-23 03:17:14 +05:30
Veloman Yunkan e214efecd4 Language code conversion via ICU
Language code is converted from ISO 639-3 to ISO 639 (which is
understood by Xapian) via ICU. The previous approach via an explicit
map had its advantages since Xapian has more than one stemmer
implementations for some languages (selectable via Xapian-specific
identifiers). This commit relies on the defaults associated with the
ISO 639 language codes.
2021-03-17 14:32:03 +01:00
Veloman Yunkan 09233bf4f3 Support for partial queries in catalog search
The search text in the catalog query is interpreted as partial by
default, but partial query mode can be disabled in C++. The latter
possibility is not exposed via the /catalog/search kiwix-serve endpoint,
though.
2021-03-17 14:32:03 +01:00
Veloman Yunkan a599fb3892 Initial version of Xapian-based catalog search 2021-03-17 14:32:03 +01:00
Veloman Yunkan a17fc0ef2d Library::getBooksByTitleOrDescription() 2021-03-17 14:32:03 +01:00
Veloman Yunkan db06b2c7ca Library::BookIdCollection typedef 2021-03-17 14:32:03 +01:00
Veloman Yunkan a20f9e2ce1 Library::filter() works in two stages
1. Get the subset of books matching the q (title/description) parameter
   of the search

2. Filter out books not matching the other parameters of the search.

Stage 1. currently works in the old way, but will be replaced by Xapian
based search in subsequent commits.
2021-03-17 14:32:03 +01:00
Veloman Yunkan b7b0bdbdd8 Both Book::update() methods update the category 2021-03-17 14:10:57 +04:00
Veloman Yunkan 4abc4f8518 Support for book category attribute in library.xml 2021-03-17 14:10:57 +04:00
Veloman Yunkan 6b2067c236 Reading category element from OPDS stream 2021-03-17 14:10:57 +04:00
Veloman Yunkan e55bf514e8 Dedicated 'category' parameter in catalog search 2021-03-17 14:10:57 +04:00
Veloman Yunkan 80d4f7e349 Extracted InternalServer::search_catalog() 2021-03-17 14:10:57 +04:00
Veloman Yunkan 58186ffb26 kiwix::Book::getCategory() 2021-03-17 14:09:48 +04:00
Veloman Yunkan ae32ff40c0 Dropped an extra colon from book <updated> dates 2021-03-17 14:02:27 +04:00
Veloman Yunkan 26331b401e Fixed the month in OPDS feed <updated> date
`tm::tm_mon` varies in the [0, 11] range.
2021-03-17 14:02:27 +04:00
Matthieu Gautier 67caae6c32 Use the new libzim's getRandomEntry instead of implementing it ourselves. 2021-03-02 14:16:09 +01:00
Veloman Yunkan 839fc10a4f Fixed the Windows build
Opening ZIM archives by file descriptor (as well as embedded
ZIM archives) is not supported under Windows.
2021-02-10 14:19:47 +01:00
Veloman Yunkan 5a8b825c70 Testing of JNIKiwixReader.getDirectAccessInformation() 2021-02-10 14:19:47 +01:00
Veloman Yunkan 7a465e66d7 Renamed org.kiwix.kiwixlib.{Pair->DirectAccessInfo} 2021-02-10 14:19:47 +01:00
Veloman Yunkan 5a99634dfd Java wrapper test checks favicon.png too 2021-02-10 14:19:47 +01:00
Veloman Yunkan e028bcbb04 Android's java.io.FileDescriptor is different 2021-02-10 14:19:47 +01:00
Veloman Yunkan 9cdf7a44c0 JNIKiwixReader can open an embedded ZIM archive 2021-02-10 14:19:47 +01:00
Veloman Yunkan 4d23e44de7 JNIKiwixReader ctor taking a file descriptor
... and a corresponding unit test
2021-02-10 14:19:47 +01:00
Veloman Yunkan 98d69ef59b Added testReader unit-test for the java wrapper 2021-02-10 14:19:47 +01:00
Veloman Yunkan e40827fbac Renamed the java wrapper unit test runner script 2021-02-10 14:19:47 +01:00
Veloman Yunkan a798e0c0a1 Made the java wrapper unit test run & pass
The kiwixlib java wrapper unit test can be run manually via the
src/wrapper/java/org/kiwix/testing/compile_test.sh script.

The test ZIM files in src/wrapper/java/org/kiwix/testing were created
using the create_test_zimfiles. They must be updated/re-generated and
committed in git whenever their source data or the create_test_zimfiles
script changes. Note: small.zim.embedded is not used at this point, it
was created for testing the enhancement coming in a few commits.
2021-02-10 14:19:47 +01:00
Matthieu Gautier 24b2e6e585 Remove unnecessary include. 2021-01-26 17:53:25 +01:00
Matthieu Gautier 3fd1310008 Use c++11 std::thread instead of pthread. 2021-01-26 17:53:25 +01:00
Matthieu Gautier 4749656828 Do not crash if zim file has no `Counter` metadata. 2021-01-26 15:15:27 +01:00
Emmanuel Engelhart 84895c4036 Better </head> detection regex 2021-01-18 13:16:56 +01:00
Emmanuel Engelhart a8bf9dd5b4 Better Kiwix Serve Taskbar insertion (after charset definition) 2021-01-18 11:18:53 +01:00
Emmanuel Engelhart a61c94ef10 Add GPLv3 header 2021-01-18 10:54:33 +01:00
Emmanuel Engelhart 8c43fd8d36 Fix taskbar insertion in case of '<head>' attributes 2021-01-11 14:37:19 +01:00
Emmanuel Engelhart 3e2810dff4 Support 'video/*' * 'audio/*' mimetypes in getMediaCount() 2021-01-07 12:32:32 +01:00
Emmanuel Engelhart 44c4aa931a Better use kiwix::startsWith() 2021-01-03 15:17:03 +01:00
Emmanuel Engelhart 95b32b168d More robust getMediaCount() 2021-01-01 17:05:32 +01:00
Matthieu Gautier 1002c15e0d Remove unnecessary checks.
`Reader` cannot be created with a null `zimArchive`.
We don't have to check for zimArchive being not null.
2020-12-09 14:25:02 +01:00
Matthieu Gautier d51000c4a9 Use new libzim method `hasFulltextIndex` to check for fulltext index. 2020-12-09 14:25:02 +01:00
Matthieu Gautier ba302bed33 Use new libzim method `getFaviconEntry` to get the favicon. 2020-12-09 14:25:02 +01:00
Steve Wills 6900b4e506 fix build on FreeBSD
With this header, sockaddr_in and INADDR_ANY are not defined
2020-12-07 09:38:46 -05:00
Matthieu Gautier 1a5a2e7a8e Adapt kiwix-lib to the new libzim api. 2020-12-02 12:16:48 +01:00
Matthieu Gautier d87079ec13 Remove deprecated method in the reader. 2020-11-24 19:00:52 +01:00
Veloman Yunkan 0f8fe1f63f Alternative implementation of parseMimetypeCounter() 2020-10-29 14:11:27 +04:00
Matthieu Gautier 08464f23bc Better parsing of `M/Counter`
Mimetype may contain a parameters.
Then, the mimetype would be something like "text/html;foo=bar;foz=baz"

It will contains a `;` and `=` and it conflicts with the same operators
we use to separate the items in our list.

We have to use a more advanced algorithm which takes the context into
account.

Fix #416
2020-10-28 16:03:18 +01:00
Matthieu Gautier ef42abea4b Add some tests of `parseMimetypeCounter` 2020-10-28 14:44:23 +01:00
Matthieu Gautier 4407dd12bd Move mimetypeCounter parsing in its own function. 2020-10-28 14:08:06 +01:00
Matthieu Gautier 632583ede2 Add missing include 2020-10-07 18:43:57 +02:00
Matthieu Gautier 61f9d4ab3a Stop the internal server only if it exists. 2020-10-07 14:36:45 +02:00
Matthieu Gautier 470bfc3f1f Better variable name for outStream. 2020-08-28 15:27:03 +02:00
Matthieu Gautier ea3180cb8c Better error printing. 2020-08-28 15:27:03 +02:00
Matthieu Gautier 72d3f8f8e2 Fix segmentation fault with curl requests.
Use a heap allocated buffer (with lifetime of Aria2 class) instead of
a stack allocated one.

Original fix made by @ZaWertun. Kudos to him.

Fix #kiwix/kiwix-desktop#123, kiwix/kiwix-desktop#513
and kiwix/kiwix-desktop#423
2020-08-26 12:42:16 +02:00
Matthieu Gautier af9e03904c Use std::mutex and std::unique_lock instead of pthread mutex/lock.
It simplify a bit the code and ensure that mutex is correctly unlock
even in case of exception.
2020-08-26 12:30:56 +02:00
Matthieu Gautier 39611cbd60 Wait for waitingThread to exit before destroying the subprocess memory.
WaitingThread read some shared memory with the SubProcess
(`mutex`, `m_running`).
When we destroy the SubProcess, we must be sure that WaitingThread has
correctly finished else we may have invalid read/write on freed memory.
2020-08-26 12:26:04 +02:00
Matthieu Gautier 6f0d3003ac Remove `m_compress` member. 2020-08-13 11:16:41 +02:00
Matthieu Gautier ee17b0739a Fix compilation on CI native dyn.
On the CI, the native_dyn docker image is setup with a packaged version
on libmicrohttpd for which `MHD_HTTP_RANGE_NOT_SATISFIABLE` is not
defined.

When the CI will be fixed, we can revert this commit.
2020-08-13 11:16:41 +02:00
Matthieu Gautier 47436f7bdd Move some header setting in response's constructors.
It make easier to understand what is somehow constant and what depends
of the context.
2020-08-13 11:16:41 +02:00
Matthieu Gautier 3352c95314 Remove the `RedirectResponse` and use a basic `Response` with header. 2020-08-13 11:16:41 +02:00
Matthieu Gautier 77123ac74c Move the adding of 304 headers in 304 factory.
This avoid us to create a ContentResponse just to have some correct
headers.
2020-08-13 11:16:41 +02:00
Matthieu Gautier 9078f0ac6e Remove `ResponseMode`. 2020-08-13 11:16:41 +02:00
Matthieu Gautier 8d6567d067 Create a utility builder for 416 response.
Also add a map in the response to store specific headers.
2020-08-13 11:16:41 +02:00
Matthieu Gautier 6d5cddca12 Fix android compilation
Android clang complains about the fact it cannot move the
`std::unique_ptr<ContentResponse>` into a `std::unique_ptr<Response>&&`
(for the implicit `std::unique_ptr<Response>` constructor).
Let's help him a bit.
2020-08-13 11:16:41 +02:00
Matthieu Gautier a3939e9a05 Move all the content code in the ContentResponse. 2020-08-13 11:16:41 +02:00
Matthieu Gautier eee621d15b Move small utilities method to create response in Response class. 2020-08-13 11:16:41 +02:00
Matthieu Gautier 7b2ee37437 Move the entry response to its own class. 2020-08-13 11:16:41 +02:00
Matthieu Gautier f014fb2895 Introduce a ContentResponse.
This is only an "interface" for now as other type of response (entry) may
be "transformed" to a ContentResponse.
We cannot move all the code in the class.
2020-08-13 11:16:41 +02:00
Matthieu Gautier 1011d1ff0b Move the redirection response in its own class.
The redirection is the easiest to move, let's start with this one.
2020-08-13 11:16:41 +02:00
Matthieu Gautier 9e351b279e Remove `get_default_response` in favor of a static Response method.
We want to build different kind of response depending of the context.
2020-08-13 11:16:41 +02:00
Matthieu Gautier a0bdc0821c Move internalServer code into its own source files. 2020-08-13 11:16:41 +02:00
Matthieu Gautier a819d9e3e0 Make the server handle pointer to response instead of plain response.
This is a preparatory work.
We will specialize the response and so we need a pointer to response
instead of plain response.
2020-08-13 11:16:41 +02:00
manan jethwani c74b935a9b added pageLength for search_pagination 2020-08-12 02:08:02 +05:30
Matthieu Gautier a55d504017 Fix getArticleCount.
With #403, the article mimetype may be different than "text/html".
It can also be "text/html; raw=true".
(And in fact it already could have any kind of optional argument).
2020-08-11 18:27:54 +02:00
Matthieu Gautier 87b5adcaf4 Make the response responsible to detect if we must introduce taskbar.
The response detect if taskbar must be added depending of the mimetype.

Now, `set_taskbar` can be call unconditionally
(no need to check for the mimetype)

And we don't need to call set_taskbar if we have no information to set.
2020-08-11 18:27:54 +02:00
Veloman Yunkan c4e6313c90 x in a --> a.contains(x) in meson.build files 2020-08-11 18:17:18 +02:00
renaud gaudin 3f25a3d005 Fixed #391: prevent taskbar and blocker at article level
Some HTML articles are meant to be displayed through a viewer. In this case,
we know we don't want the server to inject the taskbar nor the link blocker
because the content is not a user-ready web page but a partial element of it.

Such articles still need to be `text/html` to be parsed properly by browsers.

This changes the way we decide to display the tasbar or not.
Previously, we were adding it to every article with a MIME __starting with__ `text/html`.
Now, we're additionally preventing it on `text/html` MIME if there is a `;raw=true` string inside.

This leaves articles with MIME `text/html;raw=true` (warc2zim convention) outside
of the taskbar target.

For similar reasons, the external-link blocker is set to apply to the same set of articles.
Previously, it was applied to all articles which was an (unoticable) mistake.
2020-08-07 09:26:24 +02:00
MananJethwani 599aaa4c1b added code for status code 204 for empty return of search. 2020-08-01 01:45:42 +05:30
Veloman Yunkan 3d425f44de Request header case is ignored
Originally reported against case sensitivity of the Range header
(see issue #387), this fix applies to all request headers (since
according to RFC 7230 all header fields are case-insensitive, see
https://tools.ietf.org/html/rfc7230#section-3.2). However, a
corresponding unit-test was added only for the Range header.
2020-07-30 16:01:51 +02:00
Matthieu Gautier 7ece383004 Add support for samba path on windows.
Fix kiwix/kiwix-desktop#429
2020-07-15 11:40:02 +02:00
Kelson cf8e8b94eb Fix compilation with libmicrohttpd v0.97.1 2020-07-08 14:42:46 +02:00
Matthieu Gautier 4d307e18eb Add new thread safe suggestion API.
Previous API were using an internal vector to store the suggestions search
results.

The new API takes a vector as out argument. So user can call the functions
without having to protect the search.

We should change the android API to reflect the change but it is a bit
more complex to do at JNI level. As android do not call it multithreaded
we are safe for now. And we need the new API asap for kiwix-desktop.

So we keep the same API on android for now, the new api will be made
in next version.
2020-07-01 17:16:13 +02:00
Kunal Mehta fb79cde729 Pass -latomic for architectures that need it
Some architectures, specifically armel, mipsel, m68k & powerpc in
Debian, need to explicitly link to atomic.

Use meson to see if the target's CPU family is one of those, and if so,
pass -latomic to the linker.

Tested on armel and mipsel machines to verify passing -latomic works, and
on armhf and amd64 to ensure normal builds aren't broken.

Fixes #371.
2020-06-29 00:18:13 -07:00
Matthieu Gautier ff605873ed Include missing `algorithm` header.
`min` and `max` functions are defined here.
2020-06-10 15:27:51 +02:00
Veloman Yunkan 05ef5d5f51
Assertion in ByteRange allows 0-sized content
The assertion in the ByteRange constructor was written under the assumption that the content must have non-zero size. Now it allows that corner case.
2020-06-02 21:53:47 +04:00
Veloman Yunkan f52b220d01 Dropped RequestContext::has_range() 2020-05-26 14:10:26 +04:00
Veloman Yunkan 50a850f3a9 Fixed a comment 2020-05-26 14:04:18 +04:00
Veloman Yunkan 886ae17274 Fixed a CodeFactor issue 2020-05-26 13:59:47 +04:00
Veloman Yunkan 85d6daabac Rolled back minor unneeded changes 2020-05-26 13:10:50 +04:00
Veloman Yunkan 5f1918d005 Split a long line 2020-05-26 13:04:03 +04:00
Veloman Yunkan 16bd79fa1b Final clean-up of byte_range.{h,cpp} 2020-05-26 12:50:08 +04:00
Veloman Yunkan c2ebdefe8d Handling of unsatisfiable ranges 2020-05-26 02:11:26 +04:00
Veloman Yunkan 37032892a4 Fixed compilation error under win32_*
ERROR is a macro under Windows
2020-05-26 01:58:17 +04:00
Veloman Yunkan 6b43438b74 Fixed compilation error under native_dyn
MHD_HTTP_RANGE_NOT_SATISFIABLE is not defined in the older version of
libmicrohttpd (that is used under CI/Linux native_dyn).
2020-05-26 01:54:36 +04:00
Veloman Yunkan 7301bf89bb Some refactoring of byte-range parsing 2020-05-26 01:50:29 +04:00
Veloman Yunkan ff23b28e7c Removed unnecessary qualifier 2020-05-26 01:41:37 +04:00
Veloman Yunkan 931e95f391 Invalid byte ranges result in 416 responses 2020-05-26 01:40:07 +04:00
Veloman Yunkan f7571b5b69 Content-Range header is set only for partial content 2020-05-25 17:42:18 +04:00
Veloman Yunkan 801ad18a89 ByteRange::resolve() 2020-05-25 17:27:35 +04:00
Veloman Yunkan 67a347c0c4 Moved byte-range parsing to byte_range.cpp 2020-05-25 17:21:10 +04:00
Veloman Yunkan 693905eb68 Default constructed ByteRange is a full range 2020-05-25 17:17:56 +04:00
Veloman Yunkan f3e79c6b4c Introduced src/server/byte_range.cpp 2020-05-25 16:43:44 +04:00
Veloman Yunkan 52f207eaa6 Support for single-ended byte ranges 2020-05-25 16:37:01 +04:00
Veloman Yunkan 67294217a8 ByteRange::Kind 2020-05-25 16:23:44 +04:00
Veloman Yunkan d111a40ce8 Response::m_byteRange 2020-05-23 20:35:22 +04:00
Veloman Yunkan 0c5bb3fcfe Moved ByteRange to a header file of its own 2020-05-23 20:08:53 +04:00
Veloman Yunkan 3fba8c20a0 Converted RequestContext::ByteRange to a class
Also renamed the `range_pair` data member of `RequestContext` to `byteRange_`
2020-05-23 19:59:47 +04:00
Veloman Yunkan 54db6049b7 Byte-range parsing not exposed in the header file 2020-05-23 18:58:19 +04:00
Veloman Yunkan 81c38d6b2b parse_byte_range() without side-effects 2020-05-23 18:53:16 +04:00
Veloman Yunkan e6a86c02ae Got rid of RequestContext::accept_range 2020-05-23 17:15:42 +04:00
Veloman Yunkan a0f7f32570 Re-ordered function definitions 2020-05-23 17:11:26 +04:00
Veloman Yunkan c39fce8839 RequestContext::parse_byte_range() 2020-05-23 17:09:51 +04:00
Veloman Yunkan de37489c53 Range header starts with a unit spec
After this commit valid ranges of the form "bytes=firstbyte-lastbyte" should
be handled correctly.
2020-05-22 17:17:31 +04:00
Veloman Yunkan 2a35a86de6 Fixed the size value used creating a response
In case of a partial response the size of the response is different
from the served entry size.
2020-05-22 16:49:35 +04:00
Veloman Yunkan 0a30a77c08 Handling of out of bound byte ranges 2020-05-22 16:46:38 +04:00
Veloman Yunkan 1a99bacfe3 Byte ranges are inclusive
The second component of a byte range, if present, designates the
index of the last byte to be included in the partial response.
2020-05-22 16:30:43 +04:00
Kelson 94c2ab4395 Add two OPDS related mime-types to compress for HTTP 2020-05-18 08:19:51 +02:00
Kelson 0f07cab920 Small HTTP header beautification 2020-05-17 20:19:19 +02:00
Veloman Yunkan 5f0a9d0b08 Added a comment clarifying a non-obvious case 2020-05-15 15:17:04 +04:00
Veloman Yunkan 54f5dbbd35 Handling of If-None-Match conditional requests 2020-05-14 17:01:22 +04:00
Veloman Yunkan 95a5cde359 ETags are set in the response as needed
Also added server-unit tests related to ETags in the response.
2020-05-14 17:01:22 +04:00
Veloman Yunkan 3d08ef43f2 HEAD request is not rejected
libmicrohttpd handles HEAD requests by dropping the body of the response
(if any). Hence letting a HEAD request through into the code that
processes GET requests is safe.

Also added server unit-tests related to the handling of HEAD requests.
2020-05-14 17:01:22 +04:00
Veloman Yunkan bfa51c2d87 Refactoring: got rid of duplicate get_mime_type() 2020-04-29 18:33:25 +04:00
Veloman Yunkan 81e781133d Refactoring: utilized is_compressible_mime_type() 2020-04-29 18:33:01 +04:00
Veloman Yunkan 9ec7757efe Refactoring: smart Response::set_entry()
Response::set_entry() was upgraded from a simple setter to a method
performing certain business logic that was previously taken care of by
InternalServer::handle_content().
2020-04-29 18:22:15 +04:00
Veloman Yunkan 7bd7ec4937 Refactoring: preparing to move some code 2020-04-29 18:22:15 +04:00
Veloman Yunkan 14d8583c83 Refactoring in InternalServer::handle_content()
Deduplicated common code found in the two branches of the last
if(){}else{} statement in InternalServer::handle_content().
2020-04-29 18:22:15 +04:00
Veloman Yunkan a004d96cd7 Refactoring: extracted get_range_len() 2020-04-29 18:22:15 +04:00
Veloman Yunkan 21c6de2f80 Refactoring: split Response::create_mhd_response()
The changes are easier to understand in ignore-white-space mode
(git diff -w, git show -w).
2020-04-29 18:22:15 +04:00
Veloman Yunkan a8e78f27e1 Refactoring: extracted Response::create_mhd_response() 2020-04-29 18:22:15 +04:00
Veloman Yunkan 6c7ab6ff54 Refactoring: moved local variable declarations 2020-04-29 18:21:40 +04:00
Veloman Yunkan 659ee6ba71 Refactoring: extracted InternalServer::build_redirect() 2020-04-29 16:08:10 +04:00
Veloman Yunkan 83ee8dec15 Made InternalServer::get_default_response() const 2020-04-29 16:08:10 +04:00
Veloman Yunkan 87cbbed9e3 Refactoring: extracted is_compressible_mime_type() 2020-04-29 16:08:10 +04:00
Veloman Yunkan a058520628 Refactoring: extracted get_mime_type() 2020-04-29 16:08:10 +04:00
Veloman Yunkan 1ef5ebfb52 Refactoring: extracted InternalServer::get_reader() 2020-04-29 16:08:10 +04:00
Veloman Yunkan bbc06931ad Refactoring: extracted get_book_name() 2020-04-29 16:08:10 +04:00
Veloman Yunkan 2d3bf9b981 Refactoring: extracted InternalServer::homepage_data()
Also typedef'ed kainjow::mustache::data as MustacheData
2020-04-29 16:08:10 +04:00
Veloman Yunkan fd80f2a89f Refactoring: extracted fullURL2LocalURL()
Also dropped RequestContex::valid_url
2020-04-29 16:08:10 +04:00
Veloman Yunkan abb3dec700 Refactoring: extracted str2RequestMethod() 2020-04-29 16:08:10 +04:00
luddens 0586ef6d41 fix open external zim
Check if the parameter `pathToSave` is empty before use it otherwise the
book path is empty too, which causes crash on opening external zim files
2020-04-20 15:22:36 +02:00
Matthieu Gautier 9d8bf8ddcb Create the dataDirectory before returning its path. 2020-04-15 08:24:55 +02:00
Matthieu Gautier 4c8aad0e68 Do not use std::fstream has it doesn't support wchar path.
This is surprising, but C++11 fstream doesn't have a constructor
that take wchar as path.
So, on windows, we cannot open a stream on a path containing non ascii
char. VC++ provide an extension for that, but it is not standard and
g++ mingwin doesn't provide it.

So move all our write/read tools function to the plain old c versions,
using _wopen to open wide path on windows.
2020-04-14 18:13:35 +02:00
Matthieu Gautier eb6f0f710c Correctly detetect the dataDir on windows.
We must use the wide version of the getenv to correctly handle the case
we have accents in the user directory.

This also change the default dataDirectory on windows from $APPDATA to
$APPDATA/kiwix.
2020-04-14 12:12:34 +02:00
Matthieu Gautier cbf5bd57a8 Adapt to new libzim api.
It is not possible to create a iterator without argument anymore.
2020-04-13 16:06:17 +02:00
Matthieu Gautier 533541cf45 Write the articleCount and mediaCount metadata in the OPDS stream.
This is not standard OPDS. But clients need this information.
2020-04-07 12:22:44 +02:00
renaud gaudin 7155c788e2 attach taskbar to `<head>` instead of `<head>\n`
Fixed a regression introduced in block-external-links feature.

For cleaner source, the taskbar (and the block-external JS file) were both
attached to `<head>\n`.
Unfortunately, this isn't safe enough as some ZIM files might have all kinds of HTML
syntax. Sotoki for instance have no CR after head, rendering the attachment impossible.

Note: realizing this method is somehow fragile as any HTML content with extra attribute
on the `<head>` tag or without a `<head>` tag would break the taskbar and the block external feature.
2020-04-03 16:53:43 +02:00
renaud gaudin 4709a42f4f disable external links blocking on 500 handler 2020-03-30 14:42:37 +00:00
renaud gaudin d04d9bf7f3 Unblock external link in catch page in JS code
Instead of disabling the blocking for the handler, the JS code detects it is
displaying the handler and allows external links to go through
2020-03-27 12:26:22 +00:00
renaud gaudin 412f0d9c61 moved blockExternalLink outside of taskbar
- `setBlockExternalLinks()` on server
- zero-dependency JS code
- JS script added in `inject_externallinks_blocker()`
- changed URL to `/catch/external?source=<source>`
2020-03-27 11:25:39 +00:00
renaud gaudin 0ad8bf45fc Add external links blocking in serve
In many use cases, it is not wanted to have user accidentaly click on external links
and leave the served ZIM content.
This could be because the result is unpredictible (reader not implementing this properly)
or because the serve user knows there's no backup internet connexion or because there is
an induced cost behind external links that doesn't affect served content.

using a new flag (`blockExternalLinks`) on `Response`/`setTaskBar`, a piece of JS code
is injected into the taskbar code.
This code adds a JS handler on all link click events and verifies the destination.
If the destination appears to be an external link (1), the link target is changed to
a specific URL:

```
/external?source=<original_uri>
```

(1) external is a link that's not on the same origin and starts with either `http:` `https:` or `//`.

Server implements a new handler on `/external` that displays a new page (`captured_external.html`)
which returns a generic message explaining the situation and offering to click on the link
again should the user really want to.
This is done by specifically asking `set_taskbar` to not block external requests on that page.

This approach allows integrators using a reverse proxy to handle that endpoint differently (rebrand it)

1. `Server` now has an `m_blockExternalLinks` defaulting to `false`
1. `Server.setTaskbar` is extended to support an additional bool to set the variable.
1. `Response` now has an `m_blockExternalLinks`
1. `Response` constr expects an additional bool for `blockExternalLinks`.
1. `Response.set_taskbar` is extended to support an additional bool to set the variable.
1. JNI/Java Wrapper reflects the extensions.
1. New resource file `templates/block_external.js` (included in head_part). Should it be in skin?
1. New resource file `templates/captured_external.html` for `handle_captured_external()`
1. Added a comment on `head_part.html` to help with JS insertion at the right place
1. `introduce_taskbar()` conditionnaly inserts the JS inside the taskbar
2020-03-26 12:06:36 +00:00
Matthieu Gautier 064d5f3fa6 Make the search argument constant. 2020-03-06 12:08:05 +01:00
Matthieu Gautier 46626a3f98 Add the method get bookByPath in library. 2020-03-06 12:08:05 +01:00
Matthieu Gautier 76c293e403 [JAVA] Add a method to get the size of an article.
Fix #327
2020-03-04 16:40:03 +01:00
Matthieu Gautier 2e60a088ab [JAVA] Use a long to store the offset of a article in the zim file.
Fixes kiwix/kiwix-android#1769
2020-02-19 14:27:51 +01:00
Matthieu Gautier ea29557a33 [Java] Add a wrapper on method to update book from another book or reader. 2020-02-11 17:44:14 +01:00
Matthieu Gautier b53f531f2b Fix typo getTagStr in the wrapper 2020-02-10 17:25:31 +01:00
Matthieu Gautier 1632e9c55b Fix launching command with path containing spaces.
We must correctly quote path with space on windows.
This is needed as we can't launch command using a array of string on
windows but by giving only one string using space as separator.

Fix kiwix/kiwix-desktop#268
2020-02-06 11:46:33 +01:00
Kelson 6a975994cc Include stdexcept to fix GCC v10 compilation 2020-02-01 13:52:27 +01:00
Matthieu Gautier ce6e956434 [OPDS] Add the url argument to filter by size and name.
Fix kiwix/kiwix-tools#231
2020-01-30 19:02:33 +01:00
Matthieu Gautier f560a1f815 Be able to filter the books by name. 2020-01-30 19:02:33 +01:00
Matthieu Gautier 34257cfc1f Trust the library.xml information by default.
Do not try to read the zim file and update the book when parsing a
library.xml.
Needed by kiwix/kiwix-tools#319
2020-01-30 18:22:07 +01:00
Matthieu Gautier a756e7f8f3 Add flavour attribute to book.
Fix #259
Fix kiwix/kiwix-tools#316
2020-01-30 17:48:56 +01:00
Matthieu Gautier bc257d2d6d Add method to get value of tag from a book.
Only the `getTagStr` method is available on android because we need a
proper exception handling on wrapping side.

Fix #298
2020-01-30 17:48:56 +01:00
Matthieu Gautier 7275f9b8e3 Move function to convert and use tags inside otherTools. 2020-01-30 17:48:56 +01:00
Matthieu Gautier 2881face70 Add missing setting of attribute. 2020-01-30 15:42:54 +01:00
Matthieu Gautier 77ba09c310 Reorder setting and dumper of book attribute.
No real change. Reordering setting and dumping of attribute in the same
order (mostly) they are declared in book.h make it easier to detect missing
attribute.
2020-01-30 15:42:54 +01:00
Matthieu Gautier 49aa0fbb9f Use a macro to write the filters. 2020-01-30 15:42:54 +01:00
Matthieu Gautier 7846b45bef Fix opds filtering by tag. 2020-01-30 15:42:54 +01:00
Matthieu Gautier 7e26f3502d Fix opds dumper/parser.
Add missing attributes.
2020-01-30 15:42:54 +01:00
luddens 4b6c26bd0b add parameter option to start a download with aria2
Downloader::startDownload has a new parameter option which is a vector of
pair that represents the options that can be set for adding a uri with aria2
with the function Aria2::addUri.

Aria2::addUri uses this parameter to set the struct of parameters for the
aria2 command
2020-01-29 15:38:09 +01:00
luddens 6bcecc2677 remove useless error handling functions
Some of the changes from https://github.com/kiwix/kiwix-lib/pull/292
that doesn't work properly are removed
2020-01-28 16:58:55 +01:00
Matthieu Gautier 82afb804e1 Add a small java test on the kiwix-lib.
Will the compilation should be made by meson.
It seems it is not possible to specify a existing jar
to link with. Use a custom script for now.
2020-01-28 12:08:18 +01:00
Matthieu Gautier 0951546356 Create the jar library when creating the java wrapper. 2020-01-28 12:08:18 +01:00
Matthieu Gautier f09c739c1f Be able to create a wrapper for java.
Android is a specific wrapper.
Java is another one.
2020-01-28 12:08:18 +01:00
Matthieu Gautier df9ddd5451 Use correct mutex on android and java. 2020-01-28 12:08:18 +01:00
Matthieu Gautier fe513951d3 Use a macro to print error log.
This allow use to compile the JNI wrapper not for
android.
2020-01-28 12:08:18 +01:00
Matthieu Gautier 7f0d509a88 Add the filter functionality. 2020-01-28 12:08:18 +01:00
Matthieu Gautier 54f671b2f1 Add some methods to get information from the library.
Else, the library is useless.
2020-01-28 12:08:18 +01:00
Matthieu Gautier c2c89c6c86 Rename the JNIKiwixLibrary class to Library.
This mainly use the "new memory system". No need to call dispose function.
Rename the class to Library to conform with the naming semantics
(JNIKiwix* use old memory system)
2020-01-28 12:08:18 +01:00
Matthieu Gautier 75652d0e9f WIP Add a wrapper around the kiwix::manager class.
The JNIKiwixManager is used to manage (insertion of book in) the library.

It is created, as needed, using an existing Library as input.
It is then used to add books, parse library.xml or opds content.
Then it can be destruct (and must be) with the `dispose` method.

```java
library = JNILibrary(...);
manager = JNIManager(library);
manager.parseOpds(opdscontent);
manager.dispose();
// library contains the books declared in the opds content.
// Use the library methods to get the books' info.
```
2020-01-28 12:08:18 +01:00
Matthieu Gautier 6535dc2e38 Add a wrapper for the Book class. 2020-01-28 12:08:18 +01:00
Matthieu Gautier 6b2f768c8f Use template function c2jni to convert c++ type to jni. 2020-01-28 12:08:18 +01:00
luddens ff21a095cb update book even if the members aren't empty
remove the conditions to always update the book
2020-01-27 17:50:44 +01:00
Matthieu Gautier 91db055d86 Remove function to read file using a native path.
All path must be utf8. This is already the case in all our project.
(If this not the case, this is a bug)

So we don't need to have a version with a native and utf8 path.
2020-01-13 16:59:58 +01:00
Matthieu Gautier 5540149e2b Correctly open the library path on windows.
We need to convert the path to wstring on Windows to handle directory
with accented characters.

Fix kiwix-desktop#269
2020-01-13 16:54:09 +01:00
Matthieu Gautier 071e9e3fec Correctly filter the catalog when we don't what to filter.
Set the different filter's fields only when we are requested to filter
them. Else, we ends to requests that some fields are empty.

If the request has no argument, we raise an exception (catched) and so
we don't set the corresponding field in the filter.

Fix #303
2020-01-07 17:29:43 +01:00
Matthieu Gautier 4a01303438 Correctly set the id of the opds stream.
Set the id of the stream *after* the uuid is generated.
2020-01-07 17:29:29 +01:00
Kelson b7c5e5f339 Remove trailing spaces 2019-12-08 11:52:16 +01:00
Kelson 52e165cf78 Reintroduce kiwix-serve taskbar 2019-11-26 11:54:00 +01:00
Kelson 9c4867a95a Update Changelog 2019-11-20 13:06:24 +01:00
Emmanuel Engelhart de7b7c34b5 Remove absolute internal URL support 2019-11-07 18:05:58 +01:00
luddens 20a2c78733 add get aria2 launch cmd method 2019-11-01 15:27:21 +01:00
luddens 9850be7267 add Curl error message 2019-11-01 15:27:21 +01:00
luddens 0dd996c6a3 add try catch around aria2 first commands 2019-11-01 15:27:21 +01:00
luddens c9a15c9961 Add a parameter to getBookmarks fct to get valid bookmarks only
The default value of this parameter is false, in this case all the bookmarks
are returned, otherwise only those who are related to books of the library.
2019-10-31 14:05:21 +02:00
luddens 9975e0b369 add setPort() method 2019-10-28 15:56:49 +01:00
Aditya-Sood 2af9ba4eab Readd original getNextSuggestion() 2019-10-01 13:30:35 +05:30
Aditya-Sood c007373b46 Re-add comment 2019-10-01 13:30:34 +05:30
Aditya-Sood e1acf9acff Code & local repository cleanup 2019-10-01 13:30:34 +05:30
Aditya-Sood daaadf3e1c Comment out previous definitions 2019-10-01 13:30:34 +05:30
Aditya-Sood 74bd482335 Preliminary work 2019-10-01 13:30:34 +05:30
Matthieu Gautier 67170709bb Convert path get from windows environment to utf8.
Fix kiwix/kiwix-desktop#203
2019-09-25 18:07:42 +02:00
Matthieu Gautier 0db06d98a8 Add missing implementation of android's getArticleCount and getMediaCount.
Fix #281
2019-09-24 11:47:05 +02:00
Matthieu Gautier 598dd3c175 [API Break] Fix pathTools (and a bit stringTools).
Api changes :
 - removeLastPathElement do not takes extra arguments
   `removePreSeparator` and `removePostSeparator`.
   This is not needed as path do not need special tailing separator.
 - Only one function `split`. Arguments can be implicitly convert to
   string. No need for overloading functions to explicitly cast them.
 - `split` function takes another argument `trimEmpty`. If true, empty
   element are removed.

Path manipulation now almost pass trough a vector<string> to store each
path's part.

Most of the complex works is now made in the normalizeParts function.
2019-09-19 18:16:06 +02:00
Matthieu Gautier 2f4636e2df Fix stringTools join function. 2019-09-17 16:22:28 +02:00
Matthieu Gautier 9b4419f3fc [ABI Break] Correctly detect the executable path in appimage.
There are two executable path :
- The user one (the appimage path)
- The real one (in the appimage archive)

When we search of `library.xml` we need the user one.
But when we search of `aria2c` or `kiwix-serve` we need the real one.

Fix kiwix/kiwix-desktop#256
2019-09-17 11:23:16 +02:00
Matthieu Gautier 6ee174b546 Add a method to get the value of a specific tag.
Fix #258
2019-09-17 10:37:53 +02:00
Matthieu Gautier 2a6772b76d [API Change] Convert tags to the new convention.
Use the new convention describe here : https://wiki.openzim.org/wiki/Tags
2019-09-17 10:30:24 +02:00
Matthieu Gautier 660d5d7fb7 [API Change] Rename getMatatag to getMetadata. 2019-09-16 10:36:04 +02:00
Matthieu Gautier 157c1c939c Add a string tool to join a list of strings together. 2019-09-16 09:42:10 +02:00
Matthieu Gautier bd91e89785 Add missing method to get the zim metadata.
According to https://wiki.openzim.org/wiki/Metadata
2019-09-12 15:33:07 +02:00
Matthieu Gautier 1245d4e467 Use a macro to get the content of the metadata. 2019-09-12 15:26:53 +02:00
Matthieu Gautier 420be55bfa Reorder methods to get metadata.
Use the same order than https://wiki.openzim.org/wiki/Metadata
2019-09-12 15:24:17 +02:00
Matthieu Gautier e42e061d45 Add a way to specify a library to use with kiwix-serve.
If kiwix-desktop use a `library.xml` in the same directory than the
executable, we need to use it instead of the default one.

Instead of detect again the `library.xml` to use, let `kiwix-desktop` set
the library to use.

This also fix a issue when `/` is not a valid path separator in windows.
2019-09-11 14:04:21 +02:00
Matthieu Gautier 3294508d87 Correctly cast double to int.
Ms cl compiler complains about the implicit conversion.
2019-09-10 14:10:40 +02:00
Matthieu Gautier a32363e6a2 Correctly detect the executable path if we use a AppImage.
AppImage works by decompressing the "program" in a temporary directory.
So the executable path is not the path of the AppImage file.

By using the environment variables set by appimage we can find the correct
"path" of the executable.

Fix kiwix/kiwix-desktop#46
2019-09-09 18:27:53 +02:00
Matthieu Gautier 87dc145dc7 Correctly set searcher information even if resultStart equals resultEnd. 2019-09-09 14:43:51 +02:00
Matthieu Gautier a13244dc0e Rename `hasResult` to `hasResults` 2019-09-09 14:43:51 +02:00
Matthieu Gautier 78dbd66522 [HTML Rendering] Do not render page navigation buttons if only one page. 2019-09-09 14:43:51 +02:00
Matthieu Gautier fdc291b7c2 [HTML Rendering] Do not do division by zero.
We must correctly handle the case if resultStart is equal to resultEnd.
2019-09-09 14:43:51 +02:00
Kelson d0833bdcd4 Fix fulltext search link in kiwix-serve suggestions 2019-09-04 17:07:05 +02:00
Matthieu Gautier 12a93c3e29 Fix use of strtok on windows.
On windows, strtok_r is called strto_s.
2019-08-19 17:35:22 +02:00
Emmanuel Engelhart cea201b394 Specs says ZIM favicon should be at '-/favicon', should be tried first 2019-08-19 16:27:55 +02:00
mhutti1 8672aede97 Add JNIKiwixString constructors 2019-08-13 17:55:48 +02:00
Matthieu Gautier 6ab52e2b9e Add a JNI method to check if an url exists and is a redirection. 2019-08-13 11:19:39 +02:00
Matthieu Gautier d90a27af11 Fix nameMapper initialization. 2019-08-12 16:16:26 +02:00
Matthieu Gautier a65e192f0f [JNI] Allow android to know that an article is a redirect.
Android need to handle the redirection by doing a redirection in the web
view, not by providing the content of the targeted article.

This is already what we do in kiwix-serve or ios.

The API should be far better by returning a Entry but for now,
we just change the given url if the article is a redirection.
2019-08-12 13:03:20 +02:00
Matthieu Gautier 513cc9c90f [JNI] Fix log typo. 2019-08-12 12:43:52 +02:00
Matthieu Gautier 231ae095f6 Correctly set that book's path is valid when updating it from a reader. 2019-08-12 12:43:52 +02:00
Matthieu Gautier 9a0c6da018 Library construction doesn't take argument 2019-08-12 12:43:52 +02:00
Matthieu Gautier 52299ef767 Fix computeAbsolutePath.
Correctly delete the duplicated string.
Use strtok_r to be thread safe.
2019-08-12 12:43:52 +02:00
Matthieu Gautier c4963268ba Fix regexTools.
The buildMatcher must not take a rvalue as it will keep a reference
to it.
2019-08-12 12:05:51 +02:00
Matthieu Gautier 73a29ccb24 [JNI] Fix implementation of setDataDirectory.
Now that setDataDirectory is a static method, we need to take an
jclass instead of a jobject.
2019-08-11 15:09:26 +02:00
Matthieu Gautier 7060afae66 [SERVER] Catch any error and return a 500 response instead of crashing.
The server will be running some code on the behalf of the calling code.
We really don't what to crash the library (and the binary) because
of a wrong request.
2019-08-11 11:30:43 +02:00
Matthieu Gautier 4d3df4e889 Add JNI wrapper around the library and the server. 2019-08-11 11:30:43 +02:00
Matthieu Gautier cd050ddcc8 Use camelCase. 2019-08-11 11:30:43 +02:00
Matthieu Gautier 635d4438e5 Make the server take a pointer to the library instead of a reference. 2019-08-11 11:30:43 +02:00
Matthieu Gautier ce09375c6c Reduce complexity of handle_search. 2019-08-11 11:30:43 +02:00
Matthieu Gautier fae0918f49 Reduce complexity of handle_catalog. 2019-08-11 11:30:43 +02:00
Matthieu Gautier d90f8b0f05 Add a name_mapper mapping the HumanReadable name to the id. 2019-08-11 11:30:43 +02:00
Matthieu Gautier abb5db0193 Clean up includes in server.cpp 2019-08-11 11:30:43 +02:00
Matthieu Gautier e452f5cf36 Ensure that the root is correctly formatted. 2019-08-11 11:30:43 +02:00
Matthieu Gautier c890e1c87e Add support of a binding to a specific ip address. 2019-08-11 11:30:43 +02:00
Matthieu Gautier e5ef3780db Rename humanReadableBookId to bookName.
`humanReadableBookId` is a bit long and doesn't represent what it is
(this is not a id).
`bookName` is far better.
2019-08-11 11:30:43 +02:00
Matthieu Gautier 2aeed65205 Better handling of invalid request.
Do no crash if we can get a book or a reader for the requested content.
2019-08-11 11:30:43 +02:00
Matthieu Gautier c1faf55ae8 Introduce the server functionality in the kiwix-lib.
This code is mainly copied from kiwix-tools.

But :
- Move all the response thing in a new class Response.
- This Response class is responsible to handle all the MHD_response
  configuration. This way the server handle a global object and do
  no call to MHD_response*
- Server uses a lot more the templating system with mustache.
  There are still few regex operations (because we need to
  change a content already existing).
- By default, the server serves the content using the id as name.
- Server creates a new Searcher per request. This way, we don't have
  to protect the search for multi-thread and we can do several search
  in the same time.
- search results are not cached, this will allow future improvement in the
  search algorithm.
- the home page is not cached.
- Few more verbose information (number of request served, time spend to
  respond to a request).

TOOD:
 - Readd interface selection.
 - Do Android wrapper.
 - Remove KiwixServer (who use a external process).
 -
2019-08-11 11:30:43 +02:00
Matthieu Gautier 64dfea2547 Move the search html renderer in a different class than the searcher.
This is two different functionnalies, we don't need to polute the searcher
api with things to render the html.
2019-08-11 10:19:48 +02:00
Matthieu Gautier cca5980b27 Remove limitation of the search len in the Searcher.
The limitation should be made elsewhere (the code using the searcher).
2019-08-11 10:19:48 +02:00
Matthieu Gautier ce8fff0b42 Make the library create the reader. 2019-08-11 10:19:48 +02:00
Matthieu Gautier e56335109c Make appendToFirstOccurence take argument by reference. 2019-08-11 10:19:48 +02:00
Matthieu Gautier 656bf183b7 Make getHumanReadableFromPath method const. 2019-08-11 10:19:48 +02:00
Matthieu Gautier cbe8e20118 Fix include in otherTools.h 2019-08-11 10:19:48 +02:00
Matthieu Gautier e013d38cc6 Add startWith function in stringTools.h 2019-08-11 10:19:48 +02:00
Matthieu Gautier 6234457920 Fix include in stringTools.h 2019-08-11 10:19:48 +02:00
Matthieu Gautier 72223d69fe Fix include in pathTools.h 2019-08-10 11:02:23 +02:00
Matthieu Gautier ddeb862395 Add getMimeTypeForFile in pathTool. 2019-08-10 10:59:13 +02:00
Matthieu Gautier 61c28f0e3d Make the regexTool thread safe. 2019-08-10 10:59:13 +02:00
Matthieu Gautier 654a8e304c Set the verbosity of the underlying zim::Search. 2019-07-30 17:03:44 +02:00
Sean Mac Gillicuddy 5adf7891cc #239 ReLinker should be used to load library 2019-07-29 18:31:57 +02:00
luddens c17abdae5e port can be set with the constructor
add a method to return the port
2019-06-27 16:50:22 +02:00
Matthieu Gautier 31c9375a3a Better API to filter books in a library.
Instead of having a single method `listBooksIds` that tries to be
exhaustive about all the filter and sort option, split the method in
two separated methods `filter` and `sort`.

The `filter` method takes a `Filter` object that represent on what we are
filtering. This object has to be construct before calling `filter`.

```cpp
Filter filter;
filter.query("Astring");
filter.acceptTags({"nopic"});
// return all book in eng and with "Astring" in the tile or description".
library.filter(filter);
//equivalent to
library.listBooksIds(ALL, UNSORTED, "Astring", "", "", "", {"nopic"});
// or better
library.filter(Filter().query("Astring").acceptTags({"nopic"}));
```

The method `listBooksIds` has been marked as deprecated.

Add a small test on the library.
2019-06-26 16:41:01 +02:00
Matthieu Gautier 3d689e790b Force the exit of the forked process.
The `exit` function will call the functions registered.
Qt may (and does) register function and may (and does) hangup waiting
for some mutex to be free.

Here we are in a forked process and we want to process to exit without
doing any cleanup (because we are a clone of a process that will continue
and we don't want to mess it).
2019-06-25 16:52:46 +02:00
luddens 87f5b56b72 add isRunning method 2019-06-24 15:05:20 +02:00
luddens df76db4f47 Fix crash searchbar's suggestions
The suggestion's list was filled without checking if the current suggestion
was wrong which caused crash in this case.
2019-06-24 11:48:14 +02:00
Matthieu Gautier 6fe6f88b10 Fix waitpid option.
`WEXITED` is an option for the `waitid` function not `waitpid`.

We don't need to pass a option to `waitpid`.
2019-06-24 11:34:03 +02:00
Matthieu Gautier 4e746916a7 Remove unnecessary include.
And unistd.h is not available on windows.
2019-06-12 14:27:15 +02:00
luddens 5dc96d7145 kiwix-serve integration 2019-06-11 10:35:16 +02:00
Matthieu Gautier 48347825a9 suggestions method return a unique_ptr instead of a raw pointer. 2019-05-28 15:08:16 +02:00
luddens 519dd110f5 add tags in opds stream 2019-05-14 10:51:22 +02:00
luddens 491b6d655b add remove fct with aria2 2019-05-13 10:30:19 +02:00
luddens ec8f1ffe9c Add pause and unpause functions for aria2
This functions enable to stop and resume download with aria2.

The Downloader's constructor now checks the paused downloads with the
function "tellWaiting()" to get them at the start of kiwix-desktop.
2019-05-13 10:30:19 +02:00
luddens 12ffad55f7 update download's status when we get it 2019-05-13 10:30:19 +02:00
Matthieu Gautier 0b2c9fa7ce Revert Do not use this_thread::sleep.
We already have a custom sleep method.
2019-05-03 15:38:47 +02:00
Matthieu Gautier fe682f855a Do not use this_thread::sleep.
Mingw doesn't implement it. So, we should not use it.
I suppose that it was working before because mingw package for debian trusty
simply no provides a "thread" header.
We may face to include the native "thread" header.
2019-04-17 15:44:14 +00:00
cyrillemoureaux e1980d190f Detect and reject infinite redirect loops.
This will prevent kiwix/kiwix-tools#207.
2019-04-15 16:55:48 +02:00
Vitaly Zaitsev 28f144796d
Removed Rpath from compiled binaries.
Rpath is not needed to run installed binaries. Most of GNU/Linux
distributions strictly forbid it.

Signed-off-by: Vitaly Zaitsev <vitaly@easycoding.org>
2019-04-15 16:09:53 +02:00
Matthieu Gautier e7e88617d5 Remove unused include unistd.h
We don't use it anymore and it is not present on Windows.
2019-04-15 10:50:26 +02:00
Matthieu Gautier f8522fb26e Remove getNetworkInterfaces and getBestPublicIp function.
They are not used at all and the windows version need some extra libs
that complexify the code.

Remove them for now. If it happens that they are needed, we will readd
them.
2019-04-04 17:00:45 +02:00
Matthieu Gautier 938e2a81c1 Rewrite makeTmpDirectory to not use Uuid methods on windows.
`UuidCreate`, `UuidToString` and `RpcStringFree` need special library
on windows. Lets not use them.
2019-04-04 16:11:36 +02:00
Matthieu Gautier d40982f760 Remove error message when trying to opening a wrong zim file.
The library should not print a message.

Fix kiwix/kiwix-tools#235
2019-04-02 15:28:46 +02:00
Matthieu Gautier c6254d9504 Allow the library to be filtered by tags.
This add an argument to `listBooksIds` to filter by tags.
So, this is an API break.
2019-03-07 17:08:39 +01:00
Matthieu Gautier 93af3aa2d1 Fix the language mapping.
The previous mapping was taken from an unknown (:/) source.

The new mapping is generated with a script taking
https://www.loc.gov/standards/iso639-2/php/code_list.php as source.

The source list is sanitized to keep only language for which we
(http://library.kiwix.org/) have content.
2019-03-05 17:53:37 +01:00
Matthieu Gautier af9734c87f Correctly initialize variable. 2019-02-19 14:05:37 +01:00
Matthieu Gautier 35d812a5f7 Ensure the book's `path` is absolute.
We must use absolute path whenever possible.
Relative path has sense only related to the "interaction" with the user
(current directory, library location, ...).
2019-02-07 15:22:33 +01:00
Matthieu Gautier 432f9c30a3 Remove unused variable url. 2019-02-07 15:20:18 +01:00
Matthieu Gautier af7689e3e8 [API break] Move all the tools in the tools directory instead of common.
The `common` name is from the time where kiwix was only one repository
for all the project (android, desktop, server...).

Now we have split the repositories and kiwix-lib is the "common" repo,
the "common" directory is somehow nonsense.
2019-01-23 15:31:38 +01:00
Matthieu Gautier b996a2877c Correctly initialize retVal. 2019-01-23 11:51:30 +01:00
Matthieu Gautier be498c3b16 Make the string Tools functions available in android. 2019-01-09 18:29:20 +01:00
Matthieu Gautier 92c9a47a0d Move the templating system to mustache instead of ctpp2.
Mustache templating system is a bit simpler than ctpp2 and ctpp2 is no
more maintained (see #189).
We are moving to the kainjow's Mustache project
(https://github.com/kainjow/Mustache).

It simplify a lot our system has it is header only and we don't have to
precompile the template.

Fix #21
2019-01-09 18:28:48 +01:00
Matthieu Gautier 5159d985c6 Remove support for external index.
This feature is considered obsolete for a while.
In fact, it was already not supported since June 2018 as we were compiling
xapian without the chert backend support.

Assume that we don't support it and remove it from the code.
See kiwix/kiwix-tools#245

This is a API break. library.xml files will still work but the indexPath
and indexType will be dropped silently from the file.
2019-01-07 16:47:08 +01:00
Matthieu Gautier 29046bfc05 Create the datadirectory to not fail to write the aria2 session file.
Fix kiwix/kiwix-desktop#69
2018-12-14 15:24:13 +01:00
Matthieu Gautier e6d7ba06fb Convert the standard opds date to our format (YYYY-MM-DD) 2018-12-11 17:02:02 +01:00
Matthieu Gautier 0f812c6584 The update entry of the book should be the date of the book, not the feed. 2018-12-11 17:01:33 +01:00
Matthieu Gautier 716c87dd20 Remove duplicate language attribute in the libxml dumper.
Silly copy/paste.
2018-12-11 17:00:56 +01:00
Matthieu Gautier 12498e2cfe Add bookmarks support.
The library now contains (simple) methods to handle bookmarks.
The bookmark are stored in a separate xml file.

Bookmark are mainly a couple (`zimId`, `articleUrl`).
However, in the xml we store a bit more data :
- The article's title (for display)
- The book's title, lang and date (for potential update of zim files)
2018-12-02 15:47:29 +01:00
Matthieu Gautier b5ce60a627 Move the dump of the library into library.xml in a specific class.
The same way the dump into a opds feed is in a specific class.
2018-11-28 12:09:28 +01:00
Matthieu Gautier 94a053e821 Wait a bit more between attempts to connect to aria2c rpc. 2018-11-12 16:14:21 +01:00
Matthieu Gautier 4b9692bbd5 Correctly run aria2c when packaged with kiwix-desktop in appimage.
By default, we are searching in the PATH env var.
However, with an appImage, the executable directory is not in the PATH,
so we have to use an absolute path if we can.

If we cannot find the aria2c executable in the executable directory let's
try to use the system one.
2018-11-12 14:35:17 +01:00
Matthieu Gautier 4b31842c4a Correctly convert filesize from Kbyte to byte.
`reader.getFileSize()` return the size of the zim in Kbyte in a
`unsigned int` (32 bits). This is ok as it would overflow if the size
of the size is greater than 4294967295 kbytes (so ~4Tbytes).

However, we need to convert the return size into a unsigned 64 bits integer
else, when converting to bytes, we will overflow at 4Gbytes.
Even in `m_size` is a uint64_t.
2018-11-12 12:16:05 +01:00
Matthieu Gautier cf1cfe774e Correctly check for ArticleCount and MediaCount before writing them. 2018-11-12 10:58:10 +01:00
Matthieu Gautier 8c4b9fbe95 Add missing `en`->`eng` mapping to `codeisomapping`.
The most common used language was missing :/

Fix kiwix/kiwix-desktop#51
2018-11-12 10:36:45 +01:00
Matthieu Gautier ab63cb2fb8 Sort `codeisomapping` alphabetically.
This is only code formating, no real change.
2018-11-12 10:34:19 +01:00
Matthieu Gautier 3958b2a06f Make the internal map codeisomapping static.
Symbole should not be visible outside of the compilation unit.
2018-11-12 10:33:35 +01:00
Matthieu Gautier d4ecda40ff Use the correct separator when computing relativePath. 2018-11-03 12:18:54 +01:00
Matthieu Gautier 9ab44e6a5f Get information about the total number of book of a search.
When we do a search and paging the result, we need to display to the
user the total number of book, not only the `itemsPerPage`.

So, we need to parse correctly the xml to keep information of the total
number of book.
2018-11-02 17:04:55 +01:00
Matthieu Gautier 5f4c04e79e Fix use of getAsI when parsing download rpc.
The value is store as a string in in the xml, so we cannot use getAsI.
We have to get the string and parse it to an int.
We cannot use strtoull because android stdc++ lib doesn't have it.

We have to implement our how parseFromString function using a
istringstream.
2018-11-02 17:03:03 +01:00
Matthieu Gautier bb07ff5610 Do not add NULL at end of commandLine on Windows. 2018-10-31 13:56:42 +01:00
Matthieu Gautier 1787e30440 Better launch of the aria2 process.
By setting the ApplicationName to NULL, CreateProcessW will
search for the application in the path.
2018-10-31 13:56:42 +01:00
Matthieu Gautier ccb3d8639d Use correct name for aria2c on windows. 2018-10-30 18:43:30 +01:00
Matthieu Gautier 29e554b47b Include pthread 2018-10-29 14:30:35 +01:00
Matthieu Gautier 68dc4d40b5 Include windows.h before synchapi.h 2018-10-29 12:20:00 +01:00
Matthieu Gautier 2682fa8f9c Remove unecessary variable or output. 2018-10-26 14:19:10 +02:00
Matthieu Gautier a22f962722 Correctly store the size of the book in the library.
`reader.getFileSize()` return ko.
2018-10-26 14:18:40 +02:00
Matthieu Gautier a1876e3b27 Add a method converta2toa3 to convert language code alpha2 to alpha3.
Qt give use alpha2 language code but we use alpha3.
2018-10-26 14:18:06 +02:00
Matthieu Gautier ad654ead08 Do not force the download port to be 80.
We may want to use url with port != 80.
2018-10-24 11:56:38 +02:00
Matthieu Gautier c6206edfb4 Do not always download the favicon of a book. Download as needed.
When parsing a opds feed, the favicon is a url, not a dataurl.
If we download the favicon all the times, it may take a lot of time to
parse the feed.

We store the url and download the favicon only when needed (when displayed)
2018-10-24 11:56:05 +02:00
Matthieu Gautier c20ae18bff An opds feed can also be the openSearch result.
We must be able to set the correct entry in the feed for a searchResult.
2018-10-24 11:51:38 +02:00
Matthieu Gautier b1508c0b98 Better listBooksIds supported mode.
Only have REMOTE or LOCAL is a bit restrictive. By using flags a user
can specify for complex request.
2018-10-24 11:50:11 +02:00
Matthieu Gautier 34021994cd Fix for Android
- No std::to_string. We have to implement it with a ostringstream
- No pthread_cancel. So we use pthread_kill to send a signal to the thread.
2018-10-24 10:48:53 +02:00
Matthieu Gautier 910ce5f10d Fix for Windows
- "winsock2.h" needs to be included before "windows.h". But if a
  compilation unit include "windows.h" and after "networkTools.h", we
  fails and it is complicated to handle. The include must not be in the
  header but in the cpp
- windows define some ERROR macro. It is a pitty but we cannot use `ERROR`
  in our enum.
- If build statically using mingw we need to define `CURL_STATICLIB`
2018-10-24 10:47:12 +02:00
Matthieu Gautier c66c7e9c20 Store the size of the book in OPDSFeed. 2018-10-24 10:47:12 +02:00
Matthieu Gautier ad69fdd8c0 Move the download method from the downloader to networkTools.
The download method is a simple method to download content.
It use curl to download the content instead of aria.
2018-10-24 10:47:12 +02:00
Matthieu Gautier a73ef23f6e Keep the book size in byte in memory (instead of in kb)
We keep the size in kb in library.xml for compatibility.
2018-10-24 10:47:12 +02:00
Matthieu Gautier fe6d5fa93e Store the downloadId in the book (and in the library). 2018-10-24 10:47:12 +02:00
Matthieu Gautier 43ff8565d1 Add a Download class to encapsulate a aria2 download. 2018-10-24 10:47:12 +02:00
Matthieu Gautier f718c4c472 Add a LibraryManipulator.
Library client (kiwix-desktop) need to know when a book is added to
library by the manager. By using a LibraryManipulator, we can do
dependency injection.
2018-10-24 10:47:12 +02:00
Matthieu Gautier 8176a6eded Be more resilient to potential aria2 error. 2018-10-24 10:47:12 +02:00
Matthieu Gautier bb1f777078 Store the aria2 session and recover from it. 2018-10-24 10:47:12 +02:00
Matthieu Gautier 829c34dd69 Store in the book instance if the given path is valid.
The path may exist and not be valid if the zim file is not truncated
(ie, interrupted download)
2018-10-24 10:47:12 +02:00
Matthieu Gautier 9c0f9696ed Better beautifyInteger and beautifyFileSize. 2018-10-24 10:47:12 +02:00
Matthieu Gautier be6dc01b4f Add few helper methods to xmlrpc objects. 2018-10-24 10:47:12 +02:00
Matthieu Gautier 18fc5cb4df Correctly set the aria2 secret rpc. 2018-10-24 10:47:12 +02:00
Matthieu Gautier 996829e4d7 Allow a OPDSDumper to dump only a subset of the library. 2018-10-24 10:47:12 +02:00
Matthieu Gautier 5128861136 Remove default value for book pointer of `readBookFromPath`.
This is a nonsense to accept NULL pointer here.
2018-10-24 10:47:12 +02:00
Matthieu Gautier 7804bf2276 Reimplement listBooksIds.
No real improvement.
2018-10-24 10:47:12 +02:00
Matthieu Gautier 99e313f915 Clean includes of manager.h 2018-10-24 10:47:12 +02:00
Matthieu Gautier 839320d5e7 Move the `Book` class in its own source file. 2018-10-24 10:47:12 +02:00
Matthieu Gautier 1e8f85eaff Rename methods `title()` into `getTitle()`.
Same for all attributes.
2018-10-24 10:47:12 +02:00
Matthieu Gautier e0704b3b21 Move the initialization code of a book from xml|opds into Book. 2018-10-24 10:47:12 +02:00
Matthieu Gautier 57fbb98bca Do not store the favicon base64 encoded in the book.
The fact that the favicon is base64 encoded in a storage detail.
2018-10-24 10:47:12 +02:00
Matthieu Gautier c7f9218350 base64_encode takes a string instead of a char* 2018-10-24 10:47:12 +02:00
Matthieu Gautier 66a9a69480 Move the code updating a book from a reader in the Book class. 2018-09-06 18:30:37 +02:00
Matthieu Gautier 04b05dd68b Remove `removeBookById` from the Manager.
Use the same method of the `Library`.
2018-09-06 18:30:37 +02:00
Matthieu Gautier aa6772b345 Remove the "last" book functionnality.
- This is not used by any application.
- This is application specific and should not be stored in the library
  (who is a list of book).
2018-09-06 18:30:37 +02:00
Matthieu Gautier efae3e0d2f Do not make the `Manager` responsible to create the `Library`.
The `Manager` manage a library already existing.
This avoid the Library clone stuff.
2018-09-06 18:30:37 +02:00
Matthieu Gautier bba3c252e4 Make the member of the book protected.
It is up to the book to manage its attribute.

Also remove the `absolutePath` (and `indexAbsolutePath`). The `Book::path` is always stored
absolute.
The fact that the path can be stored absolute or relative in the
`library.xml` is not relevant for the book.
2018-09-06 18:30:37 +02:00
Matthieu Gautier 57ac6f0305 Use a map to store the Library's books.
Having the books sorted is useless.
We handle books by id not by index.
2018-09-06 18:30:37 +02:00
Matthieu Gautier 541fb0cfd1 Remove the "current" book functionnality.
- This is not used by any application.
- This is application specific and should not be stored in the library
  (who is a list of book).
2018-09-06 18:30:37 +02:00
Matthieu Gautier c9eac04050 Make the Library`s book vector private.
Move a lot of methods from Manager to Library. Because books is private
and thoses methods are better in Library.
2018-09-06 18:30:37 +02:00
Matthieu Gautier 741c67786a Add `update` method to `Book`. 2018-09-06 18:30:37 +02:00
Matthieu Gautier db9000f706 Make the downloader use the aria2c wrapper instead of the aria2 library. 2018-09-06 18:30:34 +02:00
Matthieu Gautier 0a93cb0872 Add aria2 downloader using subprocess aria2c. 2018-09-06 18:29:49 +02:00
Matthieu Gautier f4846c1ac8 Add a tool's function to get the data directory.
The data directory is where kiwix application should store data.
2018-08-29 15:28:52 +02:00
Matthieu Gautier 9b516ac35d Add a small wrapper around pugixml do handle xmprpc. 2018-08-29 15:28:52 +02:00
Matthieu Gautier 79b780b75b Move the function to convert from xml_node to string in otherTools.
This can be usefull elsewhere than in opds_dumper
2018-08-29 15:28:52 +02:00
Matthieu Gautier f3dd83907d Add backend to launch subprocess.
The windows backend is not tested.
2018-08-29 15:28:52 +02:00
Matthieu Gautier 7c634738dd Update jni build script to java jdk 8+.
With jdk8, `javac` has an option `-h` to generate the header files of
native classes.
So there is no need to run `javah` several times.

As there is now only one command to run (`javac`), there is no need for
the wrapper script `gen_kiwix.sh`.

Fix #167
2018-08-20 12:17:29 +02:00
Matthieu Gautier fd9b6569af Include unistd.h only on unix platform. 2018-08-03 19:22:39 +02:00
Matthieu Gautier 3cf58b5f5b Make libaria2 an optional dependency.
We don't compile libaria2 on Windows.
2018-08-03 19:22:39 +02:00
Matthieu Gautier dbcc9140b9 [Android] Better error message when failing to read in zim file.
Let's print the exception's message to allow us to better understand
what went wrong.
2018-07-31 11:49:44 +02:00
mhutti1 d61580f599 Set JNI values to NULL on error 2018-07-27 10:11:24 +02:00
mhutti1 3227b29c90 Follow redirects in favicons 2018-07-27 10:11:24 +02:00
mhutti1 709baae934 Convert all JNI cerrs to Android log messages 2018-06-29 16:35:11 +02:00
mhutti1 ea8cd9f1a9 Correctly pass 0 through JNI if ZIM file is corrupted 2018-06-29 15:30:13 +02:00
Matthieu Gautier 5025ee4963 Fix icudt version.
We have move to icu version 58 a while ago.
2018-06-11 14:38:32 +02:00
Matthieu Gautier 9aaf82a36d Explicitly use icu namespace.
Fix #145
2018-06-11 14:36:34 +02:00
mhutti1 fa99cce68d Decode reserved characters in URLs 2018-05-20 18:41:39 +01:00
Matthieu Gautier 622d2fc23d Remove unused static resources.
Stop words are not use anymore since a long time now that indexing has
been moved to libzim. No need to embedded them in kiwix-lib.
2018-05-15 11:30:30 +02:00
Matthieu Gautier c0b1c6013e Fix parsing of url
Fix kiwix/kiwix-tools#193
2018-05-14 17:41:05 +02:00
Matthieu Gautier 1dd828e79c Fix pathExists and check for correct path for xapian index.
The correct path for xapian database should be "X/fulltext/xapian",
not "Z//fulltextIndex/xapian".

So lets check for the right path and fallback to the wrong one (but
used in old zims).

The double '/' in the path is a bug of zimwriterfs and is specific
to the xapian database.
We must handle this correctly in `hasFulltextIndex` and not (buggly) in
`pathExists`.
(Hopefully, it seems that pathExists were used only by hasFulltextIndex)
2018-04-19 18:04:07 +02:00
Matthieu Gautier 135028c16a Introduce better API to manipulate entries in a zim file.
The previous API suffer different problems:
- It was difficult to handle articles redirecting to other article.
- It was not possible to get few information (title) without getting
  the whole content.

The new API introduce the new class `Entry` that act as a proxy to an
article in the zim file.

Methods of `Reader` now return an `Entry` and the user has to call
`Entry`'s methods to get useful information.
No redirection is made explicitly.
If an entry is not found, an exception is raised instead of returning
an invalid `Entry`.

The common pattern to get the content of an entry become :

```
std::string content;
try {
  auto entry = reader.getEntryFromPath(path);
  entry = entry.getFinalEntry();
  content = entry.getContent();
} catch (NoEntry& e) {
  ...
}
```

Older methods are keep (with the same behavior) but are marked as
deprecated.
2018-04-19 18:04:07 +02:00
Matthieu Gautier 47ce044e3e Add method to `Manager` to populate the library from a opds stream.
The library's books are created in the metadata in the opds.
As the opds stream is by definition a distant "library", there is no
zim to read to complete missing information.

This can lead to incomplete `library.xml`.
2018-04-19 17:53:08 +02:00
Matthieu Gautier 1f091da3f4 Add a downloader tools to download files.
The downloader is using libaria2.

For now, only one download can be run a the time.
A download will start only if (and as soon as) no download is running.
2018-04-19 17:53:08 +02:00
Matthieu Gautier d4fefd1a57 Add a function to create a temporary directory. 2018-04-19 17:53:05 +02:00
Matthieu Gautier 9f86b59d1d Add a function to get the content of a file. 2018-04-19 17:53:02 +02:00
Matthieu Gautier 2164faba44 Add a potential search description link in the opds stream. 2018-04-19 17:08:01 +02:00
Matthieu Gautier ad92af928b Be able to filter a library.
This generate a new library only with the corresponding books.
2018-04-19 17:08:01 +02:00
Matthieu Gautier ee51c470b4 Allow the manager to dump the opds feed of the whole library. 2018-04-19 17:08:01 +02:00
Matthieu Gautier ec097ab267 Try to compile kiwix-lib without warning. 2018-04-18 16:57:27 +02:00
Matthieu Gautier d686de7ec3 Set the RPATH of kiwix-lib.
As we cannot change (DY)LD_LIBRARY_PATH on macos, we have to use rpath.
2018-04-17 16:27:31 +02:00
Matthieu Gautier 313f6731b0 Correctly pre-increment loopCounter.
If we check the later the `loopCounter` with 42, we must pre-increment the
content. Else, in case of infinite loop, the `loopCounter` will be 43.

Related to kiwix/kiwix-tools#168
2018-03-25 17:21:40 +02:00
Matthieu Gautier ee6831d665 Check `internal->_search` before using it.
If a search has been set and a user try to get the nextResult or
restart the search, `internal->_search` will be NULL.
2018-03-12 17:45:18 +01:00
Matthieu Gautier f8a2e4c503 Only add a reader to the searcher if the reader as fulltext index.
`libzim` will not search in zim file without embedded fulltext index.
If we don't want to mess up with result index, we must not store "wrong"
reader.

Fix #111
2018-03-12 17:34:45 +01:00
Matthieu Gautier 57a197d38d Make getCurrentBookId const. 2018-03-12 17:34:45 +01:00
Matthieu Gautier b6ba10af2a Remove unnecessary currentArticleOffset.
This protected member is never used.
2018-03-12 17:34:45 +01:00
Matthieu Gautier f93f50087b Remove unnecessary setBookIndex.
We can use default argument instead of creating a new method.
2018-03-12 17:34:45 +01:00
Matthieu Gautier 6adf95c329 Make JNIKiwixReader throw an exception if something goes wrong at creation.
If the `nativeHandle` is null, the JNIKiwixReader is invalid and we must
not use it.

Throwing an exception for the caller code to handle this properly.
And previously, user code has no way to detect something went wrong :/
2018-02-01 17:18:54 +01:00
mhutti1 97bcf57d53 Allow JNI to access search snippets 2017-12-15 16:02:49 +00:00
mhutti1 f303c7502d Fix JNI to work with kiwix-android 2017-12-14 17:32:03 +00:00
Matthieu Gautier 16bd34e6a6 Add a method in the JNI API to get direct access information.
For binary content (not compressed), it could be interesting to
directly read the content in the zim file instead of using `kiwix-lib`.

This method returns the needed information to do so (if possible).
2017-12-13 17:22:26 +01:00
Matthieu Gautier 5a953f191b Remove a small warning. 2017-12-13 17:11:10 +01:00
Matthieu Gautier 41c92cfc3c Better calculate the start of the last search page.
The increment between pages should always be a multiple of
`resultCountPerPage`.
2017-11-27 12:39:04 +00:00
Matthieu Gautier 64dc5131c0 Be able to specify the global contentHumanReadableId without a index.
Even if we use the add_reader method to search into embedded full text
index, we need to specify the global `contentHumanReadableId` as it will
be used to generate "page links".
2017-11-27 12:37:13 +00:00
kelson42 28b0588df4 Better URL encoding 2017-11-23 19:26:41 +01:00
Matthieu Gautier 4e5d9f0360 Add a API to get only a part of a article content.
Add the jni method `getContentPart` to get only a part of the artcicle
content.

The method can be used to get a part of the content or to know the size
of the full content.
2017-11-22 19:06:54 +00:00
mhutti1 520c1edf31 Fix JNI android integration 2017-11-22 14:54:03 +00:00
mhutti1 d2f7503cfa Fix JNI for android integration 2017-11-22 14:54:03 +00:00
Matthieu Gautier 7a59779b77 Change JNI API of kiwix-lib (mainly to support multi-zims search)
This is a major API break. User code will have to be rewritten.

Before this commit, API was a unique object wrapping the library and
handle a global state with one `Reader` and one `Writer` at the time.

Now, the API is axed around three main objects :
 - The `JNIKiwixReader`, a wrapper around a `kiwix::Reader` (who allow to
   read one zim)
 - The `JNIKiwixSearcher`, a wrapper around a `kiwix::Searcher` (who allow
   to search through one or more reader(s))
 - The `JNIKiwixSearcher.Result` a result of a search. Allowing to get all
   information about a result (title, url, content, snippet, ...)
2017-11-22 14:54:03 +00:00
Matthieu Gautier 766b64dddc Update gen_kiwix.sh to not be dependent of the number of arguments. 2017-11-22 14:46:01 +00:00
Matthieu Gautier b9ac7084ac Add small API to do geo query.
This is a small quick and dirty API to do geo query.

It is not possible with this API to do a query search and a geo search.
It's either one or the other.

We should think about a better global API to do searching and provide
both of them in the same time (libzim does it).
2017-11-14 17:32:06 +01:00
Matthieu Gautier 0e8c8f68c5 Extend search limits to 140.
70 is a too small limit for the number of results.
Users need at least 100.

As the html rendering will fails with more than 144 results,
explicitly limits the number of search to 140.

Fixes kiwix/kiwix-tools#92
2017-11-06 12:23:13 +01:00
Matthieu Gautier 382655d83c Explicitly set ctpp2 iIMaxSteps to extends search beyond 68 results.
Ctpp2 templates have a limit step number. If the template to render is
too big, the rendering fails, throwing an exception.

From our tests, it seems that, with the template we have, the default
step limit allow us to render 68 results only.

By doubling the limit, we can render up to 144 results.
2017-11-06 12:10:48 +01:00
Matthieu Gautier 5c2280e7c7 Claims that multi part zim has no embedded full text index.
We cannot search into an embedded fulltext index if the zim is multipart.
Instead of crashing, let's pretend we have no fulltext index.
2017-08-15 14:19:53 -04:00
Chris Li cf93c8719f Allow kiwix-lib to compile without ctpp2c.
ctpp2c is used to pre-compile the template resource.
However, on OSX, ctpp2c seems to be difficult to compile, as we don't need
ctpp2 at all on OSX/iOS, lets just stop to force the use of ctpp2c.
2017-08-14 10:42:16 -04:00
Matthieu Gautier 1ff1bf6168 Always set the humanReadableName with the readable in kiwix-search.
We always need a humanReadableName associated with a content to search in.
Do not separate the two values (human readable name and zim) in two
different functions.

This way, we avoid miss-use of the Searcher who could lead to segfault.
2017-08-10 09:20:11 -04:00
Matthieu Gautier d3acae1fd2 getContent* methods also allow to get the title.
Add a `title` write argument to `getContent*` methods.
This argument is filled with the title of the content get.

Also update the JNI accordingly.

Related to kiwix/kiwix-android#214
2017-07-26 11:03:32 +02:00
kelson42 1d1dfbf4da Fix JNI licensing #71 2017-07-22 09:29:28 +02:00
Matthieu Gautier bc5f4f5de4 Use right contentId to generate the article url in search template.
As we do multisearch, we must use the associated contentID of the result
to generate the url.
2017-07-18 10:04:40 +02:00
Matthieu Gautier 9cc329dbd2 Support multi-zims search in kiwix-lib.
All the code was already in zimlib.
It is mainly a update of the code using zimlib.

No JNI change for now to not break the API.
2017-07-18 10:04:40 +02:00
Matthieu Gautier 3991e648ed Be able to get the reader index from a search result. 2017-07-17 18:16:11 +02:00
Matthieu Gautier 8d39b0b343 Search result objects now have a get_content method.
This was not necessary when searching in only one zim file as `url` was
enough to get the article (and so the content).

If we want to search in several zim in the same time, we need a way to get
the content directly.
2017-07-17 18:16:11 +02:00
Matthieu Gautier 4a51dd9e00 Fix memory link.
If a `searcher` is already created we must delete it.
If we set the pointer to NULL before, we will never delete it.
2017-07-17 18:16:11 +02:00
Matthieu Gautier d0371cd133 Suggestions now use xapian database when available.
If a embedded fulltext database is present, suggestion will search in it :
 - insensitive case search.
 - search for terms in the middle of the title.
 - xapian will try to complete the last word of the query (as if a '*'
   were added at the end)
2017-07-17 17:17:13 +02:00
Matthieu Gautier f76e9d2dbf Format all the code using clang-format.
Add a script `format_code.sh` to easily format the code.
2017-07-05 15:22:34 +02:00
Matthieu Gautier 96f199a327 Dump the version to 0.2.0
Time to make a release.
2017-06-27 14:26:13 +02:00
Steve Wills 4f57e765e5
Fix type error in build
Compilation fails on clang 3.4.1 (and presumably later, tho I haven't tested) with

```
src/reader.cpp:131:59: error: no viable conversion from 'iterator' (aka '__map_iterator<typename __base::iterator>') to 'std::map<std::string, unsigned int>::const_iterator' (aka '__map_const_iterator<typename __base::const_iterator>')
      std::map<std::string, unsigned int>::const_iterator it = counterMap.find("text/html");
                                                          ^    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/v1/map:713:29: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'iterator' (aka '__map_iterator<typename __base::iterator>') to 'const std::__1::__map_const_iterator<std::__1::__tree_const_iterator<std::__1::__value_type<std::__1::basic_string<char>, unsigned int>, std::__1::__tree_node<std::__1::__value_type<std::__1::basic_string<char>, unsigned int>, void *> *, long> > &' for 1st argument
class _LIBCPP_TYPE_VIS_ONLY __map_const_iterator
                            ^
```

because we are not using the right type for the map iterator. As we are using
C++11, let's use `auto` and make compiler set the right type for us.
2017-06-16 08:47:59 -04:00
Matthieu Gautier eb2c750431 Use the android abi instead of cpu_family as install dir name.
Android will look in specific repository to find native libs.
We need to use the `android_abi` name defined in the cross_compilation
file instead of `cpu_family`.
2017-06-14 10:20:18 +02:00
Matthieu Gautier c44b2acb56 Re-add xapian searcher in kiwix-lib.
libzim only know how to read embedded full text index in a zim file.
This is nice as we want to embedded the full text index in zim file and
not have separated full text index.

However, we still have some zim+separated index we have to read.
So we have to support the search in separated index for a while.
2017-05-24 16:08:00 +02:00
Matthieu Gautier 7005b65901 Check that 'M/Counter' exists before trying to read it.
Some old zim files may not have a 'Counter` metadata article.
We have to handle this correctly.
2017-05-23 15:26:01 +02:00
Matthieu Gautier 2e43b7e82d Update kiwix-lib to new kiwix-android way of building.
`kiwix-android` is using `kiwix-lib` as an external java application now.
So we need `kiwix-lib` build system to also install application files
(manifest, resources, ..).
2017-04-24 16:37:31 +02:00
Matthieu Gautier 3be4d92c53 Correctly check if we are compiling for linux or not.
In C++11 `linux` is not a reserved word, so compilators do not define it.
A correct way to check if we are compiling for linux is to check for
`__linux__`.
2017-04-10 14:28:25 +02:00
Matthieu Gautier 44a77f5846 Update android jni wrapper to new API. 2017-04-10 14:28:25 +02:00
Matthieu Gautier 5ca419bee7 Use the new search API in zimlib.
We do not use xapian anymore. This is all handled by zimlib.
2017-04-10 14:28:25 +02:00
Matthieu Gautier 37f29da63e Beautify a bit the code.
No real change. Just do less code or use higher level API.
2017-04-10 14:28:25 +02:00
Matthieu Gautier 94670847ef Use const when possible in the reader.
Most read operation do not modify the content. So let's use const
as far as possible.
2017-04-10 14:28:25 +02:00
Matthieu Gautier ef661a2e25 Move unicode headers in cpp.
Unicode headers ends by defining the DONE symbol in a enum.
It can clash with other includes.
(For instance the httpd.h from apache who use `#define DONE -2`).

Both project should not declare such common symbols publicly but we have
to do with them anyway.
2017-04-06 16:17:00 +02:00
Matthieu Gautier e28dbe7c7e Remove the indexer functionnality from kiwix-lib.
This is not used anymore.
2017-04-06 15:35:30 +02:00
Matthieu Gautier ce6c782b66 Do not use remove readStopWords method.
Commit b8d950c removes this symbol.
The indexer is not used anymore and will be soon removed.
So for now, just remove the call to readStopWords until we totally
remove the indexer code.
2017-04-06 13:20:59 +02:00
Matthieu Gautier b8d950c1a0 Use the stop words stored in the database to configure the queryparser.
To properly search in the xapian database, we need to use the same
stop words that the ones used during the indexing.
2017-04-04 17:06:49 +02:00
Matthieu Gautier 998db0eb2b Use the language stored in the database to configure the queryparser.
To properly search in the xapian database, we need a stemmer using the
same language that the one used during the indexing.
2017-04-04 17:06:49 +02:00
Matthieu Gautier 72e41082ca The `Result` class is not in the `kiwix` namespace.
The commit 83d2725 adapt the jni wrapper to the new search API but try to
use the `Result` class from the `kiwix` namespace but `Result` is not in
the namespace.

A correct fix would be to move `Result` in `kiwix` but it also change the
API for other tools (kiwix-tools). As we will move the search
functionality in `zimlib` it is better to just do this silly fix and
update the API latter when moving the search functionality.
2017-03-29 17:12:23 +02:00
Matthieu Gautier cecb65e314 Remove C++11 syntax introduced by commit 9be2abe.
The `for( auto elem: elems)` syntax is a C++11 syntax.
We are not using C++11 (even if it would be good idea).
This works on recent compiler (on Fedora 25) but fails on older one
(on Travis).
2017-03-28 17:14:25 +02:00
Matthieu Gautier 074c1bcffa Try to generate the snippet if it is not present in the database.
We generate the snippet from the content of the article in the zim so
we need to have a access to the reader.
2017-03-21 16:28:03 +01:00
Matthieu Gautier 9be2abedf3 Check if a valuemaps metadata is available in the database and use it.
This way, we do not make assumption of where the values are stored.
2017-03-21 16:26:03 +01:00
Matthieu Gautier 83d27255cf Do not create all the results at once. Be a bit lazy.
We don't need to generate a vector of result when we do a search.
We better to just keep the handle to the current MSetIterator and
generate the wanted values when needed.
2017-03-21 16:20:17 +01:00
Matthieu Gautier 72a6b578e6 Fix meson's project version. 2017-03-21 16:01:07 +01:00
Matthieu Gautier 43703a9d58 Add JNI wrapper for android.
This wrapper mainly come from the already existing android part.
2017-03-07 20:14:29 +01:00
Matthieu Gautier aafe9a4435 [ANDROID] Deactivate some features if we are compiling for android.
Android libc doesn't support all thread feature (as pthread_cancel).
Do not compile those files if we are compiling for android.
2017-02-22 16:56:21 +01:00
Matthieu Gautier 84796abb4c Make use of ctpp2 optional. 2017-02-22 12:05:44 +01:00
Matthieu Gautier 72f1be3dbd We must include <time.h> with mingw.
This is a error anyway, linux or window. So just add the include.
2017-01-17 15:54:44 +01:00
Matthieu Gautier 753a39beb0 Fix header include for cross-compilation to Windows.
On unix, filenames are case sensitive and all include files are lowercase.
2017-01-17 10:20:59 +01:00
Matthieu Gautier 4afe3e8ddc "myhtmlparse.h" is a private header. Remove it from indexer.h. 2017-01-03 11:51:31 +01:00
Matthieu Gautier cba71b4e75 Add a new script to compile resources.
- No more dependency to reswrap binary (everything is done in python)
- Resource strings can be directly accessed.
  As side effect, it add a check at compilation if the resource is
  declared and compiled in the binary.
- The resource content can be overwritten at runtime with a env variable.

There is also few clean in the static as some files shoul be in the tools
directory.

The compile_resource script is install to let other project use it.
2016-12-23 13:06:12 +01:00
Matthieu Gautier 8ce1fb0ba8 Switch build system to mesonbuild.
There is no more integrated build of dependencies in the build system.
Dependencies are discovered using pkg-config except for ctpp2 where there
is no pkg-config file.
2016-12-22 12:01:20 +01:00
Kelson 1c68cf87b9 Fix ft index open for chunked files #346 2016-10-17 13:16:43 +02:00
Kelson d328c4ffb0 Remove exec permission on source files 2016-10-14 17:04:47 +02:00
Kelson 75da598ba8 Add kiwix::removeAccents for Android 2016-10-14 16:59:32 +02:00
Kelson 2889d7c651 Make fulltext searcher Android compatible 2016-10-14 13:12:26 +02:00
Kelson 01794e6e91 New Reader::getZimFilePath() 2016-10-14 13:11:14 +02:00
Matthieu Gautier 82cba79179 Do not make unnecessary copy when serving binary content.
Binary content do not need to be modified, so we don't need to copy it.
We can directly serve it from the internal zim (cluster) buffer.

The handle_content function now getArticleObjectByDecodedUrl instead of
getContentByDecodedUrl.

This is to get the mimetype of the article and copy the content only when
needed (getContentByDecodedUrl always copy the content).
Thus, handle_content is a bit more complex as it need to do some
manipulation previously made in getContentByDecodedUrl.

The main change is that if the content is binary, we serve the content
with a callback response who will get the content chunks directly from
the blob buffer.
2016-10-07 18:23:47 +02:00
Matthieu Gautier 657fc05225 Fix url of the fulltext index.
The fulltext index in the zim is at the url /Z/fulltextIndex/xapian.
We do not need to specifie the Z in the url as this is automatically add with the namespace.
2016-10-03 15:39:00 +02:00
Matthieu Gautier e526026407 Properly fail when creating XapianSearcher on a zim without embedded index.
The XapianSearcher creation must fail with a exception if we cannot open
the xapian database.
So, if we try to open a zim and there is no embedded index, we must fail.

We raise the custom exception NoXapianIndexInZim in this case.
2016-10-03 13:32:59 +02:00
kelson42 e3c2a13fa6 Add Reader::hasFulltextIndex 2016-10-02 18:29:08 +02:00
kelson42 39ad3313df Add Reader::urlExists 2016-10-02 17:16:39 +02:00
kelson42 c1b5eb42bf Fix a problem with 'name' metatag 2016-09-11 10:26:12 +02:00
kelson42 934a15a0b4 Fix kiwix-mange problem with relative paths 2016-09-04 21:43:26 +02:00
Kelson 1500cb8329 Implement ZIM reader getTags() and getName() 2016-09-04 20:41:10 +02:00
Kelson fc9f3a3a25 Implement ZIM reader getTags() and getName() 2016-09-04 19:59:22 +02:00
Matthieu Gautier 6074b5e557 Add support of embedded ft-index in zim for kiwix-serve and kiwix-search.
- kiwix-search will try the embedded ft-index if no indexPath is specified
  in the library.xml or command argument.
- kiwix-search also uses xapianSearcher and xapianSearcher needs zimlib,
  so add zimlib in compilation flag for searcher.
- kiwix-serve already depends on zimlib.
2016-08-11 17:50:25 +02:00
kelson42 d384131341 remove useless code 2016-07-04 21:31:11 +02:00
Chris Li 801e622644 add "eng" to stop word map + pragma mark in indexer.cpp + progress calculation 2016-07-04 21:10:03 +02:00
Chris Li 566a01ce7f optional progress callback + progress calculation fix 2016-07-04 20:59:57 +02:00
Kelson d5e5cd9340 Deal with new glass/single_file Xapian indexes 2016-06-20 23:35:13 +02:00
dattaz 8c443855c7 correction of double inclusion 2016-05-21 19:08:44 +02:00
Emmanuel Engelhart 009d08fe89 Simplify suggestion init 2015-08-22 15:42:21 +02:00
Emmanuel Engelhart a1486e05d3 Fix regression by suggestion search 2015-08-16 19:22:10 +02:00
Emmanuel Engelhart f3c4556431 Fix to get new getSuggestions() working on Android 2015-08-16 18:09:18 +02:00
Emmanuel Engelhart e829d7428d Better suggestion search 2015-08-16 17:18:29 +02:00
kelson42 7903458e38 Fix hardlinking in copyFile() 2015-04-09 14:26:11 +02:00
Kelson42 54d9445148 Update 2015-03-02 16:17:06 +01:00
Emmanuel Engelhart dc60455cd3 in copyFile() try to make a link if possible, copy the content otherwise 2015-02-21 16:56:38 +01:00
Emmanuel Engelhart b2c38ebfc2 update 2015-02-19 16:43:02 +01:00
Emmanuel Engelhart 999a5b07ae New version 2015-02-19 16:20:49 +01:00
Kelson42 78db06e127 Merge branch 'master' of ssh://git.code.sf.net/p/kiwix/kiwix
Conflicts:
	CHANGELOG
2015-01-08 12:55:53 +01:00
Kelson42 8287a64172 FIXED: kiwix-serve XSS attack vulnerability (#763) 2015-01-08 12:51:42 +01:00
Kelson42 45ad751d89 New function getMimeTypeByUrl() 2014-12-08 01:37:06 +01:00
Kelson42 d7fd4064ab FIXED: Buggy pagination in fulltext result page 2014-11-24 20:52:46 +01:00
Kelson42 4290eaf751 + Avoid to load the welcome page by loading random ones 2014-10-18 18:14:46 +02:00
Julian Harty 378c66e16c Revert "Re move ANDROID pragma. This is not useful anymore (libicu) is now included"
This reverts commit 8a458417aa612409b8da9a44b9015786c2063244.
2014-10-08 10:44:55 +01:00
kelson42 f6f49a63ce Re move ANDROID pragma. This is not useful anymore (libicu) is now included 2014-10-01 20:26:43 +02:00
Kelson42 49d13f09f1 + search filter addtionaly on language ISO code 2014-09-23 13:20:46 +02:00
Kelson42 bc161034c1 + Fix unstable fulltext library search 2014-09-20 16:05:19 +02:00
Kelson42 517a02b814 + fix taskbar CSS 2014-09-18 14:54:49 +02:00
Kelson42 4beccfbc0b + improve taskbar CSS 2014-09-18 14:50:11 +02:00
Kelson42 ec6c0b2ea3 + fix kiwix-serve taskbar css 2014-09-18 14:22:47 +02:00
Kelson42 f6dedb11b9 + fix tasbar css 2014-09-18 14:17:56 +02:00
Kelson42 436d57976e + improve kiwix-serve taskbar css 2014-09-18 14:08:36 +02:00
Kelson42 d8a924121b + more robus mimetype retrieving 2014-09-14 18:40:39 +02:00
kelson42 c1a8b4a206 + small optim 2014-08-31 19:27:23 -06:00
kelson42 45fccb2e38 + bool Manager::setBookIndex(const string id, const string path) 2014-08-30 10:37:52 -06:00
kelson42 baf8e37b4f Speedup mimetype detection in kiwix-serve 2014-08-29 19:10:01 -06:00
kelson42 f585bed1cb FIXED: kiwix-serve crash if unattended search "start" & "end" values 2014-08-28 13:36:58 -06:00
kelson42 518ce6635d + fix a few problem with the autocompletebox on kiwix-serve 2014-08-27 21:08:14 -06:00
kelson42 1cc94e1b95 + simplifaction of the code = cleaning 2014-08-27 17:13:29 -06:00
kelson42 a353513c9c NEW: Improve file size display in kiwix-serve 2014-08-27 13:22:34 -06:00
Kelson42 86ef9800cc + new smaller version 2014-07-28 13:38:54 +02:00
kelson42 beae77db89 Avoid kiwix-serve taskbar jquery dependencies to conflict with potential other version (in the ZIM file) 2014-07-20 22:06:52 +02:00
kelson42 d6d7274b0f + Fix code in ZIM reader class breaking DOM integrity 2014-07-19 19:29:51 +02:00
kelson42 17724fe70b + fix Doctype HTML line (was breaking Opera output) 2014-07-09 21:11:34 +02:00
Cristian Patrasciuc 5a695963d5 Port networkTools.cpp to Mac OS X.
On Mac OS X the ioctl(SIOCGIFCONF) call will return all IPv4, IPv6
and MAC addresses, so we have to filter only the ones that have
sa_family equal to AF_INET.

I also added the common interface names for Mac OS X (en0 and en1),
to the getBestPublicIp() function. I refactored a bit the code so
that addition and reordering are easier.

Bugs: https://sourceforge.net/p/kiwix/bugs/658/
2014-06-17 17:46:54 +02:00
renaud gaudin 1b192c2557 Added 10.0.0.0 private network scheme 2014-05-31 13:53:46 +00:00
Shivam a37fd98585 Updated resourceTools.h to reflect moved toolbar css file (not sure if necessary) 2014-05-25 18:10:44 -04:00
Shivam b341b9654d Added a random article button for server mode 2014-05-25 12:25:13 -04:00
kelson42 0411b0988c + update license header 2014-05-22 19:08:10 +02:00
kelson42 e885cc342a + Avoid buffer overflow on linux 2014-05-22 19:05:42 +02:00
Kelson42 f69edcf9c0 + fix fileExists() for Windows 2014-05-19 23:55:56 +02:00
Kelson42 b3c352f71a + use _strdup on Windows 2014-05-16 15:06:06 +02:00
Kelson42 1faca99274 Merge branch 'master' of ssh://git.code.sf.net/p/kiwix/kiwix 2014-05-16 14:40:36 +02:00
Kelson42 2c530906c5 + XR29 port for Windows 2014-05-16 14:39:50 +02:00
rgaudin 38108f064e moved loadICUExternalTables code to stringTools so it can be used by kiwix-serve 2014-05-16 01:19:27 +00:00
Kelson42 c3ae9d06b1 + Windows fix 2014-05-14 18:43:49 +02:00
Kelson42 1209e27e8c + Fix path computation functions for Windows 2014-05-14 18:33:11 +02:00
rgaudin 9b32647fa4 Added ICU external dat table support for components (req. check for binaries) 2014-05-08 20:22:23 +00:00
rgaudin 8c574270cf merged with master on zimAccessor/VMStringLoader/configure.ac 2014-05-06 08:40:19 +00:00
kelson42 587b76bf6d + Fix deal with relative urls 2014-05-07 17:33:38 +02:00
kelson42 005b75f7ab + Introduction of base url in kiwix-serve (to handle relative urls in ZIM files) 2014-05-07 16:08:43 +02:00
rgaudin f1dc03d5aa removed duplicate type declaration on OSX (needed by c++11) 2014-05-06 08:09:25 +00:00
kelson42 4e05d0dea6 Change CTPP2 #include paths (add 'ctpp2/') 2014-05-05 14:14:11 +02:00
rgaudin 26b2e504f3 fixed includes for CTPP2 2014-05-04 20:11:48 +00:00
Kelson42 8573f9eda0 make kiwix::urlDecode() available for Android 2014-05-04 14:41:48 +02:00
kelson42 9a5e6d70d6 + Change license header 2014-05-03 18:00:54 +02:00
kelson42 1c2a03e287 + getContentByEncodedUrl() and getContentByDecodedUrl() 2014-04-29 17:42:46 +02:00
kelson42 8397c81e6f * merge unescapeurl() with the kiwix:urlDecode()
* create getContentByDecodedUrl() and getContentByEncodedUrl()
2014-04-29 17:41:52 +02:00
kelson42 5e974f67c6 + merge Reader::unescapeUrl() and Kiwix::urlDecode() 2014-04-29 17:40:12 +02:00
kelson42 3265de1579 + Use now kiwix::sleep(0 2014-04-02 17:04:19 +02:00
kelson42 69684009e3 + Add necessary header includes 2014-04-02 17:03:53 +02:00
kelson42 aeaede3834 + add kiwix::sleep() method 2014-04-02 16:55:35 +02:00
kelson42 ae390de24b + Stub class to store custom generic varia functions 2014-04-02 16:43:56 +02:00
kelson42 71082cf581 + Add indexer destructor (to avoid compilation warning with "non-virtual destructor") 2014-04-02 16:38:15 +02:00
kelson42 84b8a86c3e Try to fix some problem with unicode path loading on windows 2014-01-06 03:27:03 +01:00
kelson42 fc6254090a + remove clucene source files 2013-12-16 15:22:21 +01:00
kelson42 840442fccd + remove debug messages 2013-12-16 13:33:29 +01:00
renaud gaudin 6ed0acfde4 Merge branch 'master' into extern_zimlib 2013-12-09 12:07:38 +00:00
renaud gaudin fd9c9ac17e catching up with master 2013-12-09 12:02:58 +00:00
rgaudin fcf84811c6 Fixed extern compile (static) on linux and removed Clucene dependency 2013-12-09 10:51:06 +00:00
Kelson42 bf4381d4f4 Remove usage of UnicodeString.tempSubString() which is only available in recent versions of ICU 2013-11-24 13:22:05 +01:00
renaud gaudin 8bb7c040b1 Fixed CTPP2 loader to use resource instead of file 2013-11-14 18:50:26 +00:00
renaud gaudin 2a8d7fde56 Fixed CTPP2 not working
* Fixed templates
* Changed behavior of getHtml() to use higher level VM API
2013-11-13 19:58:56 +00:00
kelson42 ad20fde08f + android integrates now libicu 2013-11-12 02:03:34 +01:00
kelson42 39fa510af5 + improve suggestions search, search also entitled words 2013-11-10 17:54:11 +01:00
kelson42 987d6f672f + take care the nore than maxsuggestions is delivered 2013-11-10 17:42:52 +01:00
kelson42 2ad238f095 + fix kiwix:ucFirst and kiwix:lcFirst 2013-11-05 12:08:43 +01:00
kelson42 835388daf9 cleaner unescapeUrl() 2013-10-07 13:10:37 +02:00
kelson42 48fa26028f + fix small regression in the unescape_url() code 2013-10-06 20:58:20 +02:00
kelson42 696cfc90c3 + small beautifying of the code 2013-09-22 16:56:16 +02:00
kelson42 0bab890203 + fix small regression 2013-09-22 13:17:11 +02:00
kelson42 c9d1f562ff + fix a few typo/style stuff 2013-09-22 13:11:19 +02:00
Kiran Mathew Koshy 1299c4c264 wix::Manager::getBooksIDs() now returns diff files too. 2013-09-22 15:09:23 +05:30
Kiran Mathew Koshy 14a4394f6d Permanant fix for bug651.
Kiwix manager class is back to original state, additional function for obtaining origID has been added to Kiwix::Reader class.
Kiwix-manage and Kiwix-serve back to normal.
2013-09-20 18:35:20 +05:30
Kiran Mathew Koshy 2f89926616 Updated Kiwix::Manager class to accept origID as the last optional parameter, so as not to break existing tools which use Kiwix::Manager 2013-09-20 02:52:29 +05:30
Kiran Mathew Koshy c22b5a1d4c Rmoved title, favicon, faviconmimetype,description,author, publisher, date, language parameters from library file for diff files. 2013-09-18 15:55:29 +05:30
Kiran Mathew Koshy 7fbf20936e Updated kiwix::manager class to prevent it from returning a diff file as a normal book. 2013-09-13 14:59:44 +05:30