Common code base for all Kiwix ports
Go to file
Veloman Yunkan e3e4bfa533 Support for serving customized resources
During work on the kiwix-serve front-end, the edit-save-test cycle is
a multistep procedure:

1. build and install libkiwix
2. build kiwix-tools
3. run kiwix-serve
4. reload the web-page in the browser

When making changes in static resources that are served by kiwix-serve
unmodified, the steps 1-3 can be eliminated if kiwix-serve is capable of
serving resources from the file-system. This commit adds such a
functionality to kiwix-serve. Now, if during startup of kiwix-serve the
environment variable `KIWIX_SERVE_CUSTOMIZED_RESOURCES` is defined it is
assumed to point to a file where every line has the following format:

URL MIMETYPE RESOURCE_FILE_PATH

When a request is received by kiwix-serve and its URL matches any of the
URLs read from the customized resource file, then the resource data is
read from the respective file RESOURCE_FILE_PATH and served with
mime-type MIMETYPE.

Though this feature was introduced in order to facilitate the
development of the iframe-based content viewer, it can also be useful to
users who would like to customize the kiwix-serve front-end on their own
(without re-building all of kiwix-serve).

There is some overlap with a feature of the kiwix-compile-resources
script that also allows to override resources. The differences are:

1. The new way of customizing front-end resources has all such resources
   listed in a text file and there is a single environment variable
   from which the path of that file is read. kiwix-compile-resources
   associates a separate environment variable with each resource.

2. The new way uses regular paths to identify a resource. The
   kiwix-compile-resources method encodes the resource path by replacing
   any non-alphanumeric characters (including the path separator) with
   underscores (so that the resulting resource identifier can be used
   to construct the name of the environment variable controlling that
   resource).

3. The new method allows adding new front-end resources. The old method
   only allows to modify existing resources.

4. The new method allows (actually requires) to specify the URL at which
   the overriden resource should be served (similarly, the MIME-type can/must
   be specified, too). The old method only allows to override the contents of
   a resource.

5. The new method only allows to override front-end resources that are
   served without any preprocessing by kiwix-serve at runtime. The old
   method allows to override template resources as well (note that
   internationalization/translation resources cannot be overriden using the
   old method, either).
2022-06-22 10:59:41 +02:00
.github Fix CI for android 2022-06-15 14:36:02 +02:00
android-kiwix-lib-publisher New version 10.1.1 2022-04-11 17:13:58 +02:00
debian i18n data is kept in and generated from JSON files 2022-04-13 16:40:20 +02:00
docs [WIP] Remove pinning of Sphinx<4 2022-03-28 15:37:05 +02:00
include Render the search result using (opensearch/atom) xml format. 2022-06-03 15:46:41 +02:00
scripts Specify utf8 encoding when opening i18n resource file. 2022-05-17 18:36:35 +02:00
src Support for serving customized resources 2022-06-22 10:59:41 +02:00
static Update i18n_resources_list.txt 2022-06-13 18:46:29 +02:00
subprojects Make the gtest dependency optional. 2019-08-12 15:31:16 +02:00
test Dropped test/server_xml_search.cpp 2022-06-10 15:34:18 +02:00
.clang-format Format all the code using clang-format. 2017-07-05 15:22:34 +02:00
.codecov.yml Remove test directory from codecoverage 2019-09-26 12:00:33 +02:00
.gitignore Update .gitignore 2022-01-19 12:47:40 +01:00
AUTHORS Create AUTHORS file #48 2017-07-19 21:56:20 +02:00
COPYING Remove trailing spaces 2019-12-08 11:52:16 +01:00
ChangeLog New version 11.0.0 2022-06-15 11:49:03 +02:00
README.md Add documentation badge in README 2022-04-23 10:38:15 +02:00
format_code.sh Drop usage of Reader from InternalServer::handle_meta 2021-07-03 14:04:02 +05:30
kiwix.pc.in Fix version in pkg_config. 2017-10-23 15:20:56 +02:00
meson.build New version 11.0.0 2022-06-15 11:49:03 +02:00
meson_options.txt Remove the java wrapper. 2022-06-09 10:23:02 +02:00

README.md

Libkiwix

The Libkiwix provides the Kiwix software suite core. It contains the code shared by all Kiwix ports (Windows, GNU/Linux, macOS, Android, iOS, ...).

Release Repositories Build Status Doc CodeFactor Codecov License: GPL v3

Disclaimer

This document assumes you have a little knowledge about software compilation. If you experience difficulties with the dependencies or with the Libkiwix compilation itself, we recommend to have a look to kiwix-build.

Preamble

Although the Libkiwix can be (cross-)compiled on/for many sytems, the following documentation explains how to do it on POSIX ones. It is primarly thought for GNU/Linux systems and has been tested on recent releases of Ubuntu and Fedora.

Dependencies

The Libkiwix relies on many third party software libraries. They are prerequisites to the Libkiwix compilation. Following libraries need to be available:

  • ICU (package libicu-dev on Ubuntu)
  • ZIM (package libzim-dev on Ubuntu)
  • Pugixml (package libpugixml-dev on Ubuntu)
  • Mustache (Just copy the header mustache.hpp somewhere it can be found by the compiler and/or set CPPFLAGS with correct -I option). Use Mustache version 4.1 or above.
  • Libcurl (libcurl4-gnutls-dev, libcurl4-nss-dev or libcurl4-openssl-dev on Ubuntu)
  • Microhttpd (package libmicrohttpd-dev on Ubuntu)
  • Zlib (package zlib1g-dev on Ubuntu)

To test the code:

The following dependency needs to be available at runtime:

  • Aria2 (package aria2 on Ubuntu)

These dependencies may or may not be packaged by your operating system. They may also be packaged but only in an older version. The compilation script will tell you if one of them is missing or too old. In the worse case, you will have to download and compile bleeding edge version by hand.

If you want to install these dependencies locally, then use the libkiwix directory as install prefix.

Environment

The Libkiwix builds using Meson version 0.45 or higher. Meson relies itself on Ninja, pkg-config and few other compilation tools.

Install first the few common compilation tools:

These tools should be packaged if you use a cutting edge operating system. If not, have a look to the Troubleshooting section.

Compilation

Once all dependencies are installed, you can compile the Libkiwix with:

meson . build
ninja -C build

By default, it will compile dynamic linked libraries. All binary files will be created in the build directory created automatically by Meson. If you want statically linked libraries, you can add --default-library=static option to the Meson command.

Depending of you system, ninja may be called ninja-build.

The android wrapper uses deprecated methods of libkiwix so it cannot be compiled with werror=true (the default). So you must pass -Dwerror=false to meson:

meson . build -Dwrapper=android -Dwerror=false
ninja -C build

Testing

To run the automated tests:

cd build
meson test

Installation

If you want to install the Libkiwix and the headers you just have compiled on your system, here we go:

ninja -C build install

You might need to run the command as root (or using sudo), depending where you want to install the libraries. After the installation succeeded, you may need to run ldconfig (as root).

Uninstallation

If you want to uninstall the Kiwix library:

ninja -C build uninstall

Like for the installation, you might need to run the command as root (or using sudo).

Troubleshooting

If you need to install Meson "manually":

virtualenv -p python3 ./ # Create virtualenv
source bin/activate      # Activate the virtualenv
pip3 install meson       # Install Meson
hash -r                  # Refresh bash paths

If you need to install Ninja "manually":

git clone git://github.com/ninja-build/ninja.git
cd ninja
git checkout release
./configure.py --bootstrap
mkdir ../bin
cp ninja ../bin
cd ..

Custom Index Page

to use custom welcome page mention customIndexPage argument in kiwix::internalServer() or use kiwix::server->setCustomIndexTemplate(). (note - while using custom html file please mention all external links as absolute path.)

to create a HTML template with custom JS you need to have a look at various OPDS based endpoints as mentioned here to load books.

To use JS provided by kiwix-serve you can use the following template to start with ->

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width,initial-scale=1" />
    <title><-- Custom Tittle --></title>
    <script
      type="text/javascript"
      src="{{root}}/skin/jquery-ui/external/jquery/jquery.js"
    ></script>
    <script
      type="text/javascript"
      src="{{root}}/skin/jquery-ui/jquery-ui.min.js"
    ></script>
    <script src="{{root}}/skin/isotope.pkgd.min.js" defer></script>
    <script src="{{root}}/skin/iso6391To3.js"></script>
    <script type="text/javascript" src="{{root}}/skin/index.js" defer></script>
  </head>
  <body>
  </body>
</html>
  • To get books listed using index.js add - <div class="book__list"></div> under body tag.
  • To get number of books listed add - <h3 class="kiwixHomeBody__results"></h3> under body tag.
  • To add language select box add - <select id="languageFilter"></select> under body tag.
  • To add language select box add - <select id="categoryFilter"></select> under body tag.
  • To add search box for books use following form -
        <form id='kiwixSearchForm'>
        <input type="text" name="q" placeholder="Search" id="searchFilter" class='kiwixSearch filter'>
        <input type="submit" class="searchButton" value="Search"/>
        </form>
    

If you compile manually Libmicrohttpd, you might need to compile it without GNU TLS, a bug here will empeach further compilation otherwise.

If the compilation still fails, you might need to get a more recent version of a dependency than the one packaged by your Linux distribution. Try then with a source tarball distributed by the problematic upstream project or even directly from the source code repository.

License

GPLv3 or later, see COPYING for more details.