added mips32r2_uclibc_gclibcxx_dyn, (preferred atm, tested on prod hw) mips32r2_uclibc_gclibcxx_static targets (shared) and renamed previous uclibc ones to mips32r2_uclibc_uclibcxx_dyn, mips32r2_uclibc_uclibcxx_static for clarity (i.e. non-shared, pure ones) reworked builder classes to use inheritance of properties and methods (instead of copying boiler plate code) mips32r2_uclibc_gclibcxx_dyn target compiles and tested to run on production targets such as avm routers modified with a freetz env. See cm8/freetz@41d97c3789 for one of many possible projects to build a working toolchain with. In short - git clone - umask 0022 - make menuconfig (choose expert, disable toolchain download and let the toolchain/make scripts built a gcc-5.x one, do not forget to set FREETZ_LIB_libuClibc__WITH_WCHAR=y) - read the commit message for further info on long double math peculiarities - tested here with 0.9.33.2 Remember that swap will need to be running on the box, or else kiwix-serve is likely to quit with "invalid lzma stream in cluster" errors (if memory is too low). If the box lacks library support such as libstdc++.so*, you can copy them from the target toolchain libdir over to BUILD_mips32r2_uclibc_gclibcxx_dyn/INSTALL/lib if there are unsatisfied dependencies at runtime. (Which libraries need to be supplemented this way depends on your firmware and/or freetz configuration). Another issue is the execution in non-standard installation directories. LD_LIBRARY_PATH env needs to point to "our" lib directory. If you tar INSTALL/ dir, transport the result to an embedded device, untar and wan't to run from there, it is best to wrap all the elf binaries with a shell script that correctly sets LD_LIBRARY_PATH. This step has been automated, but needs testing, see kiwixbuild/patches/fixenv-run-in-nonstd-installdir.sh for details. This fixenv script is copied to INSTALL/bin during the build and it should be run on the box, if kiwix-tools is to reside in nonstd location (i.e. if files are not installed or installable to /bin, /lib or their usr/ pendants). Feel free to improve on automation of the necessary setup steps to make mips port build and deployment easier. |
||
---|---|---|
kiwixbuild | ||
scripts | ||
travis | ||
.gitignore | ||
.travis.yml | ||
LICENSE | ||
MANIFEST.in | ||
README.md | ||
build_custom_app.pl | ||
build_custom_app.py | ||
kiwix-deploy.py | ||
requirements_build_custom_app.txt | ||
setup.py |
README.md
Kiwix is an offline reader for web content. It's especially thought to make Wikipedia available offline. This is done by reading the content of the project stored in a file format ZIM, a high compressed open format with additional meta-data.
This repository contains advanced tools to (cross-)compile easily Kiwix softwares and library and deploy them. They have been tested on Fedora 23 and Ubuntu 16.10.
Prerequesites
You will need a recent version of meson
(0.34) and ninja
(1.6) If
your distribution provides a recent enough versions for them, just
install them with your package manager. Continue to read the
instructions otherwise.
Before anything else you need to install Python3 related tools. On Debian based systems:
$ sudo apt-get install python3-pip virtualenv
Create a virtual environment to install python module in it instead of modifying the system.
$ virtualenv -p python3 ./ # Create virtualenv
$ source bin/activate # Activate the virtualenv
Then, download and install kiwix-build and its dependencies:
$ git clone git://github.com/kiwix/kiwix-build.git
$ cd kiwix-build
$ pip install .
$ hash -r # Refresh bash paths
If your distribution doesn't provide ninja version > 1.6 you can get it this way :
$ wget https://github.com/ninja-build/ninja/releases/download/v1.8.2/ninja-linux.zip
$ unzip ninja-linux.zip ninja -d $HOME/bin
Compilation
The compilation is handled by the kiwix-build
command. It will compile
everything. If you are using a supported platform (Redhat or Debian
based) it will install missing packages using sudo
. You can get
kiwix-build
usage like this:
$ kiwix-build --help
Target
You may want to compile a specific target so you will have to specify it on the command line :
$ kiwix-build kiwix-lib # will build kiwix-build and its dependencies
$ kiwix-build zim-tools # will build zim-tools and its dependencies
By default, kiwix-build
will build kiwix-tools
.
Target platform
If no target platform is specified, a default one will be infered from the specified target :
kiwix-android
will be build using the platformandroid
- Other targets will be build using the platform
native_dyn
But you can select another target platform using the option
--target-platform
. For now, there is ten different supported
platforms :
- native_dyn
- native_static
- win32_dyn
- win32_static
- android
- android_arm
- android_arm64
- android_mips
- android_mips64
- android_x86
- android_x86_64
So, if you want to compile kiwix-tools
for win32 using static linkage:
$ kiwix-build --target-platform win32_dyn
Android
Android apk (kiwix-android) is a bit a special case.
kiwix-android
itself is architecture independent (it is written in
java) but it use kiwix-lib
who is architecture dependent.
When building kiwix-lib
, you should directly use the
target-platform android_<arch>
:
$ kiwix-build kiwix-android --target-platform android_arm
But, kiwix-android
apk can also be multi arch (ie, it includes
kiwix-lib
for several architectures). To do so, you must ask to build
kiwix-android
using the android
platform:
$ kiwix-build --target-platform android kiwix-android
$ kiwix-build kiwix-android # because `android` platform is the default `kiwix-android`
By default, when using platform android
, kiwix-lib
will be build for
all architectures. This can be change by using the option --android-arch
:
$ kiwix-build kiwix-android # apk for all architectures
$ kiwix-build kiwix-android --android-arch arm # apk for arm architectures (equivalent to `kiwix-android --target-platform android_arm`)
$ kiwix-build kiwix-anrdoid --android-arch arm --android-arch arm64 # apk for arm and arm64 architectures
IOS
When building for ios, we may want to compile a "fat library", a library for several architectures.
To do so, you should directly use the target-platfrom ios_multi
.
As for android
, kiwix-build
will build the library several times
(once for each platform) and then create the fat library.
$ kiwix-build --target-platform iOS_multi kiwix-lib
You can specify the supported architectures with the option --ios-arch
:
$ kiwix-build --target-platform iOS_multi kiwix-lib # all architetures
$ kiwix-build --target-platform iOS_multi --ios-arch arm --ios-arch arm64 # arm and arm64 arch only
Outputs
Kiwix-build.py will create several directories:
ARCHIVES
: All the downloaded archives go there.SOURCES
: All the sources (extracted from archives and patched) go there.BUILD_<target_platform>
: All the build files go there.BUILD_<target_platform>/INSTALL
: The installed files go there.BUILD_<target_platform>/LOGS
: The logs files of the build.
If you want to install all those directories elsewhere, you can pass the
--working-dir
option to kiwix-build
: