Commit Graph

367 Commits

Author SHA1 Message Date
Matthieu Gautier f96ae19c88 Disable websocket support in aria2.
We don't need it. It could speed up a bit the compilation.
2018-11-30 11:37:23 +01:00
Matthieu Gautier 703cac1f70 Do not compile libcurl in flatpak. 2018-11-30 11:37:23 +01:00
Matthieu Gautier ecf3b8b1ff Add an option `--asume-packages-installed` to not try to install packages.
This is different than `--dont-install-packages` that will try to
compile the dependencies.

`--asumme-packages-installed` will no try to install packages, but will
not try either to compile dependencies.
2018-11-27 17:22:47 +01:00
Matthieu Gautier a3ce2e0126 Fix the flatpak manifest.
- Rename the .desktop file and the icons for desktop integration.
- Fix finish-args for the right permissions
- Remove curl binary, not aria2c.
2018-11-27 17:22:47 +01:00
Matthieu Gautier a18c28b168 Build the flatpak as user, not system. 2018-11-27 17:22:47 +01:00
Matthieu Gautier feec272d1f Use the correct key `app-id` instead of `id`. 2018-11-27 17:22:47 +01:00
Matthieu Gautier 833c5844a1 Use the right option to build kiwix_desktop and pugixml on flatpak. 2018-11-27 17:22:47 +01:00
Matthieu Gautier ed63ffe24c Add aria2 dependency.
kiwix-desktop needs the aria2 binary to download content.
kiwix-desktop will use aria2 throw lib-kiwix so it is somehow a kiwix-lib
dependency, but it is used only by kiwix-desktop.

And we know to install it only on platforms where we are supporting
kiwix-destkop.
2018-11-27 17:22:47 +01:00
Matthieu Gautier d375cc06bc Do not try to compiled dependencies already in the org.kde.Platform. 2018-11-27 10:11:08 +01:00
birros 8d58d8d7cb First version of a flatpak builder.
This commit add a first version code to build a flatpak of kiwix-desktop.
It is mainly based on the PR #254 of @birros (hence he is the author of this commit)

However there is some differences :
- I (@mgautier) create a new builder to run the flatpak build instead of using a new
  dependency.
- Use the flatpak platform to install org.kde.Platform and org.kde.Sdk

This code version doesn't correctly work but I wanted to commit the birros' code
without too many modification (even if there is a lot).
2018-11-27 09:43:52 +01:00
Matthieu Gautier 0b9f13a2f0 Create a property in the remotefile to get the correct url. 2018-11-14 11:08:44 +01:00
Matthieu Gautier 054f7802d0 Update version of kiwix-lib and kiwix-tools. 2018-11-12 16:54:44 +01:00
Matthieu Gautier 500e2c1e02 update version of kiwix-lib and kiwix-desktop 2018-11-03 12:35:12 +01:00
Matthieu Gautier 85f0d4ccb1 Update version of kiwix-lib and kiwix-desktop. 2018-11-02 17:37:02 +01:00
Matthieu Gautier 1440dd8b25 Update version of kiwix-lib, kiwix-tools and kiwix-desktop. 2018-10-31 17:20:24 +01:00
Matthieu Gautier 0600be96ac Force the build of libcurl instead of using system one.
On ubuntu, the libcurl package is compiled with libkrb (kerberos).
However, no libkrb5.a is provided and so, we cannot link statically.

As we are not using kerberos, we can (and must) compile ourselves libcurl.
2018-10-31 15:33:12 +01:00
Matthieu Gautier 1e67a6b50f Add libcurl dependency.
This is needed for new kiwix-lib version.
We also remove libaria as kiwix-lib use aria2 binary directly.
2018-10-23 11:57:58 +02:00
Kelson e9ae0da048 Move to Xapian 1.4.7 2018-09-20 18:27:30 +02:00
Matthieu Gautier 1569068332 Do not set version for main project if we are not making release. 2018-09-06 16:10:43 +02:00
Matthieu Gautier ecc6927de8 Use git archive to create dist source archive.
qmake seems to generate wrong "dist" rules for "make".
It try to copy the translation files as if it was in current directory
instead of looking the source dir.

dist archive is usefull with autotools where we want to run autogen before.
Or with meson when meson run tests before making the archive.

With qmake, it seems useless, let's simply archive what is git.
2018-09-05 19:38:58 +02:00
Matthieu Gautier df0b684869 Correctly find the version of a dependency.
We have separated the version in two separated dicts since a long time :/
2018-09-05 19:31:38 +02:00
Matthieu Gautier e988165282 Update version of libzim and kiwix-desktop. 2018-09-05 16:56:46 +02:00
Matthieu Gautier 07e8b36efb Do not overwrite release build archives.
We new release of a project is made, projects dependending of it are
rebuild and republish to keep the binaries up-to-date.

To avoid overwrite the already build archives and silent the fact that
new archives are available, we need to have a extra build version.

`release_versions[project]` has to be set each time a dependency of
`project` is bump but the project version has not changed.

If the `project` version changes, the `release_versions[project]`
has to be cleared.

If we don't want to release this version again, remove project from
`release_versions`.

See kiwix/kiwix-tools#222
2018-09-04 17:56:01 +02:00
Matthieu Gautier aed1194570 Update version of kiwix-tools to 0.6.1. 2018-08-30 16:04:28 +02:00
Matthieu Gautier 095eec303a Use the more recent msvcr100 instead of msvcr90. 2018-08-28 16:42:06 +02:00
Matthieu Gautier 91f282d8f2 Update versions of libzim and kiwix-desktop. 2018-08-24 15:35:01 +02:00
Matthieu Gautier 3b483f9011 Do not build for mips and mips64 by default. 2018-08-22 16:44:53 +02:00
Matthieu Gautier 6f291b1c86 Set the default platform to `android` for `kiwix-android-custom` project.
Fix #206
2018-08-20 11:18:01 +02:00
Matthieu Gautier 81612c2f2c Set the default option in the method parsing arguments, not in the builder. 2018-08-20 11:16:33 +02:00
Matthieu Gautier a64fd8cd20 New versions. 2018-08-03 19:34:50 +02:00
Matthieu Gautier e22ceab92a Do not use buffered header on android.
Android devices are low memory devices, use last libzim compilation option
to avoid copy/mmap index header in memory.
2018-08-01 18:07:13 +02:00
Matthieu Gautier de6ae772d4 Update to last vresion of xapian. 2018-07-27 10:46:00 +02:00
Matthieu Gautier 5c83ba9d10 Fix compilation of kiwix-custom-app.
This is forgotten dependency. Not updated when we moved to multi-arch
build.
Now, it is fixed.
2018-07-19 16:19:55 +02:00
Emmanuel Engelhart 79a08e001b Use faster svn export instead of svn checkout 2018-07-16 10:15:28 +02:00
Matthieu Gautier 1a4267a887 Print a '.' every 30 seconds when running a command.
It will avoid travis-ci to kill the job because of the absence of output.
2018-07-13 10:39:18 +02:00
Matthieu Gautier 0f389f7cc9 Get icu sources from an archive instead of svn.
SVN checkout can be pretty long to do (especially if icu serveur is pretty
slow).

By downloading an archive on our own server, we avoid a spending too many
time downloading.

The archive is a simple targz of the SVN checkout.
2018-07-11 11:00:55 +02:00
mhutti1 20ac697d43 Build with android logging library 2018-06-29 15:26:54 +02:00
Emmanuel Engelhart ef2e95b42e Bump-up deps meta version to 9 2018-06-24 20:51:31 +02:00
Emmanuel Engelhart e90ff48ec9 Remove support of old Xapian chert backend 2018-06-24 20:51:31 +02:00
Matthieu Gautier d5b81294ca [TRAVIS] Fix release dist.
- Correctly make dist.
- zimwriterfs is now using meson, we don't need a special case.
2018-06-19 10:39:07 +02:00
Matthieu Gautier d076c6b8d7 Do not set zimwriterfs version directly in the dependency.
The version is tracked in `kiwixbuild/versions.py`, so we must no set
it in `kiwixbuild/dependencies/zimwriterfs.py`.
2018-06-18 17:56:13 +02:00
Matthieu Gautier fbb84f8df2 Fix a small typo breaking the release build process. 2018-06-18 14:17:06 +02:00
Matthieu Gautier d53f1aad4a Update to new versions of libzim, kiwix-lib, kiwix-tools, zim-tools, zimwriterfs. 2018-06-15 19:39:45 +02:00
Matthieu Gautier 37ebcf83ae Use Docker to build kiwix-desktop only.
Keep default travis env for everything else.
2018-06-15 15:57:57 +02:00
Matthieu Gautier b3d9ff5a6b Add toolchains steps even if we are building with `--build-nodeps`.
Toolchains are not really a deps, they have to be build everytime.
2018-06-14 22:58:43 +02:00
Matthieu Gautier b090353001 Do not use the system ctpp2 in native_dyn debian.
The CI is using ubuntu artful and the deb package is compiled with another
compiler that gcc-4.8 (used in other project CI).

As we compile everything with gcc-4.8, we must compile our own ctpp2.
2018-06-14 18:18:46 +02:00
Matthieu Gautier 1cbfcef9ce Fix kiwix-desktop dependencies. 2018-06-14 18:18:45 +02:00
Matthieu Gautier eed80c1ff9 Correctly associate the source with the builder. 2018-06-14 18:18:45 +02:00
Matthieu Gautier 8506cdeffd Correctly pass CMAKE_* to qmake.
qmake doesn't use the env variables, we must pass the variables in the
command line.
2018-06-14 18:18:45 +02:00
Matthieu Gautier 4047ec20dd Add xapian patch to correctly include sys types. 2018-06-14 18:18:45 +02:00
Matthieu Gautier 2a18f74786 Do not build with gcc-5.
We build with gcc-5 because of zimSplit using ofstream.swap method.
Latest version of zimSplit doesn't use it anymore so let's compile
everything with default gcc.

See openzim/zim-tools@a959609839

We also explicitly set the list of package to install per job, so
less packages are installed.
2018-06-13 10:57:32 +02:00
Matthieu Gautier 398f77f9f3 Add kiwix-desktop dependency. 2018-06-12 16:12:31 +02:00
Matthieu Gautier cf8f303362 Add qt(webengine) dependency.
Add qt dependency and QMakeBuilder.
2018-06-12 16:12:15 +02:00
Matthieu Gautier 63003d5bce Correctly extract platform from the dependency.
If the `dep` is a two char string (as "qt"), the `platform, name = dep`
will split the string and search for 't' dependency in 'q' platform.

So, we have to be sure that the dep is a tuple before splitting it.
2018-06-12 16:12:15 +02:00
Matthieu Gautier 1109b6b155 Better compilation of icu4c.
- Disable icuio and layoutex.
- Hardcode UTF8 as charset (we always use utf8 and hardcoding him improve
  performances)
- Do not include default utf headers
- Do not use `using namespace icu`

See http://source.icu-project.org/repos/icu/trunk/icu4c/readme.html#RecBuild
for more information about options.

Increment base dependencies version as we are compiling icu differently.
2018-06-11 18:05:10 +02:00
Matthieu Gautier 4dea44d305 Fix travis CI 2018-06-08 10:29:26 +02:00
Matthieu Gautier daefadecc9 Better gradle option.
Enable `--no-daemon` : Using the daemon make gradle keep "configuration"
(like plugin path). If gradle installation path change, it will break
kiwix-android build.

Enable `--build-cache` : This is not the case by default :/
This should greatly improve compilation speed.
2018-06-07 10:50:30 +02:00
Matthieu Gautier 369b805e59 Add ios_multi_arch fatlib.
Fix #113
2018-06-06 17:41:12 +02:00
Matthieu Gautier cfdf03c854 Select the default platform depending of the target.
Default platform `android` is better for `kiwix-android`
(because `kiwix-android` can build only on `android` platform).
2018-06-06 17:41:12 +02:00
Matthieu Gautier 5f5fbfe082 Rename `targets` option to `target`. 2018-06-06 17:41:12 +02:00
Matthieu Gautier db82455f03 Be able to build multi-arch android apk.
By building kiwix-android on the `android` platform, we can now build
`kiwix-lib` on all `android_<arch>` platforms, and so have all
architectures in the same apk.

Fix #108
2018-06-06 17:41:12 +02:00
Matthieu Gautier 9b85791705 Gradle is a dependency of the android platform, not of other dependencies. 2018-06-06 17:41:11 +02:00
Matthieu Gautier 6de2cc80b6 Add an android metaPlatformInfo to build all android architecture. 2018-05-31 11:30:26 +02:00
Matthieu Gautier 74ba5885a0 Add a MetaPlatformInfo.
A metaplatform allow to regroup sereval platform together.
When a target is added to the platform, the target is dispatched/add to
all sub platforms.

As the step (metaplatformName, target) is not really added, we have to
track which steps are really added, so `add_targets` need to return the
list of added targets.
2018-05-31 11:30:26 +02:00
Matthieu Gautier 3eae47240b Make the platform responsible to add the targets. 2018-05-31 11:30:26 +02:00
Matthieu Gautier c8c0192675 Simplify cross-compilation
Now we can make reference to build step in another target platform, we can
simplify cross-compilation fo libmagic and icu4c.
2018-05-31 11:30:26 +02:00
Matthieu Gautier a87ba06bf3 Do not infinit loop when ordering steps 2018-05-31 11:30:26 +02:00
Matthieu Gautier 7e8ed1a623 Correctly install packages. 2018-05-31 11:30:26 +02:00
Matthieu Gautier 4d6b6994bc Merge toolchain and dependencies.
Toolchain now became "simple" dependencies.
They are referenced by Platform.

Platform are now created at demande and a neutralPlatform now exists.
2018-05-31 11:30:25 +02:00
Matthieu Gautier 9496ffb86c Clean android_ndk 2018-05-31 11:25:59 +02:00
Matthieu Gautier 211c0f9651 buildEnv doesn't need to know about toolchains.
Having access to the platform is enough.
2018-05-31 11:25:59 +02:00
Matthieu Gautier 41d23b6249 Create the associated platform at demand. 2018-05-31 11:25:59 +02:00
Matthieu Gautier daad1c98c6 Make the options global. 2018-05-31 11:25:59 +02:00
Matthieu Gautier 041826d0e8 Move from target logic to steps logic.
This is the big change !!!!

Instead of handling target as primary object and prepare/build targets,
we are handling build steps.

A build step may be a source (preparation) or a build (of the source).
Actualy, a step is a tuple (context, Builder or Source).

The context define the context of the step. It can be :
- 'source', for a Source step
- 'neutral' or the name of a platform for Build step.

Target becomes a "Class only" class.
2018-05-31 11:25:59 +02:00
Matthieu Gautier 754df8df1d Make the platform responsible to create the buildenv.
It is now the platform who is responsible to:
- correctly add the toolchains as needed
- setup the cross environment
- create the cross files.
2018-05-31 11:25:54 +02:00
Matthieu Gautier 115fbfa147 Move dependencies declaration of a target into the builder.
This is the builder that depends on other target, not the target itself.
2018-05-31 11:14:31 +02:00
Matthieu Gautier 7e0b403ccc Move `run_command` method out of buildEnv class. 2018-05-31 11:14:31 +02:00
Matthieu Gautier 816e06a512 Make the neutralEnv global.
No need to pass the neutralEnv from instance to instance.
2018-05-31 10:42:37 +02:00
Matthieu Gautier a413c5f064 Fix root_path lookup in cmake_cross_file generation.
Sometime the root_path is dependent of the target platform and sometime
not. But sometime dependent of the build arch :/

[TODO] We should move the cross_file generation to the PlatformInfo class.
2018-05-22 17:47:21 +02:00
Matthieu Gautier 98ebba65f6 fix imports 2018-05-22 16:58:40 +02:00
Matthieu Gautier b950feb893 Move Builder and BuildEnv in specific module.
Introduce also a "NeutralEnv", a build environment independent of the
targeted platform. All `Source` now build using the neutralEnv.

Most of toolchains are also using neutralEnv except android_ndk who is
specific to a platform.

As toolchain are neutral, platform specific environment variables are now
set by the platformInfo directly instead of the toolchain.
2018-05-22 16:58:39 +02:00
Matthieu Gautier ac83dec674 Move ios sdk in toolchains module. 2018-05-22 16:58:06 +02:00
Matthieu Gautier 363bdf74ed Move target informations into a platforms module.
`TargetInfo`s are information about the target platform.
Let's rename them into PlatformInfo and move them in a specific module.
2018-05-22 16:58:02 +02:00
Matthieu Gautier f1d1fc654c Better MakeBuilder configure system.
`all_configure_option` should be ALL configure option.
2018-05-22 16:43:37 +02:00
Matthieu Gautier 53585b838a Split the dependencies.py file into several parts. 2018-05-22 16:43:37 +02:00
Matthieu Gautier fa829e1dec Rename dependency_versions.py to versions.py 2018-05-22 16:39:55 +02:00
Chris Li ad21da6a01 change armv7s to armv7 2018-05-21 16:05:17 +02:00
Matthieu Gautier c745e325c6 Make kiwix-build install libtool.
Fix #169.
2018-05-21 10:52:52 +02:00
Pseudo 66cad23bcc Remove Darwin warning, Darwin is now supported. 2018-05-17 07:57:20 +02:00
Emmanuel Engelhart d001372a8b Use alternative (HTTPS) file source tarball URL 2018-05-14 15:46:57 +02:00
Matthieu Gautier ee5a8eda96 Build zimwriterfs using meson. 2018-05-14 15:46:57 +02:00
Matthieu Gautier a417b044f8 Add libmagic dependency. 2018-05-14 15:46:57 +02:00
Matthieu Gautier cc5cfd1901 Update MACOS_DEPLOYMENT_TARGET to 10.7
This is needed to cross-compile to iOS using C++11.
2018-04-24 14:58:14 +02:00
Matthieu Gautier a9818d18c4 Move ios cross file templates in the right directory. 2018-04-24 14:57:23 +02:00
Matthieu Gautier f0c45c7b42 Update to new versions of libzim, kiwix-lib, kiwix-tools. 2018-04-23 20:52:59 +02:00
Matthieu Gautier 9f877180df Do not compile libaria2 with sqlite3 and libxml2.
Those dependencies are used for bittorent and metalink.
For now, we don't support those features (we should in the future).

However, it seems that if development package are installed on the system,
libaria will try to link with libsqlite3 and libxml3 anyway.

We have to investigate this, but it breaks static build of kiwix-tools.
As we are not using those feature for now, let's just explicitly
compile libaria2 without sqlite3 and libxml2.
2018-04-23 12:08:06 +02:00
Matthieu Gautier 65969f4999 Add missing library in the list of libraries to link with in win32.
Libaria2 use methods in `iphlpapi` on Windows.
2018-04-23 10:45:08 +02:00
Matthieu Gautier 2270a070d2 Add an option to allways check for needed system packages. 2018-04-19 10:34:24 +02:00
Matthieu Gautier 1773f0fa28 Install libmagic in macos for zimwriterfs. 2018-04-18 17:48:32 +02:00
Matthieu Gautier 3539a8affe Specify the LIB_SUFFIX for CTPP2. 2018-04-17 17:41:23 +02:00
Matthieu Gautier f536d625a9 Add a patch to icu to correctly set the rpath in zimwriterfs build. 2018-04-17 17:41:23 +02:00
Matthieu Gautier 9acd6b3447 Run the xcodebuild command only we actually use iOSTargetInfo. 2018-04-17 17:40:11 +02:00
Matthieu Gautier 72d9e99bf9 except the correct exception URLError when downloading.
`urllib.request.open` can raise a URLError or a HTTPError.
2018-04-17 17:39:13 +02:00
Matthieu Gautier 24c771aaa8 Avoid to make a svn_update at each time. 2018-04-17 17:37:07 +02:00
Matthieu Gautier f94ca108f7 Enable RPATH for Icu. 2018-04-16 19:42:07 +02:00
Matthieu Gautier 6d507651a4 Do not compile icu tools when cross-compiling icu4c.
We do not need tools when cross-compiling icu4c.
And when cross-compiling to iOS, tools are not compiling at all.
2018-04-16 17:45:03 +02:00
Chris Li 5eedb3d96a First version of kiwix cross-compilation to IOS. 2018-04-16 17:45:03 +02:00
Matthieu Gautier 72f3691ba6 Compile i586 without sse instructions.
We need a to compile binaries without sse instructions
(https://github.com/kiwix/kiwix-build/issues/94#issuecomment-376814014)
2018-04-09 12:09:18 +02:00
Matthieu Gautier 83a4841ea2 Do not build ctpp2 with iconv support. 2018-04-06 18:59:15 +02:00
Matthieu Gautier 27bce3cd99 Add missing package for i586 ubuntu in travis. 2018-04-06 18:59:15 +02:00
Matthieu Gautier 468c3edff9 Add a i586 toolchain.
This will allow to build kiwix-tools for i586 architectures.
2018-04-06 18:59:15 +02:00
Matthieu Gautier 5d08673a52 Correctly stop the build if there is an error during the downloading. 2018-04-05 23:32:46 +02:00
Matthieu Gautier 8f0c6a321b Add an option to not build all dependencies but only the specified target. 2018-04-05 23:32:46 +02:00
mhutti1 c01ca69130 Increment gradle wrapper version 2018-04-05 17:22:08 +02:00
Matthieu Gautier 372d11e23d Add libaria2 to the list of all base dependencies. 2018-04-05 15:40:04 +02:00
Matthieu Gautier eb75660e48 Do not try to compile xz and xzdec tools.
libaria2 use xz to configure its source.
If we compile the xz tool, libaria2 will use the compiled binary.

However, on cross compilation, xz will be compiled for the target platform
and run on the host platform. And so, it will fails.

As we do not use those tools at all, lets not compile them.
2018-04-05 15:39:26 +02:00
Matthieu Gautier 97666b0fc9 Create a real module kiwixbuild and make it installable. Split files.
Make kiwix-build installable using pip.
Module is now called `kiwixbuild` because `kiwix-build` is not a valid
python identifier.

Also split toolchains in separated directory.
2018-04-04 15:43:16 +02:00