Commit Graph

367 Commits

Author SHA1 Message Date
Matthieu Gautier 2a18f74786 Do not build with gcc-5.
We build with gcc-5 because of zimSplit using ofstream.swap method.
Latest version of zimSplit doesn't use it anymore so let's compile
everything with default gcc.

See openzim/zim-tools@a959609839

We also explicitly set the list of package to install per job, so
less packages are installed.
2018-06-13 10:57:32 +02:00
Matthieu Gautier 398f77f9f3 Add kiwix-desktop dependency. 2018-06-12 16:12:31 +02:00
Matthieu Gautier cf8f303362 Add qt(webengine) dependency.
Add qt dependency and QMakeBuilder.
2018-06-12 16:12:15 +02:00
Matthieu Gautier 63003d5bce Correctly extract platform from the dependency.
If the `dep` is a two char string (as "qt"), the `platform, name = dep`
will split the string and search for 't' dependency in 'q' platform.

So, we have to be sure that the dep is a tuple before splitting it.
2018-06-12 16:12:15 +02:00
Matthieu Gautier 1109b6b155 Better compilation of icu4c.
- Disable icuio and layoutex.
- Hardcode UTF8 as charset (we always use utf8 and hardcoding him improve
  performances)
- Do not include default utf headers
- Do not use `using namespace icu`

See http://source.icu-project.org/repos/icu/trunk/icu4c/readme.html#RecBuild
for more information about options.

Increment base dependencies version as we are compiling icu differently.
2018-06-11 18:05:10 +02:00
Matthieu Gautier 4dea44d305 Fix travis CI 2018-06-08 10:29:26 +02:00
Matthieu Gautier daefadecc9 Better gradle option.
Enable `--no-daemon` : Using the daemon make gradle keep "configuration"
(like plugin path). If gradle installation path change, it will break
kiwix-android build.

Enable `--build-cache` : This is not the case by default :/
This should greatly improve compilation speed.
2018-06-07 10:50:30 +02:00
Matthieu Gautier 369b805e59 Add ios_multi_arch fatlib.
Fix #113
2018-06-06 17:41:12 +02:00
Matthieu Gautier cfdf03c854 Select the default platform depending of the target.
Default platform `android` is better for `kiwix-android`
(because `kiwix-android` can build only on `android` platform).
2018-06-06 17:41:12 +02:00
Matthieu Gautier 5f5fbfe082 Rename `targets` option to `target`. 2018-06-06 17:41:12 +02:00
Matthieu Gautier db82455f03 Be able to build multi-arch android apk.
By building kiwix-android on the `android` platform, we can now build
`kiwix-lib` on all `android_<arch>` platforms, and so have all
architectures in the same apk.

Fix #108
2018-06-06 17:41:12 +02:00
Matthieu Gautier 9b85791705 Gradle is a dependency of the android platform, not of other dependencies. 2018-06-06 17:41:11 +02:00
Matthieu Gautier 6de2cc80b6 Add an android metaPlatformInfo to build all android architecture. 2018-05-31 11:30:26 +02:00
Matthieu Gautier 74ba5885a0 Add a MetaPlatformInfo.
A metaplatform allow to regroup sereval platform together.
When a target is added to the platform, the target is dispatched/add to
all sub platforms.

As the step (metaplatformName, target) is not really added, we have to
track which steps are really added, so `add_targets` need to return the
list of added targets.
2018-05-31 11:30:26 +02:00
Matthieu Gautier 3eae47240b Make the platform responsible to add the targets. 2018-05-31 11:30:26 +02:00
Matthieu Gautier c8c0192675 Simplify cross-compilation
Now we can make reference to build step in another target platform, we can
simplify cross-compilation fo libmagic and icu4c.
2018-05-31 11:30:26 +02:00
Matthieu Gautier a87ba06bf3 Do not infinit loop when ordering steps 2018-05-31 11:30:26 +02:00
Matthieu Gautier 7e8ed1a623 Correctly install packages. 2018-05-31 11:30:26 +02:00
Matthieu Gautier 4d6b6994bc Merge toolchain and dependencies.
Toolchain now became "simple" dependencies.
They are referenced by Platform.

Platform are now created at demande and a neutralPlatform now exists.
2018-05-31 11:30:25 +02:00
Matthieu Gautier 9496ffb86c Clean android_ndk 2018-05-31 11:25:59 +02:00
Matthieu Gautier 211c0f9651 buildEnv doesn't need to know about toolchains.
Having access to the platform is enough.
2018-05-31 11:25:59 +02:00
Matthieu Gautier 41d23b6249 Create the associated platform at demand. 2018-05-31 11:25:59 +02:00
Matthieu Gautier daad1c98c6 Make the options global. 2018-05-31 11:25:59 +02:00
Matthieu Gautier 041826d0e8 Move from target logic to steps logic.
This is the big change !!!!

Instead of handling target as primary object and prepare/build targets,
we are handling build steps.

A build step may be a source (preparation) or a build (of the source).
Actualy, a step is a tuple (context, Builder or Source).

The context define the context of the step. It can be :
- 'source', for a Source step
- 'neutral' or the name of a platform for Build step.

Target becomes a "Class only" class.
2018-05-31 11:25:59 +02:00
Matthieu Gautier 754df8df1d Make the platform responsible to create the buildenv.
It is now the platform who is responsible to:
- correctly add the toolchains as needed
- setup the cross environment
- create the cross files.
2018-05-31 11:25:54 +02:00
Matthieu Gautier 115fbfa147 Move dependencies declaration of a target into the builder.
This is the builder that depends on other target, not the target itself.
2018-05-31 11:14:31 +02:00
Matthieu Gautier 7e0b403ccc Move `run_command` method out of buildEnv class. 2018-05-31 11:14:31 +02:00
Matthieu Gautier 816e06a512 Make the neutralEnv global.
No need to pass the neutralEnv from instance to instance.
2018-05-31 10:42:37 +02:00
Matthieu Gautier a413c5f064 Fix root_path lookup in cmake_cross_file generation.
Sometime the root_path is dependent of the target platform and sometime
not. But sometime dependent of the build arch :/

[TODO] We should move the cross_file generation to the PlatformInfo class.
2018-05-22 17:47:21 +02:00
Matthieu Gautier 98ebba65f6 fix imports 2018-05-22 16:58:40 +02:00
Matthieu Gautier b950feb893 Move Builder and BuildEnv in specific module.
Introduce also a "NeutralEnv", a build environment independent of the
targeted platform. All `Source` now build using the neutralEnv.

Most of toolchains are also using neutralEnv except android_ndk who is
specific to a platform.

As toolchain are neutral, platform specific environment variables are now
set by the platformInfo directly instead of the toolchain.
2018-05-22 16:58:39 +02:00
Matthieu Gautier ac83dec674 Move ios sdk in toolchains module. 2018-05-22 16:58:06 +02:00
Matthieu Gautier 363bdf74ed Move target informations into a platforms module.
`TargetInfo`s are information about the target platform.
Let's rename them into PlatformInfo and move them in a specific module.
2018-05-22 16:58:02 +02:00
Matthieu Gautier f1d1fc654c Better MakeBuilder configure system.
`all_configure_option` should be ALL configure option.
2018-05-22 16:43:37 +02:00
Matthieu Gautier 53585b838a Split the dependencies.py file into several parts. 2018-05-22 16:43:37 +02:00
Matthieu Gautier fa829e1dec Rename dependency_versions.py to versions.py 2018-05-22 16:39:55 +02:00
Chris Li ad21da6a01 change armv7s to armv7 2018-05-21 16:05:17 +02:00
Matthieu Gautier c745e325c6 Make kiwix-build install libtool.
Fix #169.
2018-05-21 10:52:52 +02:00
Pseudo 66cad23bcc Remove Darwin warning, Darwin is now supported. 2018-05-17 07:57:20 +02:00
Emmanuel Engelhart d001372a8b Use alternative (HTTPS) file source tarball URL 2018-05-14 15:46:57 +02:00
Matthieu Gautier ee5a8eda96 Build zimwriterfs using meson. 2018-05-14 15:46:57 +02:00
Matthieu Gautier a417b044f8 Add libmagic dependency. 2018-05-14 15:46:57 +02:00
Matthieu Gautier cc5cfd1901 Update MACOS_DEPLOYMENT_TARGET to 10.7
This is needed to cross-compile to iOS using C++11.
2018-04-24 14:58:14 +02:00
Matthieu Gautier a9818d18c4 Move ios cross file templates in the right directory. 2018-04-24 14:57:23 +02:00
Matthieu Gautier f0c45c7b42 Update to new versions of libzim, kiwix-lib, kiwix-tools. 2018-04-23 20:52:59 +02:00
Matthieu Gautier 9f877180df Do not compile libaria2 with sqlite3 and libxml2.
Those dependencies are used for bittorent and metalink.
For now, we don't support those features (we should in the future).

However, it seems that if development package are installed on the system,
libaria will try to link with libsqlite3 and libxml3 anyway.

We have to investigate this, but it breaks static build of kiwix-tools.
As we are not using those feature for now, let's just explicitly
compile libaria2 without sqlite3 and libxml2.
2018-04-23 12:08:06 +02:00
Matthieu Gautier 65969f4999 Add missing library in the list of libraries to link with in win32.
Libaria2 use methods in `iphlpapi` on Windows.
2018-04-23 10:45:08 +02:00
Matthieu Gautier 2270a070d2 Add an option to allways check for needed system packages. 2018-04-19 10:34:24 +02:00
Matthieu Gautier 1773f0fa28 Install libmagic in macos for zimwriterfs. 2018-04-18 17:48:32 +02:00
Matthieu Gautier 3539a8affe Specify the LIB_SUFFIX for CTPP2. 2018-04-17 17:41:23 +02:00
Matthieu Gautier f536d625a9 Add a patch to icu to correctly set the rpath in zimwriterfs build. 2018-04-17 17:41:23 +02:00
Matthieu Gautier 9acd6b3447 Run the xcodebuild command only we actually use iOSTargetInfo. 2018-04-17 17:40:11 +02:00
Matthieu Gautier 72d9e99bf9 except the correct exception URLError when downloading.
`urllib.request.open` can raise a URLError or a HTTPError.
2018-04-17 17:39:13 +02:00
Matthieu Gautier 24c771aaa8 Avoid to make a svn_update at each time. 2018-04-17 17:37:07 +02:00
Matthieu Gautier f94ca108f7 Enable RPATH for Icu. 2018-04-16 19:42:07 +02:00
Matthieu Gautier 6d507651a4 Do not compile icu tools when cross-compiling icu4c.
We do not need tools when cross-compiling icu4c.
And when cross-compiling to iOS, tools are not compiling at all.
2018-04-16 17:45:03 +02:00
Chris Li 5eedb3d96a First version of kiwix cross-compilation to IOS. 2018-04-16 17:45:03 +02:00
Matthieu Gautier 72f3691ba6 Compile i586 without sse instructions.
We need a to compile binaries without sse instructions
(https://github.com/kiwix/kiwix-build/issues/94#issuecomment-376814014)
2018-04-09 12:09:18 +02:00
Matthieu Gautier 83a4841ea2 Do not build ctpp2 with iconv support. 2018-04-06 18:59:15 +02:00
Matthieu Gautier 27bce3cd99 Add missing package for i586 ubuntu in travis. 2018-04-06 18:59:15 +02:00
Matthieu Gautier 468c3edff9 Add a i586 toolchain.
This will allow to build kiwix-tools for i586 architectures.
2018-04-06 18:59:15 +02:00
Matthieu Gautier 5d08673a52 Correctly stop the build if there is an error during the downloading. 2018-04-05 23:32:46 +02:00
Matthieu Gautier 8f0c6a321b Add an option to not build all dependencies but only the specified target. 2018-04-05 23:32:46 +02:00
mhutti1 c01ca69130 Increment gradle wrapper version 2018-04-05 17:22:08 +02:00
Matthieu Gautier 372d11e23d Add libaria2 to the list of all base dependencies. 2018-04-05 15:40:04 +02:00
Matthieu Gautier eb75660e48 Do not try to compile xz and xzdec tools.
libaria2 use xz to configure its source.
If we compile the xz tool, libaria2 will use the compiled binary.

However, on cross compilation, xz will be compiled for the target platform
and run on the host platform. And so, it will fails.

As we do not use those tools at all, lets not compile them.
2018-04-05 15:39:26 +02:00
Matthieu Gautier 97666b0fc9 Create a real module kiwixbuild and make it installable. Split files.
Make kiwix-build installable using pip.
Module is now called `kiwixbuild` because `kiwix-build` is not a valid
python identifier.

Also split toolchains in separated directory.
2018-04-04 15:43:16 +02:00