The target_platform option has always be wrongly named.
This is not the platform we are targeted but how we compile.
This was ok at beginning as specifying how we compile somehow define
for what we compile but this is not a one to one mapping.
When we introduced CoreKiwix.xcframework, we made two mistakes:
- We forgot that some devs are on Apple Silicon and did not include arm64 arch for simulator
- We kept the `x86_64-apple-ios` target thinking simulator is a “just” an iOS on x64
This target mistake prevented XCode from finding our lib usable with the Simulator on x64.
Not including an arm64 version obviously prevented Silicon-using devs from using the Simulator.
The included libs changes now from
| Name | Content |
| ---------------------------- | ----------------------------------------------------- |
| `ios-arm64` | All iOS devices are arm64 |
| `ios-x86_64-simulator` | Single x64 binary for Simulator |
| `macos-arm64_x86_64` | Fat binary for macOS with both x64 and arm64 |
To
| Name | Content |
| ---------------------------- | ----------------------------------------------------- |
| `ios-arm64` | All iOS devices are arm64 |
| `ios-arm64_x86_64-simulator` | Fsat binary for iOS Simulator with both x64 and arm64 |
| `macos-arm64_x86_64` | Fat binary for macOS with both x64 and arm64 |
- `iOSx64` Platforn renamed to `iOSx64Simulator` with its target fixed to `x86-apple-ios-simulator` (was `x86_64-apple-ios`)
- Added platformn `iOSArm64Simulator` for Apple Silicon devs to run Simulator
- `AppleXCFramework` dependency gets two changes:
- Depends on all those platforms
- `_make_macos_fat()` turned generic and called both for macOS fat binary and iOS Simulator one
We compile only libzim on bionic, for compatibility with old libc.
No need to compile all other dependencies (and fight with some of them not
compiling)
It mainly allow to run command in directory containing space.
(Hello, `C:\Program Files\...`)
It would also allow to work on directory containning spaces
(`C:\Users\John Doe`) but xapian configure (at least) expressly doesn't
support it :/
- Run the command without shell=True
- The command must be a list instead of a string.
- All options must also be a list (or an iterable).
When running command without `shell=True`, python's subproccess will look
for variable `"PATH"` and `b"PATH"` and will complain if both are set.
So the defaultdict should not return something for `b"PATH"`.
We also escape space ` ` in the environment variables to not break
everything with directory containing space.
no_skip is thus computed automatically depending on project being our or not.
We dont want to skip configure on our own projetcs (limited impact, simplifies deps mgmt)
CoreKiwix.xcframework is the packaging format required by Kiwix apple (macOS/iOS) reader for libkiwix.
Naming will hopefully be revisited later (libkiwix.xcframework?)
This folder is a combination of all static archive (*.a): dependencies and libkiwix itself
for all apple-supported platforms: macOS x86_64, macOS arm64, iOS arm64 and iOS x86_64.
It also includes the headers.
Note: while iOS archs are separated in the framework, the macOS archs are bundled as a fat binary.
This thus adds:
- A new `apple_all_static` target that lists mentioned sub-platforms.
- A new `AppleXCFramework` virtual dependency and builder
- requires `libkiwix`
- merges the archives into one per target
- creates the fat binary for macOS
- creates the xcframework
- `libkiwix_xcframework-VERSION.tar.gz` because platformname is xcframework
- it only contains the xcframework but in libkiwix_xcframework-VERSION/lib folder
Because with this virtual target, there are no other files
- requires a new `xcframework` platform_name in builddef
- subplatforms deps and macOS/iOS _ok files to make_deps_archive
---
https://developer.apple.com/documentation/xcode/creating-a-multi-platform-binary-framework-bundle
When we use `--disable-tools` (ie, when we are cross-compiling), ICU
will build the data only if it detects we are cross-compiling.
This make sens until we cross-compiling for wasm.
Wasm output is a script file with the interpreter in the sdk we have
installed. So ICU's configure can run the test binary and so doesn't detect
we are cross-compiling.
So we have to patch ICU Makefile to make it build the ICU data anyway.
ICU is used to:
- List locales (`locales_tree`)
- Get information about those locales in different languages (`lang_tree`)
- Remove accents (`translit`)
Tracing ICU without custom data, we also load `likelySubtags` and
`metadata`. Not sure why and what happen when missing but let's be
conservatives and include them.
We don't have a version 13.0.0 for libkiwix.
So the master branch of libkiwix is generating version 12.1.1.
We must use the correct version as we try to fix the rpath on macos
and we we don't have the right version, we fail because the lib file
doesn't exist.
This is mainly a release of libkiwix 12.1.0 as 12.1.0 introduce breaking
changes.
No need to rebuild kiwix-tools (it is already builded with 12.1.0)
Don't build kiwix-destkop as current version of kiwix-desktop is not
ported to kiwix-13.0.0 (`main` is, but not last release).
Until now, mixed targets was only about native build and so we were not
using a meson cross_config file and env var was enough.
But now we also to correctly set it in the cross_config file.
cross-gcc-10.3.0-pi_64.tar.gz for 64 bits architecture and armhf
is about 32 bits.
However, we know use a pi 2 and 3 and Stretch only[*] toolchains
[*] To be tested. Maybe the only is for the target compilation but binary
run elsewhere too.
The version of libzim (and other project too) is used to know what we
need to package in the published archive (nightly and releasee).
So the version must correspond to what is build.
For nightlies, we build the `main` branch and the main branch of libzim
is still on 8.1.0 so we must have the same version.
Fixopenzim/libzim#772
As this is just a ABI fix and we recompile everything when we do
a release of our projects, we don't need to recompile our projects.
Building with libzim 8.1.0 was enough, no need to trigger a update in all
the users of prebuild binary.
The `INSTALL_DIR` was added twice. It was not a issue as we then transform
the list into a set to remove duplicated.
But with `filter_install_dir` call only on one "add", the (textual)
entries are not duplicated and so, not removed. So the files where add
twice.
Now we correctly filter initial `INSTALL_DIR` and we remove the second add.
The default detected libdir is based on the build architecture.
On ubuntu, it is `lib/x86_64-linux-gnu` which is obviously not the right
directory.
Let's simply use `lib`.
Fix#556
Version 5.2.7 include this commit
https://git.tukaani.org/?p=xz.git;a=commit;h=31d80c6b261b24220776dfaeb8a04f80f80e0a24
With this change, compiling libzim mixed (libzim dynamic and dependencies,
so lzma, statically) fails at libzim linking with a
`src/libzim.so.8.0.1: version node not found for symbol lzma_get_progress@XZ_5.2.2`
error message.
This can be "workaround" by passing `--disable-symbol-versions` to
configure script but then, it is the compilation of kiwix-desktop in
native_dyn which falling with
```
/usr/bin/ld: /usr/lib64/libsystemd.so.0: undefined reference to `lzma_code@XZ_5.0'
/usr/bin/ld: /usr/lib64/libsystemd.so.0: undefined reference to `lzma_end@XZ_5.0'
/usr/bin/ld: /usr/lib64/libsystemd.so.0: undefined reference to `lzma_stream_decoder@XZ_5.0'
/usr/bin/ld: /usr/lib64/libxml2.so.2: undefined reference to `lzma_auto_decoder@XZ_5.0'
/usr/bin/ld: /usr/lib64/libxml2.so.2: undefined reference to `lzma_properties_decode@XZ_5.0'
```
Probably because some native dependencies (Qt ?) use versionned symbols.
This have to be fixed somehow but until then, let's go back to 5.2.6
`icu4c_wasm.patch` is build by :
- Copying config.sub from liblzma source as new version of config.sub there
knows about wasm architecture.
- Copying `mh-linux` on `mh-unknown` as specified in (origin) `mh-unknown`.
This is because icu4c configure doesn't detect `emscripten` platform and
"fallback" to `mh-unknown`.
Aria2 is a special case. We actually don't really need to compile it.
kiwix-desktop through libkiwix use the aria2c binary but we always
use (package) the system binary or a prebuild release.
And we don't cross compile kiwix-desktop to windows, so we don't need
aria2c neither.
The sub-directory `android-kiwix-lib-publisher` has not been renamed.
We plan to move this code in another repository anyway.
It is better for now to use the existing name instead of change the
directory.
Wrong copy/paste.
This is not important as we reference dependencies by their name
attribute, not by the class name.
But it is better to have a correct name.
The source step (install source) always exists.
The `plaformInfo.name` step may not exists
(On mixed target, we build all deps static and the last project dyn)
So the `native_dyn` doesn't exists for `zim-testing-suite`.
Matthieu Gautier
Emmanuel Engelhart
renaud gaudin
ChrisLi
renaud gaudin
Veloman Yunkan