Commit Graph

210 Commits

Author SHA1 Message Date
Matthieu Gautier c4043f7534 [TOREVERT] Use fix_manylinux_compile_release br for libzim 2024-06-24 11:46:38 +02:00
Emmanuel Engelhart 4b6c4a2547
Use version 0.6.0 of ZIM testing suite 2024-06-22 14:55:32 +02:00
Matthieu Gautier e31147c7e5 Revert "WIP Use windows branch on windows CI"
This reverts commit 05fc3b409c.

This was a temporary commit to make the PR compile with a dev branch of libzim.
Now PR on libzim is merged. We must remove this commit.
2024-06-17 10:11:03 +02:00
Matthieu Gautier 05fc3b409c WIP Use windows branch on windows CI 2024-06-07 18:34:30 +02:00
Matthieu Gautier d2d1f686b1 Build libzim on windows with werror=false
MSVC throw a lot of warning because we also must re-export stl symbols
used in our exported class.

See [1] and [2] (and links inside) for more information.

Especially [3] which seems to say that we should have issue at link time if
it compatibility issue occurs (and so easily catchable).

So I just remove the `werror=true` for now.

[1] https://stackoverflow.com/questions/16419318/one-way-of-eliminating-c4251-warning-when-using-stl-classes-in-the-dll-interface
[2] https://stackoverflow.com/questions/2132747/warning-c4251-when-building-a-dll-that-exports-a-class-containing-an-atlcstrin
2024-06-07 18:33:55 +02:00
Matthieu Gautier 46e58b763d Don't depend on lzma. 2024-06-07 15:38:45 +02:00
Matthieu Gautier 6bfe9ec8b1 Do not build libzim with xapian dependency on Windows. 2024-06-07 15:38:05 +02:00
Matthieu Gautier 0e0ac010bb Build icu4c using meson on Windows. 2024-06-07 15:36:58 +02:00
Matthieu Gautier ca3d90d793 Use the correct path separator on Windows 2024-06-07 15:36:58 +02:00
Matthieu Gautier baa4470ebb Use setup command in meson. 2024-06-07 15:36:58 +02:00
Matthieu Gautier 3b0532b395 Update zstd 2024-06-07 15:36:58 +02:00
Matthieu Gautier 45255078f9 Build lzma using meson build system. 2024-06-07 15:36:58 +02:00
Matthieu Gautier 04942020d7 Fix extracting of several archives.
- Archive already existing must not stop download of other archives.
- Use copytree instead of rename to use second archive as patch on existing
  directory. (No need to remove tmpdir, as it is a temporary directory)
2024-06-07 15:36:58 +02:00
Matthieu Gautier 941cbc8310 Build zlib using meson.
Use the wrap coming from meson's wrapdb.
2024-06-07 15:36:58 +02:00
Matthieu Gautier 62dc698fad Build only libzim (and dependencies on Windows) 2024-06-07 15:36:54 +02:00
Matthieu Gautier 683250ca9d Use manylinux container instead of bionic
Alignement of the table in build_definition will be fixed in later commit
to not mix functionnal and cosmetic changes in the same commit.
2024-06-03 10:42:49 +02:00
Matthieu Gautier 0f82aff3c1 New version 0.5 for zim-testing-suite 2024-02-23 17:06:32 +01:00
Matthieu Gautier c0ec9c44b8 Rename option `--target-platform` to `--config`
The target_platform option has always be wrongly named.
This is not the platform we are targeted but how we compile.

This was ok at beginning as specifying how we compile somehow define
for what we compile but this is not a one to one mapping.
2024-02-05 18:03:11 +01:00
Matthieu Gautier 5a1175cf2d Format our code with black 2024-02-05 11:41:09 +01:00
renaud gaudin 8c5bca05b7
Updated CI and builddef for renamed platform ; including new arm64 simulator 2023-12-05 07:04:53 +00:00
renaud gaudin afb7c504ea
Fixed #667: CoreKiwix for iOS Simulator
When we introduced CoreKiwix.xcframework, we made two mistakes:

- We forgot that some devs are on Apple Silicon and did not include arm64 arch for simulator
- We kept the `x86_64-apple-ios` target thinking simulator is a “just” an iOS on x64

This target mistake prevented XCode from finding our lib usable with the Simulator on x64.
Not including an arm64 version obviously prevented Silicon-using devs from using the Simulator.

The included libs changes now from

| Name                         | Content                                               |
| ---------------------------- | ----------------------------------------------------- |
| `ios-arm64`                  | All iOS devices are arm64                             |
| `ios-x86_64-simulator`       | Single x64 binary for Simulator                       |
| `macos-arm64_x86_64`         | Fat binary for macOS with both x64 and arm64          |

To

| Name                         | Content                                               |
| ---------------------------- | ----------------------------------------------------- |
| `ios-arm64`                  | All iOS devices are arm64                             |
| `ios-arm64_x86_64-simulator` | Fsat binary for iOS Simulator with both x64 and arm64 |
| `macos-arm64_x86_64`         | Fat binary for macOS with both x64 and arm64          |

- `iOSx64` Platforn renamed to `iOSx64Simulator` with its target fixed to `x86-apple-ios-simulator` (was `x86_64-apple-ios`)
- Added platformn `iOSArm64Simulator` for Apple Silicon devs to run Simulator
- `AppleXCFramework` dependency gets two changes:
    - Depends on all those platforms
    - `_make_macos_fat()` turned generic and called both for macOS fat binary and iOS Simulator one
2023-12-04 14:20:24 +00:00
Matthieu Gautier ab273f7d12 Run apple command as list. 2023-12-01 11:12:32 +01:00
Matthieu Gautier b7c4854d83 Build only libzim dependencies on bionic.
We compile only libzim on bionic, for compatibility with old libc.
No need to compile all other dependencies (and fight with some of them not
compiling)
2023-12-01 11:12:32 +01:00
Matthieu Gautier a0978a6ab4 Remove SvnClone.
We don't use it since a long time.
2023-12-01 11:12:32 +01:00
Matthieu Gautier 985643089a Detect the command instead of hardcoding them. 2023-12-01 11:11:26 +01:00
Matthieu Gautier c99a9bd91f Run the command without using shell=True.
It mainly allow to run command in directory containing space.
(Hello, `C:\Program Files\...`)
It would also allow to work on directory containning spaces
(`C:\Users\John Doe`) but xapian configure (at least) expressly doesn't
support it :/

- Run the command without shell=True
- The command must be a list instead of a string.
- All options must also be a list (or an iterable).
2023-12-01 11:11:24 +01:00
Matthieu Gautier e87835c61d Correctly set the environment.
When running command without `shell=True`, python's subproccess will look
for variable `"PATH"` and `b"PATH"` and will complain if both are set.
So the defaultdict should not return something for `b"PATH"`.

We also escape space ` ` in the environment variables to not break
everything with directory containing space.
2023-12-01 11:10:34 +01:00
Matthieu Gautier 88cc75475c Build libzim and libkiwix in debug mode on android.
Fix #623
2023-11-20 11:10:24 +01:00
Matthieu Gautier 861682ecaa Update emsdk version. 2023-11-17 12:28:14 +01:00
Emmanuel Engelhart 6a9093642c Bump-up aria2 to 1.37.0 2023-11-17 12:21:39 +01:00
renaud gaudin ac22caca6d
Don't set context.no_skip on meson builder
no_skip is thus computed automatically depending on project being our or not.
We dont want to skip configure on our own projetcs (limited impact, simplifies deps mgmt)
2023-11-15 08:50:38 +00:00
renaud gaudin a063b91349
Build CoreKiwix.xcframework
CoreKiwix.xcframework is the packaging format required by Kiwix apple (macOS/iOS) reader for libkiwix.
Naming will hopefully be revisited later (libkiwix.xcframework?)

This folder is a combination of all static archive (*.a): dependencies and libkiwix itself
for all apple-supported platforms: macOS x86_64, macOS arm64, iOS arm64 and iOS x86_64.
It also includes the headers.
Note: while iOS archs are separated in the framework, the macOS archs are bundled as a fat binary.

This thus adds:

- A new `apple_all_static` target that lists mentioned sub-platforms.
- A new `AppleXCFramework` virtual dependency and builder
  -  requires `libkiwix`
  - merges the archives into one per target
  - creates the fat binary for macOS
  - creates the xcframework
- `libkiwix_xcframework-VERSION.tar.gz` because platformname is xcframework
  - it only contains the xcframework but in libkiwix_xcframework-VERSION/lib folder
    Because with this virtual target, there are no other files
  - requires a new `xcframework` platform_name in builddef
- subplatforms deps and macOS/iOS _ok files to make_deps_archive

---

https://developer.apple.com/documentation/xcode/creating-a-multi-platform-binary-framework-bundle
2023-11-15 08:50:00 +00:00
renaud gaudin 09dacaa25e
disabled bitcode support, as deprecated since xcode14
See https://stackoverflow.com/questions/72543728/xcode-14-deprecates-bitcode-but-why
2023-11-09 08:14:52 +00:00
Matthieu Gautier 2a5a5c3a42 Correctly build ICU data when building for wasm.
When we use `--disable-tools` (ie, when we are cross-compiling), ICU
will build the data only if it detects we are cross-compiling.

This make sens until we cross-compiling for wasm.
Wasm output is a script file with the interpreter in the sdk we have
installed. So ICU's configure can run the test binary and so doesn't detect
we are cross-compiling.

So we have to patch ICU Makefile to make it build the ICU data anyway.
2023-11-02 17:33:51 +01:00
Matthieu Gautier beadc3b16c Use custom data.
ICU is used to:
- List locales (`locales_tree`)
- Get information about those locales in different languages (`lang_tree`)
- Remove accents (`translit`)

Tracing ICU without custom data, we also load `likelySubtags` and
`metadata`. Not sure why and what happen when missing but let's be
conservatives and include them.
2023-11-02 11:14:04 +01:00
Matthieu Gautier 717dcd801d Move to ICU 73.2 2023-11-02 11:14:04 +01:00
Matthieu Gautier 8cf6c39d11 Extend `ReleaseDownload` source to be composed of several archives. 2023-11-02 11:13:29 +01:00
Emmanuel Engelhart e966385069 Bump-up Xapian to version 1.4.23 2023-10-31 11:39:29 +01:00
Matthieu Gautier b57f83ed25 Move all toolchain installation in TOOLCHAIN directory.
This is a clearer install and it simplify the base_deps archive creation.
2023-10-24 10:53:31 +02:00
Matthieu Gautier 149c1d25fb Use musl toolchain with gcc 10.
We need a "old" gcc version to be able to run on "old" systems
(libstdc++.6.0.28)
2023-10-23 10:19:12 +02:00
Matthieu Gautier c3d3cc5a97 Add x86_64 musl toolchain and platform. 2023-10-23 10:19:12 +02:00
Matthieu Gautier f0ad7b126a Rename `arm_musl.py` to `musl.py`.
Those files will contain toolchain for musl. Not specific to arm.
2023-10-23 10:19:12 +02:00
Matthieu Gautier 7807e10325 Rename files containing toolchain to `tc_*` 2023-10-23 10:19:12 +02:00
Matthieu Gautier 65dd03e58f Move back to the old toolchain for aarch64
Fix #634
2023-08-11 16:18:25 +02:00
Matthieu Gautier 5884909b25 Add target aarch64_musl_*
I have to mention that adding a new cross compiled target never seems
so simple.
2023-06-14 15:57:49 +02:00
Matthieu Gautier 897e7f292c Switch (again) to new arm cross-compilation toolchain.
This introduce a difference between armv6 and armv8.
2023-05-22 19:03:57 +02:00
Emmanuel Engelhart 5057444acd
Disable decompression feature in libmagic 2023-05-20 13:23:25 +02:00
Emmanuel Engelhart e0d4eea050
Bump-up libmagic to version 5.44 2023-05-19 15:28:34 +02:00
Matthieu Gautier fd85b80b2c Move back to gcc 6.3.0 when compiling to aarch64. 2023-04-26 16:54:35 +02:00
Matthieu Gautier 5ef7e2a017 New version of xapian 1.4.22
Fix #599
2023-04-25 16:54:40 +02:00