Commit Graph

154 Commits

Author SHA1 Message Date
Matthieu Gautier 7a23bb5b8b Fix creation of base dependencies archive.
The `INSTALL_DIR` was added twice. It was not a issue as we then transform
the list into a set to remove duplicated.

But with `filter_install_dir` call only on one "add", the (textual)
entries are not duplicated and so, not removed. So the files where add
twice.

Now we correctly filter initial `INSTALL_DIR` and we remove the second add.
2023-03-14 17:17:30 +01:00
Matthieu Gautier 5c0d9d87c1 Remove `share/doc` and `share/man` from the deps archive.
They are useless and they use precious disk space.
Fix #567
2023-02-22 11:57:47 +01:00
Matthieu Gautier 536748500b Remove uneeded patches for flatpak
Patches are uneeded for all arch.
But we were trying to copy them on flatpak.
2022-12-09 10:55:14 +01:00
Matthieu Gautier 07d1561a87 [CI] Build libkiwix on on native_mixed 2022-11-30 14:22:13 +01:00
Matthieu Gautier 84eea4a7ed Package all libzim dependencies on wasm.
Fix #552
2022-11-30 11:07:21 +01:00
Matthieu Gautier c27348bde3 Publish the CI only if we are on a dev branch.
If we try to publish all the time, we will conflict with `Release&Nigthly`
workflow as we would pulbish in nigthly directory.

Fix #550
2022-11-23 16:06:35 +01:00
Matthieu Gautier a44531bb8a Upload generated packages on `tmp/ci` when building on branches.
We need to be able to test our generated artefacts before we merge the
branches.
2022-11-16 16:37:05 +01:00
Matthieu Gautier ab421848c5 Use glob instead of specifying the lib_prefix.
There is no need to be specific here, we are already in INSTALL_DIR which
contains only things specific to our platform.
2022-11-16 16:37:05 +01:00
Matthieu Gautier 9881571047 Add packaging of libzim.a
Wasm build is creating a static library.
2022-11-16 16:37:05 +01:00
Matthieu Gautier 16fc490ae7 Add build of libzim on wasm in the CI 2022-11-16 16:37:05 +01:00
Matthieu Gautier 180fb249f6 Add a support to compile libzim for wasm.
`icu4c_wasm.patch` is build by :
 - Copying config.sub from liblzma source as new version of config.sub there
   knows about wasm architecture.
 - Copying `mh-linux` on `mh-unknown` as specified in (origin) `mh-unknown`.
   This is because icu4c configure doesn't detect `emscripten` platform and
   "fallback" to `mh-unknown`.
2022-11-16 16:37:05 +01:00
Matthieu Gautier dc814c626d Upload read-only archive.
By uploading read-only archive, we prevent potential (implicit) re-upload.

A re-upload will always be possible if we remove the archive and rerun
the workflow. But it will be clearly explicit in this case.
2022-09-28 16:08:42 +02:00
Matthieu Gautier f7706428a2 Make release only if we are running the workflow on a `r_xx` tag.
Also check that we are not a scheduled workflow
(But it is more a double check. This shloud not happen)
2022-09-28 15:09:53 +02:00
renaud gaudin fe92fc5080 Use notarytool to notarize on macOS
altool has been deprecated in favor of notarytool
2022-09-06 14:52:15 +00:00
Matthieu Gautier 33c2ec2c4d Build and release libzim for android on the CI 2022-06-28 12:24:06 +02:00
Matthieu Gautier 6a3d3047e0 Do not try to package kiwix-search as it as been removed from kiwix-tools 2022-06-15 14:28:32 +02:00
Matthieu Gautier dba45ced36 Build android lib in the correct directory 2022-06-14 17:48:51 +02:00
Matthieu Gautier 9c220866ac Build and release each libkiwix android builds independently.
While it is ok to build all libkiwix android builds in one step,
the "release system" upload only one archive per platform.
So we need 4 platforms to do 4 uploads.

As we don't build on "android" platform now, we can clean up our scripts.
2022-06-10 10:26:24 +02:00
Matthieu Gautier 7086009a0a We don't need to upload to bintray now. 2022-06-10 10:11:01 +02:00
Matthieu Gautier 51ff5e58bb Be able to make the release of libkiwix on android. 2022-06-09 10:31:09 +02:00
Matthieu Gautier 72c271a74c Remove LibkiwixApp
The project was used to assemble all android compilation in one
multiplatform java archive.

This allow use to remove android_sdk and Gradle builder
2022-06-09 10:28:59 +02:00
Matthieu Gautier 733a730877 Do not notarize macos build on CI.
We still sign macos build on nigthly and release.
2022-05-17 18:05:43 +02:00
renaud gaudin f103cddd77 Trust codesign using the certificate
When build_release_nightly calls codesign to sign libzim.7.dylib, it appears to be
hanging forever.
What's most likely happening is that Keychain Access is prompting a password request
without any possibility to answer, given this is running on the CI.

It's unclear whether Keychain Access wants to confirm codesign can access the certificate
or if it is trying to unlock another (System) keychain to find the certificate or key.

This addresses the former.
2022-05-12 14:58:46 +00:00
renaud gaudin 8ec2cd9a11 Using sftp for mkdir step
As there is no shell, use SFTP's mkdir command to create the folders up to final path.
2022-04-04 13:53:20 +00:00
renaud gaudin 1351fc7f8b Target uploads to new dropbox address
SSH server which was used to receive file uploads (CI, nightly and release) has been
migrated to a new one on a different address.
Username, Key and paths are unchanged.
Most notable changes are the use of `master.download.kiwix.org` as the target in
replacement of `mirror.download.kiwix.org` (although it would still work) and the
Port to which SSH is listening on (30022 instead of 22)
2022-03-30 22:10:06 +00:00
Matthieu Gautier 64ea01e445 [CI] Do not build zim-tools on armhf
zim-tools depends of docopt which need a recent enough version of gcc (4.9)
But we are using gcc 4.8.3 to cross compile on armhf.
2022-03-10 18:16:39 +01:00
Matthieu Gautier 6505e82c37 [CI] Do not build zim-tools on macOS cross-compilation target.
We still build zim-tools on macOs with native build.
2022-03-10 16:57:56 +01:00
Matthieu Gautier af3cff6aeb Build flatpak in CI 2022-03-10 16:21:14 +01:00
Matthieu Gautier 01e8f92348 Build the projects in the CI.
We used to build only the base dependencies in the CI
(and build the projects in nightly&Release workflow).
Now always build the projects.
2022-03-10 15:19:51 +01:00
Matthieu Gautier 0fb7dc63a7 Correctly detect if project is openzim or kiwix organization 2022-02-02 16:10:28 +01:00
Matthieu Gautier aa349a18e7 Rename kiwix-lib to libkiwix.
The new name of kiwix-lib is libkiwix. Let kiwix-build adopt the name.
2021-06-30 16:45:54 +02:00
Matthieu Gautier ed999d61af [CI] Do not try to install packages in the CI.
On the CI, packages should be already installed.
Don't try to install with kiwix-build.
2021-06-23 17:27:52 +02:00
Matthieu Gautier c445db171b Package zimrecreate in zim-tools. 2021-06-10 14:25:54 +02:00
Matthieu Gautier 1845879825 Add all files of zim-testing-suite to the generated base_deps archive. 2021-05-04 14:36:15 +02:00
renaud gaudin ea55cac32d Fixed #469: Notarizing libzim release for macOS
This adds the notarization (see #469) of the libzim binary for macOS during the build.
It it not dependent on RELEASE so it benefits all builds.

It basically does two things:
- sign the build with our Developer ID certificate from Apple.
- Request notarization from Apple for the binary.
At the moment, it concerns only libzim. Might expand that to libkiwix and the zim/kiwix tools
once we start releasing those.

Github Actions prepare the certificate and environment, and signing+request is done in `notarize_macos_build()` (common.py)

It required the following new secrets:

| secret | value |
|---|---|
| `APPLE_SIGNING_CERTIFICATE` | base64 of the P12 certificate |
| `APPLE_SIGNING_P12_PASSWORD` | password for the P12 certificate (we chose that when exporting to P12. Apple doesnt provide P12) |
| `APPLE_SIGNING_IDENTITY`| Common name of our certificate. Not a private info but seems better suited there than in the CI |
| `APPLE_SIGNING_TEAM`| Apple Developer Team ID (mentionned in the signing identity) |
| `APPLE_SIGNING_ALTOOL_PASSWORD`| app-specific password created to request notarization |
| `APPLE_SIGNING_ALTOOL_USERNAME`| username associated with the app-specific password. Must be an Apple ID with perms on the Certificate. Currently mine. |
2021-02-09 13:30:24 +00:00
renaud gaudin ff66a96980 Fixed #465: trigger docker publish on release
This triggers a `workflow_dispatch` event on the `docker.yml` workflow or the matching
repository for both `zim-tools` and `kiwix-tools` targets that supports it.
2021-01-13 15:58:33 +00:00
Matthieu Gautier 1dde278ad6 Correctly set the extra postfix for android kiwix-lib. 2020-08-27 19:20:05 +02:00
Matthieu Gautier a3038253e8 Do not fail if there is no file to publish. 2020-07-15 17:42:32 +02:00
Matthieu Gautier e81a2e7006 Merge tag 'r_10' 2020-07-15 16:20:16 +02:00
Alexander Sashnov da31dd428e remove 'zimwriterfs' project from build and CI configuration
'zimwriterfs' is now a part of zim-tools (openzim/zim-tools/#92)
2020-07-08 16:54:14 +02:00
Matthieu Gautier 2d1e8d0202 Also do the kiwix-lib release (for source publication) 2020-07-02 15:38:24 +02:00
Matthieu Gautier 7afa3f6a52 Correctly build release of kiwix-lib on android.
The name of the target is `kiwix-lib-app` on android but the key
in the version directory is `kiwix-lib`.
2020-07-01 11:30:26 +02:00
renaud gaudin 8478d65c90 enable compilation of native_mixed on macos
Issue #349 requires a native_mixed for macOS.
native_mixed is working for libzim so we whitelist it.

On the release CI, we fix the macos dylib rpath so it doesn't include the
full build-install step path which is probematic for a distributed file.

We build native_mixed for OSX in both CI and release mode
2020-06-17 09:04:10 +00:00
Matthieu Gautier 80c59b246f Change kiwixbot email address to the new one. 2020-06-03 10:48:41 +02:00
Matthieu Gautier 497967236e Publish release source archive on "native_mixed" platform.
`libzim` is build on "native_mixed", not "native_dyn".
2020-05-25 13:08:57 +02:00
Matthieu Gautier 22da0f3b6d Fix postfix generation to not include the `0`. 2020-05-25 12:21:10 +02:00
Matthieu Gautier 040171129a Fix release tag format.
The release tag format change to a classic tag triplet format `\d.\d.\d`
to a custom one `r_\d`.
2020-05-25 12:21:10 +02:00
Matthieu Gautier e020eb35ac [WIP] Move the release process to github actions. 2020-05-21 12:13:36 +02:00
Matthieu Gautier fb07b58812 Fix recompilation of the CI of meson project.
We were assuming that meson project correspond to our projects and so we
were always building them, even if they were already compiled.
(This way, a simple `kiwix-build` is enough to recompile the WIP code of
our project).

However, on the CI, we do not archive the source code/build directory in
the base deps archive. So when we try to compile, the compile step of
meson projects fails because the source are not here.
We have a small workaround for pugixml but as zstd is also meson, it is
time to do something correct.

By default, all projects now try to skip if a build is already present.
Our projects are marked as `force_build` and so, they do not try to skip.
2020-04-06 19:15:40 +02:00
Matthieu Gautier 08ed650c78 [CI] Clone git repository with a depth of 1 in the CI. 2020-02-25 15:53:23 +01:00
Matthieu Gautier 1b7a2c812e Add missing platforms.
No iOs for now.
2020-01-21 13:49:12 +01:00
Matthieu Gautier 1d84266751 Build all base dependencies.
Use a custom script for this.
Set `SKIP_BIG_MEMORY_TEST` to avoid launching big test on libzim.
2020-01-21 13:49:12 +01:00
Matthieu Gautier c178c6ea35 Upload an archive of the build directory in case of failure. 2020-01-21 13:49:12 +01:00
Matthieu Gautier d37acc2cbe Build basedeps archives on GithubAction.
Use a specific script to download/build the base deps.

Mainly based on script in travis/compile_all.py
2020-01-21 13:49:12 +01:00