Background
==========
Shared Library Dependencies
---------------------------
The Linux build of Arduino IDE has dynamic linkage against the libstdc++ and glibc shared libraries. This results in
it having a dependency on the version of the libraries that happens to be present in the environment it is built in.
Although newer versions of the shared libraries are compatible with executables linked against an older version, the
reverse is not true. This means that building Arduino IDE on a Linux machine with a recent distro version installed
causes the IDE to error on startup for users who have a distro with older versions of the dependencies.
For example, if Arduino IDE were built on a machine with version 3.4.33 of libstdc++, then attempting to run it on a
machine with an older version of libstdc++ would fail with an error like:
```
Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.33' not found (required by /home/foo/arduino-ide/resources/app/lib/backend/native/nsfw.node)
```
Likewise, if Arduino IDE were built on a machine with version 2.39 of glibc, then attempting to run it on a machine
with an older version of glibc would fail with an error like:
```
Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.39' not found (required by /home/foo/arduino-ide/resources/app/node_modules/nsfw/build/Release/nsfw.node)
```
Build Machine Requirements
--------------------------
The IDE builds distributed by Arduino should be compatible with a reasonable range of Linux distribution versions. In
order to achieve this, the builds must be performed in a machine with an older version of the shared libraries. The
shared libraries are part of the Linux distro, and installing a different version is not feasible. So this imposes a
maximum limit on the build machine's distro version.
The distributed builds are generated via a GitHub Actions workflow. The most simple approach is to run the build in the
machine of the GitHub-hosted runners provided for each operating system. However, GitHub provides a limited range of
operating system versions in their runners, and removes the older versions as newer versions are added. This means that
building in the GitHub-hosted runner machine would not allow for the desired range of Linux distro version
compatibility. For this reason, the Linux build is performed in a Docker container that provides an older version of
Ubuntu.
The same situation of incompatibility with Linux distro versions that have a version of the shared library dependencies
older than the version present on the build machine occurs for several of the tools and frameworks used by the build
process (e.g., Node.js, Python). In this case, the tables are turned as we are now the user rather than the distributor
and so are at the mercy of the Linux distro version compatibility range provided by the distributor. So this imposes a
minimum limit on the build machine's distro version.
Although several of the dependencies used by the standard build system have dependencies on versions of glibc higher
than the version 2.27 present in Ubuntu 18.04, it was possible to use this distro version in the Linux build container
by using alternative distributions and/or versions of these dependencies.
Workflow Artifacts
------------------
The build workflow uses GitHub actions workflow artifacts to transfer the files generated by the build job to subsequent
jobs in the workflow. The "actions/upload-artifact" action is used for this purpose.
Problem
=======
GitHub is dropping support for the workflow artifacts produced by the version 3.x of the "actions/upload-artifact"
action that was previously used by the build job. So the action version used in the build workflow was updated to the
current version 4.x. This version of the action uses a newer version of the Node.js runtime (20). Unfortunately the the
Node.js 20 runtime used by the action has a dependency on glibc version 2.28, which causes the Linux build job to fail
after the update of the "actions/upload-artifact" action:
```
Run actions/upload-artifact@v4
/__e/node20/bin/node: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
```
Unlike the other dependencies of the build process, it is no longer possible to work around this incompatibility by
continuing to use the older compatible version of the "actions/upload-artifact" action. It is also impossible to replace
the incompatible Node.js 20.x distribution used by the action, since it comes from the read-only file system of the
runner image. Likewise, it is not possible to configure or force the action to use a Node.js installation at a different
path on the runner machine.
Resolution
==========
Compatibility with the new version of the "actions/upload-artifact" action is attained by updating the version of Linux
in the build container to 18.10, which is the oldest version that has glibc 2.28. The presence of a newer glibc version
in the container also makes it compatible with several other dependencies of the build process, meaning the code in the
Dockerfile and workflow for working around the incompatibilities of Ubuntu 18.04 can be removed.
Consequences
============
Unfortunately this means the loss of compatibility of the Linux Arduino IDE builds with distros that use glibc 2.27
(e.g., Ubuntu 18.04). User of those distros will now find that Arduino IDE fails to start with an error like:
```
Error: node-loader:
Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /home/foo/arduino-ide/resources/app/lib/backend/native/pty.node)
at 85467 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:2766)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 23571 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:3374073)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 55444 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:3369761)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 24290 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:1780542)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
at 43416 (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:1770138)
at __webpack_require__ (/home/foo/arduino-ide/resources/app/lib/backend/main.js:2:6663105)
```
The build workflow produces binaries for a range of target hosts. This is done by using a job matrix in the GitHub
Actions workflow that produces each build in a parallel job. GitHub Actions workflow artifacts are used to transfer the
generated files between sequential jobs in the workflow. The "actions/upload-artifact" action is used for this purpose.
Previously, a single artifact was used for this purpose, with each of the parallel jobs uploading its own generated
files to that artifact. However, support for uploading multiple times to a single artifact was dropped in version 4.0.0
of the "actions/upload-artifact" action. So it is now necessary to use a dedicated artifact for each of the builds.
These can be downloaded in aggregate by using the artifact name globbing and merging features which were introduced in
version 4.1.0 of the "actions/download-artifact" action.
The "Sync Labels" GitHub Actions workflow is configured to allow the use of multiple shared label configuration files.
This is done by using a job matrix in the GitHub Actions workflow to download each of the files from the source
repository in a parallel GitHub Actions workflow job. A GitHub Actions workflow artifact was used to transfer the
generated files between sequential jobs in the workflow. The "actions/upload-artifact" and "actions/download-artifact"
actions are used for this purpose.
Previously, a single artifact was used for the transfer of all the shared label configuration files, with each of the
parallel jobs uploading its own generated files to that artifact. However, support for uploading multiple times to a
single artifact was dropped in version 4.0.0 of the "actions/upload-artifact" action. So it is now necessary to use a
dedicated artifact for each of the builds. These can be downloaded in aggregate by using the artifact name globbing and
merging features which were introduced in version 4.1.0 of the "actions/download-artifact" action.
The "build" workflow builds the application for all supported targets, generates workflow artifacts from which the
builds can be downloaded by users and beta testers, and publishes nightly and production releases.
As if that wasn't enough, the workflow was also configured to check the sync of the Yarn lockfile.
This monolithic approach is harmful for multiple reasons:
* Makes it difficult to interpret a failed workflow run
* Makes the build workflow more difficult to maintain
* Increases the turnaround time for contributors and maintainers to get feedback from the CI system
The sync check operation is hereby moved to a dedicated workflow, consistent with standard practices for Arduino Tooling
projects.
The "build" workflow builds the application for all supported targets, generates workflow artifacts from which the
builds can be downloaded by users and beta testers, and publishes nightly and production releases.
As if that wasn't enough, the workflow was also configured to perform the unrelated operation of running the project's
test suites.
This monolithic approach is harmful for multiple reasons:
* Makes it difficult to interpret a failed workflow run
* Unnecessarily adds a significant amount of extra content to the already extensive logs produced by the build process
* Makes the build workflow more difficult to maintain
* Increases the length of a build workflow run
* Increases the impact of a spurious failure
* Increases the turnaround time for contributors and maintainers to get feedback from the CI system
The test run operation is hereby moved to a dedicated workflow, consistent with standard practices for Arduino Tooling
projects.
The "build" workflow builds the application for all supported targets, generates workflow artifacts from which the
builds can be downloaded by users and beta testers, and publishes nightly and production releases.
As if that wasn't enough, the workflow was also configured to perform the unrelated operation of linting the project's
TypeScript and JavaScript code.
This monolithic approach is harmful for multiple reasons:
* Makes it difficult to interpret a failed workflow run
* Unnecessarily adds a significant amount of extra content to the already extensive logs produced by the build process
* Makes the build workflow more difficult to maintain
* Increases the length of a build workflow run
* Increases the impact of a spurious failure
* Increases the turnaround time for contributors and maintainers to get feedback from the CI system
The linting operation is hereby moved to a dedicated workflow, consistent with standard practices for Arduino Tooling
projects.
The "build" workflow signs the macOS and Windows builds of the application. The signing process relies on access to GitHub Actions
secrets. For this reason, the workflow is configured to only sign the builds when it has access to GitHub Actions
secrets to avoid spurious failures of the workflow that would otherwise be caused by signing failure.
A flexible general purpose system for determining whether to attempt signing of a build was established years ago. However, a redundant system was added specific to the Windows build instead of using the existing system.
The redundant system is hereby removed. This makes the workflow easier to understand and maintain.
The "build" workflow signs the Windows builds of the application. The signing process relies on access to GitHub Actions
secrets. For this reason, the workflow is configured to only sign the builds when it has access to GitHub Actions
secrets to avoid spurious failures of the workflow that would otherwise be caused by signing failure.
Previously the signing was determined based on the value of the `github.event.pull_request.head.repo.fork` context item.
That was effective for the use case of the workflow being triggered by a pull request from a fork (for security reasons,
GitHub Actions does not give access to secrets under these conditions).
However, there is another context under which the workflow might run without access to the signing secrets, for which
the use of context item is not appropriate. It is important to support the use of the workflow in forks of the
repository. In addition to the possible value to hard forked projects, this is essential to allow conscientious
contributors to test contributions to the build and release system in their own fork prior to submitting a pull request.
The previous configuration would cause a workflow run performed by a contributor in a fork to attempt to sign the
Windows build. Unless the contributor had set up the ridiculously complex infrastructure required to perform the signing
for the Windows build, which is utterly infeasible, this would cause the workflow to fail spuriously.
The appropriate approach, which has been the established convention in the rest of the workflow code, is to use the
secret itself when determining whether to attempt the signing process. If the secret is not defined (resulting in it
having an empty string value), then the signing should be skipped. If it is defined, then the signing should be
performed.
The "build" workflow builds the application for a range of target hosts. This is done by using a job matrix. A separate
parallel job runs for each target. The target-specific configuration data is defined in the job matrix array.
This configuration data includes the information related to the code signing certificates. Inexplicably, during the work
to add support for signing the Windows builds with an "eToken" hardware authentication device, this data was not used
for the Windows code signing configuration. Instead the certificate data was redundantly hardcoded into the workflow
code.
The Windows code signing certificate configuration is hereby changed to use the established flexible job configuration
data system. This makes the workflow easier to understand and maintain.
The Windows builds of the application are cryptographically signed. The signing requires an "eToken" hardware
authentication device be connected to the machine performing the signing. This means that it is necessary to use a
self-hosted GitHub Actions runner for the Windows job of the build workflow rather than the runners hosted by GitHub.
There are some unique characteristics of the self-hosted runner which the workflow code must accommodate. One of these
is that, rather than installing dependencies of the build process during the workflow run as is done for the
GitHub-hosted runners, the dependencies are preinstalled in the self-hosted runner machine. So the dependency
installation steps must be configured so that they will be skipped when the job is running on the self-hosted runner.
This is done by adding a conditional to the steps. Previously the conditional was based on the value of the `runner.os`
context item. This is not an appropriate indicator of the job running on the self-hosted runner because `runner.os` will
have the same value if the job was running on a GitHub-hosted Windows runner. That might seem like only a hypothetical
problem since the workflow does not use a GitHub-hosted Windows runner. However, it is important to support the use of
the workflow in forks of the repository. In addition to the possible value to hard forked projects, this is essential to
allow conscientious contributors to test contributions to the build and release system in their own fork prior to
submitting a pull request.
The conditionals are changed to use the more appropriate indicator of the specific name of the self-hosted Windows
runner (via the `runner.name` context item).
The Windows builds of the application are cryptographically signed. The signing requires an "eToken" hardware
authentication device be connected to the machine performing the signing. This means that it is necessary to use a
self-hosted GitHub Actions runner for the Windows job of the build workflow rather than the runners hosted by GitHub.
There are some unique characteristics of the self-hosted runner which the workflow code must accommodate. The default
working directory of the self-hosted runner is not suitable to perform the build under because the the resulting folder
structure produced paths that exceeded the ridiculously small maximum path length of Windows. So the workflow must be
configured to use a custom working directory with a short path (`C:\a`).
This custom working directory must be used only for the job running on the self-hosted Windows runner so the working
directory of the relevant workflow steps are configured using a ternary expression. Previously, this expression had
multiple conditions:
* the value of the `runner.os` context item
* the definition of a custom working directory value in the job matrix
The second condition is entirely sufficient. The use of the first condition only added unnecessary complexity to the
workflow code, and imposed a pointless limitation of only allowing the use of the custom working directory system on
Windows runners.
Removing the unnecessary condition makes the workflow easier to understand and maintain, and makes it possible to
configure any job to use a custom working directory if necessary.
The build workflow produces builds for a range of target host architectures, including macOS Apple Silicon. This is done
by running a native build in a machine of the target architecture.
At the time the support for producing Apple Silicon builds was added to the workflow, use of GitHub-hosted Apple Silicon
runner machines was charged by the minute (while use of the other runners is free). In order to avoid excessive
expenses, the workflow was configured so that the Apple Silicon builds were only produced when absolutely necessary.
This was done by defining two sets of job matrix arrays, one for jobs using free runners, and the other for jobs using
paid runners. Due to the limitations of the GitHub Actions framework, it was necessary to use workflow environment
variables for this purpose.
Since that time, GitHub made free GitHub-hosted Apple Silicon runners available. When the workflow was adjusted to use
that runner, the configuration for the Apple Silicon build job was moved to the free runner job matrix array. The system
for supporting selective use of paid GitHub-hosted runners to produce builds was not removed at that time. This is
reasonable since it is possible the need will arise for using paid runners at some point in the future (e.g., only
legacy older versions of free macOS "Intel" runners are now provided and it is likely that even these will eventually be
phased out forcing us to use the paid runner to produce builds for that target). However, the environment variable for
the paid runner job matrix array data was removed. The absence of that variable made it very difficult to understand the
workflow code for the system.
For this reason, the environment variable is replaced, but empty of data. A comment is added to explain the reason for
this.
Unfortunately the latest v2 version of the arduino/setup-task action used to install the Task task runner tool in the
runner machine has a dependency on a higher version of glibc than is provided by the Linux container. For this reason,
the workflow is configured to use arduino/setup-task@v1 for the Linux build job.
We will receive pull requests from Dependabot offering to update this outdated action dependency at each subsequent
major version release of the action (which are not terribly frequent). We must decline the bump of the action in that
specific step, but we can accept the bumps of all other usages of the action in the workflows. Dependabot remembers when
you decline a bump so this should not be too bothersome.
A Docker container is used to produce the Linux build of Arduino IDE.
The dependencies of the build are pinned to a specific version in the Dockerfile in order to ensure a stable environment
in the images.
One such dependency is Git. The version of Git available from the package repository of the Ubuntu 18.04 distro used for
the image is too outdated to be used for this purpose. For this reason, a PPA is used as the source for the Git package.
Unfortunately the maintainers of the PPA remove the previous package every time they add a package for a newer version
of Git. This breaks the version pinned installation of the Git package:
5.515 E: Version '1:2.42.0-0ppa1~ubuntu18.04.1' for 'git' was not found
For this reason it is necessary to unpin Git in the Dockerfile. This will cause whichever version is available from the
PPA to be installed each time the image is built. Although not ideal for this application, that shouldn't cause any
problems in practice since Git has quite a stable and carefully maintained interface.
Unfortunately the latest v5 version of the actions/setup-go action used to set up the Go programming language in the
runner machine has a dependency on a higher version of glibc than is provided by the Linux container. For this reason,
the workflow is configured to use actions/setup-go@v4 for the Linux build job. We will receive pull requests from
Dependabot offering to update this outdated action dependency for at each subsequent major version release of the action
(which are not terribly frequent). We must decline the bump of the action in that specific step, but accept the bumps of
all other usages of the action in the workflows. Dependabot remembers when you decline a bump so this should not be too
bothersome.
The "Arduino IDE" GitHub Actions workflow uses a Docker container for the build job of certain target, while running others directly in the
runner environment.
Due to differences between these two environments, some steps must run only when a container is used by the job, and others only when the job is running in the runner environment. This is done by a conditional on the job matrix data regarding the container. The container value is set to null for the jobs that run in the runner environment, while containing a full container configuration mapping for the jobs that run in a container.
Previously the conditional unnecessarily used the value of the image key of the container object specifically. The comparison
can be done against the container value itself. Doing this will make it a little easier to understand the workflow code, since the conditional more clearly reflects the matrix (where `container` is set to `null` instead of `container.image`).
Python 3.12.x is incompatible with the current version of node-gyp (9.4.0). For this reason, it is necessary to
configure the "GitHub Actions" workflow to use the compatible Python 3.11.x until the next release of node-gyp (which
should contain a fix for the breakage) is made.
In addition to the builds, when a nightly or production release is published, a "channel update info file" is also
uploaded to Amazon S3 by the "Arduino IDE" GitHub Actions workflow. The IDE checks the channel file on the server to get
information about available updates.
Previously the Linux production release builds were being remade manually due to the ones produced by GitHub Actions not
being compatible with older distro versions due to a dynamically linked dependency. For this reason, a step was
temporarily added to the workflow to cause it to not upload the Linux channel file in order to avoid Linux users from
receiving an update offer before the limited compatibility automated build had been replaced with the manually produced
build.
The automated build system has been adjusted to produce Linux builds with the intended range of compatibility, but the
step that deleted the channel file was not removed at that time. The obviated step is hereby removed in order to allow
complete releases to be published by the workflow.
The "Arduino IDE" GitHub Actions workflow is used to generate several distinct types of builds:
- Tester builds of commits
- Nightly builds
- Release builds
Different actions must be performed depending on which type of build is being produced. The workflow uses dedicated jobs
for publishing the nightly builds and for publishing the release builds. Those jobs are configured to run only when
certain criteria are met.
One such criteria is that the merge-channel-files job ran as expected. There are four possible result types of a job,
which should be handled as follows:
| Result | Run dependent job? |
| -------- | ------------------ |
| success | Yes |
| failure | No |
| canceled | No |
| skipped | Yes |
GitHub Actions automatically takes the desired action regarding whether the dependent job should run for the first three
result types, but that is not the case for the "skipped" result. The merge-channel-files job dependency is skipped when
a channel file merge is not needed and so this is not cause to cancel the build publishing.
The only way to make a dependent job run when the dependency was skipped is to add the `always()` expression to the job
conditional. This goes too far in the other direction by causing the job to run even when the dependency failed or was
canceled. So it is necessary to also add logic for each of the dependency job result types to the conditional, which
makes it quite complex when combined with the logic for the other criteria of the job.
In order to reduce the amount of complexity of the conditionals of the dependent jobs, a job was interposed in the
dependency chain, which was intended to act simply as a container for the logic about the merge-channel-files job
result. Unfortunately it turns out that even if the direct dependency job's result was success, if any ancestor in the
dependency chain was skipped, GitHub Actions still skips all dependent jobs without an `always()` expression in their
conditional, meaning the intermediate job was pointless. This caused the build publishing jobs to be skipped under the
conditions where they should have ran.
The pointless intermediate job is hereby removed, an `always()` expression added to the conditionals of the dependent
jobs, and the full logic for how to handle each dependent job result type added there as well.
Background
----------
The Linux build of Arduino IDE is dynamically linked against the libstdc++ and glibc shared libraries. This results in
it having a dependency on the version of the libraries that happens to be present in the environment it is built in.
Although newer versions of the shared libraries are compatible with executables linked against an older version, the
reverse is not true. This means that building Arduino IDE on a Linux machine with a recent distro version installed
causes the IDE to error on startup for users who have a distro with older versions of the dependencies. For example:
```
Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /home/per/Downloads/arduino-ide_nightly-20231006_Linux_64bit/resources/app/lib/backend/native/nsfw.node)
```
or:
```
Error: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /home/per/Downloads/arduino-ide_2.0.5-snapshot-90b1f67_Linux_64bit/resources/app/node_modules/nsfw/build/Release/nsfw.node)
```
We were originally able to achieve our targeted range of Linux distro compatibility by running the Linux build in the
ubuntu-18.04 GitHub Actions hosted runner machine. Unfortunately GitHub stopped offering that runner machine. This meant
we were forced to update to using the ubuntu-20.04 runner machine instead, which caused the loss of compatibility of
the automatically generated builds with previously supported distro versions (e.g., Ubuntu 18.04). Since that time, the
release builds have been produced manually, which is inefficient and prone to human error.
Update
------
The identified solution to restoring fully automated builds with the target range of Linux distro compatibility is to
run the build in a Docker container that provides the suitable environment. This means a combination of an older distro
version with the modern versions of the development tool dependencies of the build.
Such a combination was achieved by creating a bespoke image based on the ubuntu:18.04 image. The Dockerfile is hosted in
the Arduino IDE repository in order to allow it to be maintained in parallel with the code and infrastructure.
Image Publishing
----------------
A "Push Container Images" GitHub Actions continuous delivery workflow is added to push updated images to the GitHub
Container registry when a commit that modifies relevant files is pushed to the main branch.
This means the image does not have formally versioned tags and the IDE build uses the container that results from the
configuration at the tip of the main branch at the time of the build. I think that is a reasonable approach in this use
case where the image is targeted to a single application rather than intended to be used by multiple projects.
Container Validation
--------------------
The build workflow is configured to trigger on completion of that push workflow in order to provide validation of the IDE build using the
updated container as well as the resulting tester builds of the IDE.
A "Check Containers" GitHub Actions continuous integration workflow is added to provide basic validation for changes to
the Dockerfile. It will automatically build the image and run the container on any push or pull request that modifies
relevant files.
Container Workflow Design
-------------------------
With the goal of reusability, the image data is contained in a job matrix in the workflow to allow them to accommodate
any number of arbitrary images.
Build Workflow Configuration
----------------------------
A container property is added to the build job matrix data. If the container.image property is set to null, GitHub
Actions will run the job directly in the runner environment instead of in a container. This allows us to produce the
builds using either a container or a bare runner machine as is appropriate for each target.
Unfortunately the latest v4 version of the actions/checkout action used to checkout the repository into the job
environment has a dependency on a higher version of glibc than is provided by the Linux container. For this reason, the
workflow is configured to use actions/checkout@v3 for the Linux build job. We will likely receive pull requests from
Dependabot offering to update this outdated action dependency for the v4 and at each subsequent major version release of
the action (which are not terribly frequent). We must decline the bump of the action in that specific step, but accept
the bumps of all other usages of the action in the workflows. Dependabot remembers when you decline a bump so this
should not be too bothersome.
- fix(test): integration tests are more resilient.
- run the Create integration tests with other slow tests,
- queued `PUT`/`DELETE` requests to make the test timeout happy,
- reduced the `/sketches/search` offset to 1/5th and
- remove Create API logging.
- fix(linter): ignore `lib` folder. Remove obsolete `.node_modules`
pattern.
- feat(ci): enable Create API integration tests.
Signed-off-by: Akos Kitta <a.kitta@arduino.cc>
Co-authored-by: per1234 <accounts@perglass.com>
The "Arduino IDE" GitHub Actions workflow uploads the nightly and release builds to Amazon S3, from which they are
downloaded by the auto-update as well as directly by users via the links on the "Software" page of arduino.cc.
The workflow can also be useful in forks. Either by those who want to test contributions staged in their fork prior to
submitting a PR to the parent repo, or by those maintaining a hard fork of the project. Even though these forks wouldn't
(and couldn't due to lack of access to the encrypted credential secrets only available to the workflow when ran in a
trusted context in Arduino's repo)credentials stored in Arduino's repo) use the S3 upload component of the workflow,
they may still find it valuable for continuous integration as well as continuous deployment via the tester builds and
release builds the workflow also publishes to the GitHub repository it runs in. For this reason, the workflow contains
code to determine whether it should attempt the S3 uploads.
Previously the repository name was used as the criteria in that code. The project specificity of that approach makes the
workflow less easily reusable. A more generally applicable criterion is whether the encrypted credential certificate is
defined.
The new criterion allows the workflow to be used in any repository where the administrator has created an encrypted
secret containing their AWS credentials. That might be other projects owned by Arduino, or even 3rd party projects where
the owners want to take a similar build publishing approach using their own AWS account.
The "Arduino IDE" GitHub Actions workflow uploads the nightly and release builds to Amazon S3, from which they are
downloaded by the auto-update as well as directly by users via the links on the "Software" page of arduino.cc.
The workflow can also be useful in forks. Either by those who want to test contributions staged in their fork prior to
submitting a PR to the parent repo, or by those maintaining a hard fork of the project. Even though these forks wouldn't
(and couldn't due to lack of access to the encrypted credential secrets only available to the workflow when ran in a
trusted context in Arduino's repo)credentials stored in Arduino's repo) use the S3 upload component of the workflow,
they may still find it valuable for continuous integration as well as continuous deployment via the tester builds and
release builds the workflow also publishes to the GitHub repository it runs in. For this reason, the workflow contains
code to determine whether it should attempt the S3 uploads. Previously that code was duplicated in both the nightly and
release publishing jobs of the workflow. Since the workflow already contains a job specifically for the purpose of
determining the characteristics of the build being performed and making that information available from single source
for use throughout the rest of the workflow, it makes sense to also move the S3 upload determination code to that job.
On every release tag, and manual trigger when the "Include builds on non-free runners" checkbox is checked, make the
Arduino IDE build for native Apple Silicon host in addition to the builds that are always generated by the "Arduino IDE"
GitHub Actions workflow.
Previously, the build workflow only produced a build for x86-64 (AKA "Intel") macOS hosts. Although it is possible to
use those builds on Apple Silicon machines via the Rosetta 2 translation software, the performance is significantly
inferior to a native build so we must also provide Apple Silicon native builds.
Previously the Apple Silicon builds were produced manually. The reason for using that inefficient and error-prone
approach instead of the automated continuous deployment system used for other builds was that GitHub did not provide the
necessary Apple Silicon runner machines and Arduino was not capable of setting up such self-hosted machines in a manner
that would make them feasible for the project maintainers to use. GitHub hosted Apple Silicon runner machines are now
available so we can add the target to the build workflow.
GitHub gives unlimited use of the basic runner machines for workflow runs in public repositories. However, the macOS ARM
architecture is only provided in runner machines which are classified as "larger runner". Use of these runners is
charged on a per-minute basis, without any of the free allowances GitHub provides for the normal runners. In order to
avoid unnecessary expenditures, native Apple Silicon builds must be generated only when there is compelling reason to do
so. Such a build is needed for every release, so the workflow is configured to always generate the builds when triggered
by a tag. In addition to releases, Apple Silicon tester builds for pull requests that might have special implications
for this target. For this reason, the workflow is configured to allow Apple Silicon builds to be triggered manually by a
repository maintainer.
The workflow uses a job matrix to run the build for each target on the appropriate runner machine in parallel, using the
universally applicable workflow code for all jobs. It uses another job matrix to generate individual workflow artifacts
for the tester builds of each target. Previously it was possible to always use the same matrix configurations for all
workflow runs. With the addition of the selectively run macOS ARM job, it is now necessary to generate these matrixes on
the fly.
The electron-updater package used by Arduino IDE's auto-update capability uses a data file (known as the "channel update
info file") to check for the availability of updates. A single "channel update info file" is used for the data of the
macOS x86 and ARM builds. Since a separate job is used to produce each of those builds, this means the "channel update
info file" produced by each of the macOS build jobs must be merged into a single file.
The "Arduino IDE" GitHub Actions workflow is used to generate several distinct types of builds:
- Tester builds of commits
- Nightly builds
- Release builds
Different actions must be performed depending on which type of build is being produced. The workflow contains code that
uses various criteria to determine the build type.
Previously that code was duplicated in multiple places:
- The packaging job
- The changelog generation job
- The nightly build publishing job
- The release publishing job
This duplication is avoided by moving the code to a dedicated job that makes the build type information available to all
subsequent jobs via outputs.
The "Arduino IDE" GitHub Actions workflow uses a job matrix to make the builds for each host target in parallel.
The same steps are used for each job, but some configuration adjustments must be made on a per-target basis. This is
done through various attributes in the matrix configuration.
Previously the `os` attribute was used for two distinct things:
- The machine identifier of the GitHub Actions runner machine of the job.
- The differentiator in the human-targeted job name.
The attribute name "os" (for "operating system") was misleading because runners are differentiated by more than only the
operating system on the machine.
The use of a machine identifier as a differentiator in the human-targeted job name was a bad idea because these
identifiers would only effectively communicate the nature of a build to humans who are quite knowledgeable about the
GitHub Actions workflow syntax.
The impact of these poor decisions has not been too severe previously due to there only being a single job for each
operating system. However, there is a need for multiple jobs per operating system in order to support multiple host
architectures (e.g., macOS x86 and ARM).
The solution is to:
- Use an appropriate name for the runner identifier attribute.
- Use a dedicated attribute for the human friendly job name differentiator.
The project was recently switched to using a "trunk-based" development strategy. This necessitated some adjustments to
the configuration of the GitHub Actions workflows in order to ensure the CI system could be used to effectively validate
the project at the state staged for release in the release branch.
A `run-determination` job was added to the workflow. This job determines whether the conditions under which the workflow
was triggered indicate that the rest of the jobs in the workflow should be run.
A validation workflow should run fully under any of the following conditions:
- The trigger event was something other than a branch creation
- The trigger event was a release branch creation
Since the project is fully validated prior to a release, running the workflow when triggered by a release tag is
pointless (and even harmful in some specific standardized workflows not currently used in this repository), so (even if
for no other reason than efficiency) verification workflows are configured to not run under these conditions.
The standardized Arduino tooling workflows follow a modular design where each workflow has a narrow scope of purpose.
That path was not taken by those who set up the infrastructure for this repository. They instead created a single
massive monolithic "Arduino IDE" workflow that performs many unrelated operations under various conditions. This workflow generates
releases in addition to doing validation. Even though it is pointless to run the workflow's validation operations when
the workflow is triggered by a release tag, it is essential to run it for the release generation.
Previously, the code used in the "Arduino IDE" workflow's `run-determination` job was configured as appropriate for a
verification workflow. This meant that a release would not be generated on push of a release tag as intended. The bug is
fixed by adjusting the code to do a full run of the workflow when triggered by a release tag.
- update Theia to `1.39.0`,
- remove the application packager and fix the security vulnerabilities,
- bundle the backed application with `webpack`, and
- enhance the developer docs.
Co-authored-by: Akos Kitta <a.kitta@arduino.cc>
Co-authored-by: per1234 <accounts@perglass.com>
Signed-off-by: Akos Kitta <a.kitta@arduino.cc>
