Arduino IDE offers an update to the user when a newer version is available. The availability of an update is determined
by comparing the user's IDE version against data file ("channel update info file") stored on Arduino's download server.
These "channel update info files" are automatically generated by the build workflow.
Previously the release process was fully automated, including the upload of the "channel update info files" to the
server.
As a temporary workaround for limitations of the GitHub Actions runner machines used to produce the automated builds,
some release builds are now produced manually:
- Linux build (because the Ubuntu 18.04 runner was shut down and newer runner versions produce builds incompatible with
older Linux versions)
- macOS Apple Silicon build (because GitHub hosted Apple Silicon runners are not available)
The automatic upload of the "channel update info files" produced by the build workflow is problematic because if users
receive update offers before the "channel update info files" are updated for the manually produced builds, they can
receive an update to a different build than intended:
- Users of older Linux versions would update to a build that won't start on their machine
- macOS Apple Silicon users would update to macOS x86 build that is less performant on their machine
For this reason, the build workflow is adjusted to no longer upload the Linux and macOS "channel update info files" to
the download server on release. These files will now be manually uploaded after they have been updated to provide the
manually produced builds.
This workaround will be reverted once a fully automated release system is regained.
- updated to `electron-builder@23.6.0`
- set `CSC_FOR_PULL_REQUEST` env to run notarization for a PR build.
Signed-off-by: Akos Kitta <a.kitta@arduino.cc>
The "Check Certificates" GitHub Actions workflow uses OpenSSL to check for problems with the project's signing
certificates.
Certificates exported to PKS#12 archive files using older tools may have been encrypted using the "RC2-40-CBC"
algorithm.
Due to the availability of more secure modern alternatives, default support for RC2-40-CBC encryption was dropped in
OpenSSL 3.x.
The macOS signing certificate uses this RC2-40-CBC encryption.
The "Check Certificates" GitHub Actions workflow runs on the `ubuntu-latest` runner. Previously, this runner used Ubuntu
20.04. This has now changed to Ubuntu 22.04. With the operating system update came an OpenSSL update from 1.1.1f to
3.0.2. This caused the workflow runs to fail on the macOS certificate job:
Error outputting keys and certificates
80FBB0C5087F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:../crypto/evp/evp_fetch.c:349:Global default library context, Algorithm (RC2-40-CBC : 0), Properties ()
Even though no longer done by default, OpenSSL still supports RC2-40-CBC encryption via its "legacy" provider. So
compatibility with the certificate is restored by adding the `-legacy` flag to the `openssl pkcs12` commands.
The "Arduino IDE" GitHub Actions workflow generates a changelog from the commits since the last tag.
This changelog is published in multiple ways:
- Printed to workflow run logs
- Uploaded to Arduino's download server (mostly useful for the nightly builds)
- Initial version of release notes
For the last, the changelog text must be passed from the dedicated changelog generation workflow step to the release
step. This is done via workflow job output.
At the time the system was set up, outputs for workflow `run` steps were set using the `set-output` workflow command.
That "workflow command" system was later determined by GitHub to have potential security vulnerabilities, so it was
replaced with a `GITHUB_OUTPUT` environment file.
The "Arduino IDE" workflow was migrated to the new "environment file" approach. It was later discovered that there was
an undocumented breaking change in the method for handling multi-line strings in workflow step outputs between the old
"workflow command" system and the new "environment file". This resulted in the initial release notes having an incorrect
format. For example, what would previously have been formatted like this:
- Updated translation files (#1606) [23c7f5f]
- Use 0.29.0 CLI in IDE2 (#1683) [f1144ef]
Was now formatted like this:
- Updated translation files (#1606) [23c7f5f]%0A - Use 0.29.0 CLI in IDE2 (#1683) [f1144ef]%0A
The solution is to remove the commands that did the escaping of the changelog text in a manner that is no longer
supported and replace them with a "here document"-style format.
A random number is used as the "delimiter" (limit string) per the security recommendations in the official GitHub
documentation. Note that even though the multiline strings handling documentation was placed under the environment
variable section, it also applies to setting outputs.
GitHub Actions provides the capability for workflow authors to use the capabilities of the GitHub Actions ToolKit
package directly in the `run` keys of workflows via "workflow commands". One such command is `set-output`, which allows
data to be passed out of a workflow step as an output.
It has been determined that this command has potential to be a security risk in some applications. For this reason,
GitHub has deprecated the command and a warning of this is shown in the workflow run summary page of any workflow using
it:
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more
information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
The identical capability is now provided in a safer form via the GitHub Actions "environment files" system. Migrating
the use of the deprecated workflow commands to use the `GITHUB_OUTPUT` environment file instead fixes any potential
vulnerabilities in the workflows, resolves the warnings, and avoids the eventual complete breakage of the workflows that
would result from GitHub's planned removal of the `set-output` workflow command 2023-05-31.
The "github-stats" GitHub Actions workflow periodically gathers GitHub release asset download statistics for Arduino CLI
and pushes the results to Datadog.
There are no known problems with this workflow. However, the companion "arduino-stats" workflow that did the same for
the downloads of Arduino IDE from downloads.arduino.cc was broken and thus removed from the repository.
The GitHub stats are not very valuable on their own as they only provide an unknown fraction of the total downloads of
Arduino IDE. They have also not ended up being used.
The workflow also uses deprecated Node.js 12 runtime, which currently results in warnings printed to the workflow run
summary page, but will eventually cause the complete breakage of the workflow.
Since it doesn't provide any value and represents a maintenance burden, the workflow is hereby removed from the
repository.
The "arduino-stats" GitHub Actions workflow was designed to periodically gather download statistics from Arduino CDN and
push results to Datadog.
The recorded stats from the identical system in the Arduino CLI repository showed a periodic decrease in total download
count. Since this is patently impossible, it is clear that something is wrong with the system and that the recorded data
is not trustworthy. An investigation into the problem
was never done.
On 2022-03-14, the runs of the "arduino-stats" GitHub Actions workflow began to fail. Because there had not been any
relevant change in the repository between the last successful run and the first failing run, it seems that some external
change caused the breakage.
The workflow also uses deprecated Node.js 12 runtime-based actions and set-output workflow command, which currently
results in warnings printed to the workflow run summary page, but will eventually cause the complete breakage of the
workflow.
Since the workflow was not ever working successfully and the lack of an investigation about that indicates that the
stats are not of immediate importance, the best course of action is to simply remove the broken infrastructure from the
repository rather than investing time into fixing something that isn't being used anyway.
Arduino CLI is a tool dependency of Arduino IDE. For this reason, the necessary Arduino CLI build is acquired whenever
running the `yarn` command in the repository.
The way the Arduino CLI build is acquired depends on the type of version specified as dependency in the
`arduino.cli.version` field of the arduino-ide-extension package metadata:
- Release/nightly: download pre-built standard distribution
- Git ref: build from source
This means that, in the latter case, all build dependencies of Arduino CLI must be present. While the Go module
dependencies are automatically installed during the build, the build tool dependencies must be installed in advance:
- Go programming language
- Task task runner
Arduino IDE's infrastructure was recently changed to use the Task tool to build Arduino CLI in the supported manner. A
step to install Task was not added to some of the workflows that run `yarn`, which caused them to fail when a
non-release version of Arduino CLI was used as a dependency:
arduino-ide-extension: >>> Building the CLI...
arduino-ide-extension: /bin/sh: 1: task: not found
arduino-ide-extension: error Command failed with exit code 1.
A step for the missing tool dependency is hereby added to those workflows.
The lack of an explicit installation of the other dependency, Go did not result in an error because Go is pre-installed
on the GitHub Actions runner. However, the installed version may not match the version Arduino CLI is intended to be
built with and validated for, and the version provided by the runner may change at any time. For this reason, it will be
safest to explicitly set up the appropriate version of Go in the workflows.
The "Arduino IDE" workflow performs the following operations when triggered on push and pull request events:
- Build application
- Lint code
- Run tests
- Produce tester packages
All of these operations are specific to the TypeScript/JavaScript code base and its infrastructure.
Previously, the workflow ran whenever any file in the repository was changed. This includes files that have no
relevance, meaning the operations performed by the workflow were pointless. In addition to general inefficiency, these
lengthy and sometimes spuriously failing unnecessary workflow runs might cause delay or confusion to both the
contributors and maintainers for what would otherwise be a simple process.
GitHub Actions provides the ability to configure path filters for the workflow triggers. The workflow will only run on
events that change files satisfying these path filters. This is "AND"ed with the `branches` filters, meaning the existing
restrictions on which branches produce a run remain unchanged. The `tags` filter is independent from the `paths` and
`branches` filters, meaning the added path filters don't make any change to which tag push events will trigger the
workflow.
GitHub Actions workflows may require access to privileged information in order to perform certain operations. GitHub
provides the capability for doing this via "repository secrets".
For security reasons, repository secrets are only accessible to a GitHub Actions workflow run when it is triggered by an
event from within the repository containing the secret. This means that a workflow which requires such secrets would
fail when run in a fork (unless the fork owner was able to set up their own secrets with suitable values).
In order to make the relevant components of the CI system friendly for use in forks by contributors validating their
work in preparation for submitting a PR, when the operations that require access to a secret are supplemental, those
operations should be configured to only run from branches of the parent repository.
Due to its unfortunate monolithic design, in addition to operations useful to contributors, the "Arduino IDE" workflow
contains several such supplemental operations:
- Code signing
- Publishing release artifacts to Arduino's server
Some attempt was previously made to configure the workflow to skip these operations when run in forks, but that
configuration was not done correctly. This made the workflow only usable by contributors with a deep enough
understanding of GitHub Actions to be able to make the necessary modifications provisionally every time they needed to
use the workflow.
The average contributor would not be capable or willing to do this, which might result in PRs being
submitted in a less validated state, increasing the burden on maintainers.
The specific misconfigurations:
**`build` job was conditional on the workflow running from `arduino/arduino-ide`**
The job itself can run just fine in a fork, so there is no reason to impose this restriction.
Since the time this conditional was added, some changes have been made to the GitHub Actions system which makes this
sort of configuration unnecessary:
- GitHub Actions is globally disabled in forks by default
- Workflows which contain a `schedule` trigger (as is the case with this one) are individually disabled by default,
requiring the repository owner to enable it specifically even after enabling GitHub Actions in general.
This means this workflow will never run unexpectedly in a fork. The fork owner will always have intentionally enabled it.
So this conditional can be removed completely.
**Code signing was conditional on PR being submitted from a branch of the base repo**
This would cause a spurious failure of the signing operation on PRs made within the contributor's fork when the signing
secrets were not defined.
The more appropriate condition of whether the signing secrets are defined or not is now used. The environment variable
name has been updated accordingly.
**`release` job was conditional on running from `arduino/arduino-ide`**
The GitHub release creation step of this job can run in any repository. It is only the step that uploads to Arduino's
AWS server which would only make sense to run from `arduino/arduino-ide`.
So the conditional is moved to the AWS upload step, allowing contributors to test the workflow's release operation in
their forks to validate related proposals.
Previously, there was some code duplication of the complex code signing certificate handling commands, which made the
related code more difficult to understand, maintain, and develop.
The cause of this duplication is that there is a separate certificate for each operating system, each of which is stored
in separate repository secrets, as well as a different certificate file extension for each OS. Since the secret names
and file extensions are associated with the operating system, it is most logical to define them via attributes alongside
the operating system definition in the job matrix configuration already used to generate the parallel job runs for
native build on each OS.
That done, the certificate handling commands are universal and the system can easily expand to additional host targets
(e.g., Apple M1) as time goes on.
Contributors may submit pull requests against development branches in the repository for either of the following valid
reasons:
- Propose changes to a previous proposal, either while it is still in development, or else in the case where the changes
are more complex/extensive than can be efficiently proposed via the PR review framework.
- The proposal is dependent on work from an unmerged PR.
Previously, the "Arduino IDE" GitHub Actions workflow was unnecessarily configured to only run for PRs based on the
`main` branch. This meant that validation and tester builds were not provided for the PRs based on other branches.
The "Compose full changelog" GitHub Actions workflow generates a changelog file from the release notes and uploads this
to Arduino's server for display to the user by the IDE updater.
Previously, this workflow could be triggered by either of two events:
- Release creation
- Release edit
To reduce the possibility of endless recursion, GitHub Actions ignores events which are triggered using the
auto-generated `GITHUB_TOKEN` access token. All release creations are done automatically by the "Arduino IDE" GitHub
Actions workflow, which uses this token.
For this reason, the release creation trigger will never be used. Since the behavior of the event being ignored by
GitHub Actions under these conditions is not at all obvious, having the workflow configured for such an irrelevant
trigger can cause confusion.
The workflow will be triggered by the manual edit which is done on every release to format the raw release notes
auto-generated from the commit history. So the fact that the release creation trigger doesn't work is not a problem.
* IDE updater assorted bugfix
- add linux AppImage target
- fix hardcoded if condition that causes to always show the update dialog
- fix redundant test build version
- recalculate sha512 after notarization on macOS
* boost notarization speed
* recalculate artifacts hash
* Remove check for updates on startup setting
* Remove useless exported function
* Update template-package.json used to package IDE
* Add function to get channel file during packaging step
* Add updates check
* move ide updater on backend
* configure updater options
* add auto update preferences
* TMP check updates on start and download
* index on check-update-startup: fcb8f6e TMP check updates on start and download
* set version to skip on local storage
* add IDE setting to toggle update check on start-up
* comment out check for updates on startup and auto update settings
* Update Theia to 1.22.1
* updated CI
* download changelog and show it in IDE updater dialog
* remove useless file
* remove useless code
* add i18n to updater dialog
* fix i18n
* refactor UpdateInfo typing
* add macos zip to artifacts
* Simply use `--ignore-engines`
* Use correct --ignore-engines
* Fix semver#valid call
* Use C++17
* updated documentation
* add update channel preference
* update updater url
* updated documentation
* Fix the C++ version
* Build flag for cpp
* add disclaimer with correct node version
* Update `electron-builder`
* Fix `Electron.Menu` issue
* Skip electron rebuild
* Rebuild native dependencies beforehand
* Use resolutions section
* Update template-package.json as well
* move ide-updater to electron application
* refactor ide-updater service
* update yarn.lock
* update i18n
* Revert "Add gRPC user agent (#834)"
This reverts commit 5ab3a747a6e8bf551b5c0e59a98154d387e0a200.
* fix ide download url
* update latest file in CI
* fix i18n check
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
Co-authored-by: Francesco Stasi <f.stasi@me.com>
Co-authored-by: Mark Sujew <msujew@yahoo.de>
The `carlosperate/download-file-action` action is used in the GitHub Actions workflows as a convenient way to download
external resources.
A major version ref has been added to that repository. It will always point to the latest release of the "1" major
version series. This means it is no longer necessary to do a full pin of the action version in use as before.
Use of the major version ref will cause the workflow to use a stable version of the action, while also benefiting from
ongoing development to the action up until such time as a new major release of an action is made. At that time we would
need to evaluate whether any changes to the workflow are required by the breaking change that triggered the major
release before manually updating the major ref (e.g., uses: `carlosperate/download-file-action@v2`). I think this
approach strikes the right balance between stability and maintainability for these workflows.
