For some strange reason, Browserify will hardcode absolute paths from
the machine that generated the bundle to be able to resolve `__dirname`
and `__filename` calls. This makes no sense, given that it means that
the Browserify bundle will not work when we move it to another machine,
which went undetected probably for months.
The Browserify community apparently makes modules to fix this particular
issue (like `bundle-collapser`, and `intreq`), however none of this seem
to solve the problem for the Etcher CLI bundle.
I also gave https://github.com/zeit/pkg a go, however I gave up after
not being able to make use of native modules (nothing seems to work; the
packager result will simply not find the addons).
Finally, I ended up making the following workarounds:
- Edit the Browserify bundle file to use its own `__dirname` to
dynamically resolve the values of `__dirname` and `__filename` of the
files it contains
- Patch `lzma-native` to statically require its add-on rather than
relying on dynamic requires from `node-pre-gyp`, which makes it
impossible to resolve on the final bundle
See: https://github.com/resin-io/etcher/issues/355
See: http://stackoverflow.com/questions/21993073/browserify-with-paths-to-folders-in-my-system
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
UglifyJS v3 was recently released, which includes support for
compressing ES6 code.
This commit introduces the following changes:
- Add a boolean `-m` option to `concatenate-javascript.sh`, which when
set, will minify the final concatenated JavaScript file
- Install `uglify-es` in the CI servers
See: https://news.ycombinator.com/item?id=14285179
See: https://github.com/mishoo/UglifyJS2/tree/harmony
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
We forgot to do so on the PR that introduces this script.
This commit also moves all the CI code to separate scripts.
See: https://github.com/resin-io/etcher/pull/1371
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
Sentry error reports showcase that elevation errors on Windows are one
of the most frequent Windows errors.
In order to perform Windows elevation, we ship compiled EXEs of a third
party CLI elevation application (http://code.kliu.org/misc/elevate/)
that has several limitations:
- We have the scan the output of the script to determine if a user
cancelled the elevation request, which causes all sorts of issues on
computers where English is not the main language
- The application displays a `cmd.exe` window for some milliseconds,
which is bad UX, that we have to workaround by distributing a patched
version of the tool
- The CLI application has to be spawned, which seems to be problematic
if users have anti-virus software, leading to hard to debug issues
- We don't have any control if something goes wrong
For these reasons, we decided to implement our own elevation mechanism
in C++ as a Node.js add-on, based on the `elevate.exe` code we where
previously using.
Misc changes:
- Introduce a `lib/shared/bindings.js` module to easily require local native
add-ons
- Install `cpplint` and configure it to lint C++ files
Note that for practical reasons, the C++ code lives in this repository
rather than in a separate module. We will release this functionality in
a more accessible way in the future as part of the Etcher SDK project.
Change-Type: patch
Changelog-Entry: Fix uncaught errors when cancelling elevation requests on Windows when the system's language is not English.
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
The `npm ls` command will exit with an error code if there are invalid
or extraneous dependencies.
This commits adds that command to `make sanity-checks`, so we can catch
those cases early one.
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
We recently discovered that users not running latest GNU/Linux
distributions were having problems when Etcher tried to load native
modules. We used to build packages on Ubuntu 16.04, which ships with
GLIBC 2.23.
In order to solve that issue, we moved our build Docker containers to
Debian Jessie, which ships with GLIBC 2.19, however we noticed that
users running even older Linux distributions were still having the same
issue, so this commit switched the Docker containers to Ubuntu 12.04,
which ships with GLIBC 2.15.
See: https://github.com/resin-io/etcher/pull/1319
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
Newer npm versions (4.4+ I believe) even remove dependencies
not shrinkwrapped yet while pruning causing newly installed dependencies
to not always be shrinkwrapped / updated in the shrinkwrap.
Removing the prune allows for this to work properly again and in the future,
with the drawback that care must be taken to not have extraneous dependencies
in the module tree.
Change-Type: patch
This commit packages the Etcher CLI as a ZIP archive for Windows, and as
a tarball for GNU/Linux and macOS. These basic "installers" are the
foundation to provide Chocolatey/Brew/AUR/etc packages.
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
To prevent any potential version incompatibility caused by simply installing
the latest version of the packages in the CI services.
See: https://github.com/resin-io/etcher/pull/1326
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
We recently hit an issue where Etcher builds produced on Ubuntu relied
on a too new glibc version, making them incompatible with GNU/Linux
distributions such as Debian Jessie.
As a solution, we will start producing builds on Debian Jessie, which
ensures that the builds will be compatible with the majority of
GNU/Linux versions out there.
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
We currently require this module to be available in the environment in
order to be able to build debian packages.
This commit adds it as an optional dependency, so we can have control
over the version that we use, to prevent any potential incompatibility.
We also update the `udif` package.json version to match what its on the
shrinkwrap file to avoid npm complaining about version mismatches.
See: https://github.com/resin-io/etcher/pull/1290
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
The rc2 debian package is not working because the path where the package files
is installed has changed. The change in paths comes from an upstream change
in the electron-installer-debian node package. So, updating the path here to
match the actuall install location.
`jq` crashes when setting a value on a relative file. We workaround the
issue by using `jq` in a pipe instead of passing the file to it as an
argument.
See: https://github.com/stedolan/jq/issues/1155
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
Previously dependencies weren't actually removed from `node_modules`,
this runs `npm rm` on the optional dependencies, effectively excluding
them, and their dependencies from the shrinkwrap file.
Also the script has been hooked to the `preshrinkwrap` hook,
to remove the need of having to run it manually.
Change-Type: patch
* chore: Add a require() checking script
This adds another sanity-check bash script to scan the Etcher source files
and check all required node modules are available.
* Workaround for jq crash on Windows
We're passing a shell conditional to `deploy.script`, however Travis CI
seems to get confused about this in the deploy section, causing
GNU/Linux to run `make publish-aws-s3` directly on the Travis CI build
instead of in the Docker container, which causes the deployment to
eventually fail because of missing dependencies.
According to Travis CI:
> `deploy.script` must be a scalar pointing to an executable file or
> command.
See https://travis-ci.org/resin-io/etcher/jobs/214723725 for a failure
example.
We also move the Appveyor deploy script to a separate file for symmetry
purposes.
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
We don't actually use any build context, as we just mount the local
directory as a volume anyway. Omitting it speeds up the container
start-up time. This is an enhanced version of #1173
As removing the `node_modules` completely and then re-installing
is quite time consuming and SSD-thrashing, resort to pruning
and rebuilding native add-ons when switching targets.
Change-Type: patch
`pip install awscli` outputs the following warning:
```
Running setup.py install for PyYAML
checking if libyaml is compilable
i686-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/check_libyaml.c -o build/temp.linux-x86_64-2.7/check_libyaml.o
build/temp.linux-x86_64-2.7/check_libyaml.c:2:18: fatal error: yaml.h: No such file or directory
#include <yaml.h>
^
compilation terminated.
libyaml is not found or a compiler error: forcing --without-libyaml
(if libyaml is installed correctly, you may need to
specify the option --include-dirs or uncomment and
modify the parameter include_dirs in setup.cfg)
```
The installation gracefully continues anyway (the `--without-libyaml` is
automatically forced).
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
This commit introduces the phony `package-cli` target, used to package
the Electron CLI in a directory.
Other related changes:
- The `package` target has been renamed to `package-electron`
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
This commit makes Appveyor and Travis CI publish snapshot builds to S3
when a pull request is merged, by making use of the `publish-aws-s3`
Makefile target.
The changes required for such type of deployment are the followings:
- Set `S3_BUCKET` to `resin-nightly-downloads` when doing snapshot
builds
- Add deploy sections to `.travis.yml` and `appveyor.yml` that run `make
publish-aws-s3`
- Don't change `PRODUCT_NAME` when doing snapshot builds (given we'll be
publishing to a different S3 bucket)
- Install `awscli` in Appveyor CI and Travis CI
- Make GNU/Linux Docker containers inherit `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY` from the environment (so `awscli` is
configured correctly inside them)
- Add a prefix option to `aws-s3.sh` publish script to prepend a string
to the S3 path, so we can add a timestamp to more easily distinguish
files inside the `resin-nightly-downloads` bucket
- Print the published URL from `aws-s3.sh` for convenience purposes, so
we can click it when skimming through CI builds logs
- Add the `-R` and `-L` options when recursively copying `node_modules`
during a snapshot build to prevent weird Appveyor errors related to
hard links. The options listed before make sure that we recursively
resolve every link while copying
- Move from `wget` to `curl` to avoid certificate check failures
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
upx allows to reduce the package size by some mega bytes, but also
considerably slows down the application start time.
This commit ditches upx, since we can re-gain those benefits (and even
increase them) by other means (e.g: minifying JavaScript).
See: https://github.com/resin-io/etcher/issues/1138
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
* chore: fix `lzma-native` build issues on Windows
We've been recently hitting a weird `lzma-native` build error on Windows
(both locally and on Appveyor CI):
```
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
build
The input line is too long.
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(171,5): error MSB6006: "cmd.exe" exited with code 1. [C:\projects\etcher\node_modules\lzma-native\build\liblzma.vcxproj]
```
After a lot of experimentation, we realised the issue was gone if we
removed `node-sass` from the development dependencies.
The issue is that `node-gyp` was recently upgraded to v3.6.0, which was
picked up by `node-sass`, which declares `node-gyp` as a dependency. For
some reason, if `node-sass` causes `node-gyp` to be updated, then
`lzma-native` fails with the above cryptic error.
I was able to trace down the error to the following `node-gyp` commit:
ae141e1906
As a solution, this commit starts to shrinkwrap development
dependencies, and locks `node-gyp` to v3.5.0 until the issue is fixed.
Fixes: https://github.com/addaleax/lzma-native/issues/30
See: https://github.com/nodejs/node-gyp/issues/1151
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
* chore: ensure some modules in npm-shrinkwrap stay at specific versions
* Address code review comments
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
* chore: get the CI tests working again
* an updated eslint-plugin-lodash was creating extra linter errors where we
were using built-in String methods instead of the lodash equivalents
* an updated codespell package now installs with a different executable-name
(no .py extension), and now supports multiple dictionaries
* while I was at it I replaced the messy double-quoting-string logic with the
command-join module
Changelog-type: patch
* chore: downgrade codespell to 1.9.2 because the latest version has issues running on Windows
(we'll hopefully be able to revert this commit when codespell works again properly!)
We've recently had an incident were the `angular-mocks` version we were
running was incompatible with the `angular` version.
Each AngularJS core module is versioned equally, therefore we can write
a small script that automatically tests that they match, so we're
guarded against forgetting to similarly upgrade different dependencies.
See: https://github.com/resin-io/etcher/pull/1168
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
This is the architecture of the Raspberry Pi 3. The following changes
were necessary:
- Detect the architecture using `uname -m`
- Set the Debian equivalent architecture to `armhf`
- Handle the new architecture correctly on `dependencies-npm.sh` and
`electron-download-package.sh`, referring to it as simply `arm` (which
is what `node-gyp` expects)
This PR also includes `architecture-convert.sh`, which is a script to
find an architecture equivalence for Node and Debian.
Steps to test this PR:
- `make electron-installer-debian` from a Raspberry Pi 3
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
https://github.com/brigade/scss-lint now says "NOTICE: Consider other tools
before adopting SCSS-Lint", and recommends sass-lint as an alternative.
sass-lint has the additional advantage of being written in NodeJS, which means
it slots nicely into our existing infrastructure, and we can completely
remove our build-time dependency on Ruby.
There are a lot of new rules since the last time I revised the ESLint
rules documentation.
I've updated the main `.eslintrc.yml` to include some newer additions,
plus I added another ESLint configuration file inside `tests`, so we can
add some stricted rules to the production code while relaxing them for
the test suite (due to the fact that Mocha is not very ES6 friendly and
Angular tests require a bit of dark magic to setup).
This is a summary of the most important changes:
- Disallow "magic numbers"
These should now be extracted to constants, which forces us to think of
a good name for them, and thus make the code more self-documenting (I
had to Google up the meaning of some existing magic numbers, so I guess
this will be great for readability purposes).
- Require consistent `return` statements
Some functions relied on JavaScript relaxed casting mechanism to work,
which now have explicit return values. This flag also helped me detect
some promises that were not being returned, and therefore risked not
being caught by the exception handlers in case of errors.
- Disallow redefining function arguments
Immutability makes functions easier to reason about.
- Enforce JavaScript string templates instead of string concatenation
We were heavily mixing boths across the codebase.
There are some extra rules that I tweaked, however most of codebase
changes in this commit are related to the rules mentioned above.
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
Not including the colon means that the option parsing mechanism is not
expecting an argument for the `-t` and the `-p` option (for
`bintray-debian.sh` and `aws-s3.sh` respectively), and thus `$OPTARG`
will be undefined.
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
- Add missing backslash that caused the command to not be interpreted
completely
- Update `electron-sign-exe.exe` to `electron-sign-exe-win32.exe` in
Makefile
- Use hyphen options instead of Windows slash style, which confuses bash
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
I edited these files in Windows, which seems to have caused the
permissions bits to get lost.
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
We recently encountered a UI regression caused by forgetting to commit
the generated CSS files.
The solution is to make the CI servers run `npm run sass` and check if
there are unstaged files afterwards. If so, the tests halt.
In order to avoid duplication between Travis CI and Appveyor CI, this
logic has been encoded into a bash script at `scripts/ci`.
See: https://github.com/resin-io/etcher/pull/1120
Signed-off-by: Juan Cruz Viotti <jviotti@openmailbox.org>
