At a glance
| Tool | Platforms | Licence | Best for |
|---|---|---|---|
| GitHub Actions |
|
Hosted service (free tier for public repositories) | Hosted runners for all three systems, with a matrix strategy that fans one job out over them. |
| actionlint |
|
MIT | Static checker for workflow files: catches typos in expressions, matrix keys and shell scripts before you push. |
The cheapest cross-platform test lab is one you do not maintain. GitHub Actions provides hosted Windows, macOS and Linux machines, and a matrix strategy runs the same job on each of them. Set up well, it catches the "works on my machine" class of bugs on every pull request.
The minimal matrix
name: Tests
on:
push:
pull_request:
jobs:
test:
name: Test on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04, windows-2025, macos-15]
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: 24
cache: npm
- run: npm ci
- run: npm testTwo choices in this file are deliberate:
- **
fail-fast: false.** By default, the first failing leg cancels the others. For cross-platform testing that is the opposite of what you want: knowing that a bug happens on Windows *only* is half the diagnosis. - **Pinned labels instead of
-latest.**ubuntu-latest,windows-latestandmacos-latestmove to a new OS version on GitHub's schedule. Pinning makes that upgrade a reviewed change in your repository rather than a surprise red build on a Monday.
Know what each runner is
| Label family | Architecture | Things to know |
|---|---|---|
ubuntu-* | x86-64 (and -arm variants) | Fastest to start and cheapest; Docker available; passwordless sudo |
windows-* | x86-64 | PowerShell is the default shell; Visual Studio build tools preinstalled; slower file system |
macos-* | Apple silicon (arm64) for current labels | Xcode preinstalled; the most expensive minutes; no Docker |
Taming the shell differences
The default shell is bash on Linux and macOS and pwsh on Windows, so the same run: line can behave differently. The simplest fix is to choose one shell for the whole job — Git Bash is available on Windows runners:
defaults:
run:
shell: bash- Use
${{ runner.temp }}and${{ github.workspace }}instead of hard-coded/tmporC:\paths. - Set
git config --global core.autocrlf falsebefore checkout if your tests compare files byte for byte — otherwise Windows checkouts get CRLF line endings. - Write portable test code: path joining, case-insensitive file systems on Windows and macOS, and
\r\nin process output are the classic failures.
Platform-specific steps without duplicating the job
Use runner.os — Linux, Windows or macOS — to run a step on one system only, and include to attach extra values to a matrix leg:
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04, windows-2025, macos-15]
include:
- os: ubuntu-24.04
artifact: app-linux.tar.gz
- os: windows-2025
artifact: app-windows.zip
- os: macos-15
artifact: app-macos.zip
steps:
- name: Install Linux GUI dependencies
if: runner.os == 'Linux'
run: sudo apt-get update && sudo apt-get install -y xvfb
- name: Run UI tests
run: ${{ runner.os == 'Linux' && 'xvfb-run -a ' || '' }}npm run test:uiLinux runners have no display, so GUI and headed browser tests run under xvfb-run. Windows and macOS runners have a desktop session and need nothing extra.
Keep failures debuggable
A red macOS leg is only useful if you can see why. Upload test reports, screenshots and logs when a job fails, with the OS in the artefact name so the legs do not overwrite each other:
- name: Upload test report
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-report-${{ matrix.os }}
path: test-results/
retention-days: 14Keep it fast and affordable
- Cache dependencies with the built-in
cache:option of the setup actions, keyed by the lock file. - Mind the multipliers. For private repositories, Windows and macOS minutes count more than Linux minutes against the included quota. Run the full matrix on pull requests and the main branch; a fast Linux-only job is enough for draft branches.
- Cancel superseded runs with a
concurrencygroup per branch, so that ten quick pushes do not queue ten macOS jobs. - Split slow suites with sharding inside each leg rather than adding more OS versions than your users run.
concurrency:
group: tests-${{ github.ref }}
cancel-in-progress: trueLint the workflow itself
Workflow files are code that only runs remotely, which makes typos expensive. actionlint checks expression syntax, matrix references, runner labels and embedded shell scripts locally in a second:
actionlint .github/workflows/*.ymlWhere hosted runners stop
Hosted runners give you one current version of each system. If you must support older Windows builds, specific Linux distributions or a GPU, add self-hosted runners or run virtual machines inside the job. For browser suites, combine this matrix with Playwright's three engines and you cover nine OS-and-engine combinations with one file.