Continuous integration

A GitHub Actions matrix for Windows, macOS and Linux

Run your tests on Windows, macOS and Linux on every commit with a GitHub Actions matrix: runner labels, shell differences, caching, artefacts and costs.

At a glance

ToolPlatformsLicenceBest for
GitHub Actions
  • Windows
  • macOS
  • Linux
Hosted service (free tier for public repositories) Hosted runners for all three systems, with a matrix strategy that fans one job out over them.
actionlint
  • Windows
  • macOS
  • Linux
MIT Static checker for workflow files: catches typos in expressions, matrix keys and shell scripts before you push.

The cheapest cross-platform test lab is one you do not maintain. GitHub Actions provides hosted Windows, macOS and Linux machines, and a matrix strategy runs the same job on each of them. Set up well, it catches the "works on my machine" class of bugs on every pull request.

The minimal matrix

.github/workflows/test.yml
name: Tests

on:
  push:
  pull_request:

jobs:
  test:
    name: Test on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-24.04, windows-2025, macos-15]
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - run: npm test

Two choices in this file are deliberate:

  • **fail-fast: false.** By default, the first failing leg cancels the others. For cross-platform testing that is the opposite of what you want: knowing that a bug happens on Windows *only* is half the diagnosis.
  • **Pinned labels instead of -latest.** ubuntu-latest, windows-latest and macos-latest move to a new OS version on GitHub's schedule. Pinning makes that upgrade a reviewed change in your repository rather than a surprise red build on a Monday.

Know what each runner is

Exact software lists live in the actions/runner-images repository, one file per image.
Label familyArchitectureThings to know
ubuntu-*x86-64 (and -arm variants)Fastest to start and cheapest; Docker available; passwordless sudo
windows-*x86-64PowerShell is the default shell; Visual Studio build tools preinstalled; slower file system
macos-*Apple silicon (arm64) for current labelsXcode preinstalled; the most expensive minutes; no Docker

Taming the shell differences

The default shell is bash on Linux and macOS and pwsh on Windows, so the same run: line can behave differently. The simplest fix is to choose one shell for the whole job — Git Bash is available on Windows runners:

defaults:
  run:
    shell: bash
  • Use ${{ runner.temp }} and ${{ github.workspace }} instead of hard-coded /tmp or C:\ paths.
  • Set git config --global core.autocrlf false before checkout if your tests compare files byte for byte — otherwise Windows checkouts get CRLF line endings.
  • Write portable test code: path joining, case-insensitive file systems on Windows and macOS, and \r\n in process output are the classic failures.

Platform-specific steps without duplicating the job

Use runner.os — Linux, Windows or macOS — to run a step on one system only, and include to attach extra values to a matrix leg:

strategy:
  fail-fast: false
  matrix:
    os: [ubuntu-24.04, windows-2025, macos-15]
    include:
      - os: ubuntu-24.04
        artifact: app-linux.tar.gz
      - os: windows-2025
        artifact: app-windows.zip
      - os: macos-15
        artifact: app-macos.zip
steps:
  - name: Install Linux GUI dependencies
    if: runner.os == 'Linux'
    run: sudo apt-get update && sudo apt-get install -y xvfb
  - name: Run UI tests
    run: ${{ runner.os == 'Linux' && 'xvfb-run -a ' || '' }}npm run test:ui

Linux runners have no display, so GUI and headed browser tests run under xvfb-run. Windows and macOS runners have a desktop session and need nothing extra.

Keep failures debuggable

A red macOS leg is only useful if you can see why. Upload test reports, screenshots and logs when a job fails, with the OS in the artefact name so the legs do not overwrite each other:

- name: Upload test report
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: test-report-${{ matrix.os }}
    path: test-results/
    retention-days: 14

Keep it fast and affordable

  • Cache dependencies with the built-in cache: option of the setup actions, keyed by the lock file.
  • Mind the multipliers. For private repositories, Windows and macOS minutes count more than Linux minutes against the included quota. Run the full matrix on pull requests and the main branch; a fast Linux-only job is enough for draft branches.
  • Cancel superseded runs with a concurrency group per branch, so that ten quick pushes do not queue ten macOS jobs.
  • Split slow suites with sharding inside each leg rather than adding more OS versions than your users run.
concurrency:
  group: tests-${{ github.ref }}
  cancel-in-progress: true

Lint the workflow itself

Workflow files are code that only runs remotely, which makes typos expensive. actionlint checks expression syntax, matrix references, runner labels and embedded shell scripts locally in a second:

actionlint .github/workflows/*.yml

Where hosted runners stop

Hosted runners give you one current version of each system. If you must support older Windows builds, specific Linux distributions or a GPU, add self-hosted runners or run virtual machines inside the job. For browser suites, combine this matrix with Playwright's three engines and you cover nine OS-and-engine combinations with one file.

  • github actions
  • ci
  • matrix
  • runners
  • devops