Test runners

Six cubes, one per supported test runner, in a row above one line that ends in a check mark

vci works with six test runners: Vitest, pytest, Go, Cargo, Rails (Minitest) and RSpec. This page shows, for each one, how to set it up, what vci records, what it refuses, and what to pin so a test passed on your laptop is skipped on a Linux CI runner.

Each runner has an adapter: the part of vci that runs your tests and records what they read. The adapter decides what a unit is: the smallest piece of your suite that vci can skip. The diagram shows the unit for each runner.

The unit that can be skipped, per runner: a test file for Vitest and pytest, a package for Go, a test target for Cargo, a Minitest file for Rails and a spec file for RSpec. Each unit runs in its own process, and one repository can mix runners.

What every adapter shares. Each unit runs in its own process, so nothing leaks in from another unit. While it runs, a collector (code vci loads into the test process) records every file, package and environment variable the test actually used. (Rust has no such hook, so the cargo adapter uses a conservative rule instead: see the table below.) When the unit passes, vci signs an attestation: a record that says "this unit passed with exactly these inputs". If the collector cannot see something the test did, or an input lies outside the repository, vci refuses to attest. The unit then simply runs in CI, as it would without vci.

Adapter	Unit	How inputs are found
`vitest`	test file	observed at run time (module graph, file reads, env reads)
`pytest`	test file	observed at run time (Python audit hooks, import system)
`go`	package	observed at run time (a logger overlaid into the standard library) plus `go list`
`cargo`	test target	conservative: every file of the package and its in-repo dependencies
`rails`	Minitest file, or RSpec spec file	observed at run time (a collector wrapping Ruby's file, env and process entry points)

The rules for every adapter.

A unit is attested only when it passed, nothing it did was invisible to the collector, and every input is inside the repository.
A unit with a skipped test is not attested. Rust's #[ignore] is the exception.
An attestation vouches for the unit run on its own, in its own process.

The commands in every set-up block are the same five steps:

Command	What it does
`vci init`	writes `vci.toml` (settings), `.vci/allowed_signers` (your public key) and, when the repository has an origin remote, `.git-meta` (where attestations are published)
`git commit`	commits them. CI reads keys and policy from the base branch (the branch a pull request merges into), so merge this commit to `main` first.
`vci run`	runs tests, records inputs, signs an attestation for each unit that passed
`vci plan`	lists which units would be skipped (SKIP) and which would run (RUN)
`vci push`	publishes the attestations to `refs/meta/main`

Copy-and-paste setups for each runner, with real output, are on the Examples page.

Vitest

Needs Node 22.15+ and Vitest >=3.2 <6. The unit is a test file.

Set up

cd your-project
# or set VCI_JS_PLUGIN instead
npm install --save-dev /path/to/vci/js/vitest-plugin

vci init --key ~/.ssh/id_ed25519.pub --no-install
git add vci.toml .vci/allowed_signers .git-meta \
  && git commit -m "Enable vci"

vci run src/b.test.ts --key ~/.ssh/id_ed25519
vci plan --base-ref origin/main
vci push

What is recorded

every module in the Vite graph and every module the worker evaluated, including computed import();
every file or directory read or stat'ed, and every failed lookup, so creating a file that was probed as absent invalidates the attestation;
external packages as name@version;
env vars: the set and hash lists, the built-in hash ones (TZ, LANG, NODE_OPTIONS, VITE_*, ...), and anything the test read that is not in ignore.

Inputs of every test file

Besides what the test used, every test file also records these global inputs: package.json, lockfiles, .npmrc, the Vitest config, the tsconfig.json chain, vci.toml and the test's snapshot file.

What is refused

Work the collector cannot see: child processes, network, worker_threads, fetch.
Reading the whole environment: enumerating process.env.
Writing: snapshot writes.
Vitest modes: isolate: false, vm or browser pools, and typecheck mode.

Laptop to Linux runner

Pin the Node major version and package-lock.json.
The Vitest and Vite versions must match on both sides.

pytest

Needs uv and pytest 9. The project must be a uv project with a committed uv.lock and a pytest config file in the project directory. The unit is a test file, run in its own process.

Set up

cd your-project
# a uv-managed build that also exists for the CI platform
uv python install 3.14.4
# pin the full version
echo 3.14.4 > .python-version
# never pick up the system Python
export UV_PYTHON_PREFERENCE=only-managed
uv lock

vci init --adapter pytest --key ~/.ssh/id_ed25519.pub
git add vci.toml .vci/allowed_signers .git-meta uv.lock \
  .python-version && git commit -m "Enable vci"

vci run tests/test_b.py --key ~/.ssh/id_ed25519
vci plan --base-ref origin/main
vci push

What is recorded

every module imported from the repository, including conftest.py files, and every path an import could have resolved to (so a new module that would shadow one is noticed);
every file opened or stat'ed, failed lookups and directory listings;
installed distributions as name@version, checked against uv.lock;
env vars read through os.environ, and always TZ and the variables the interpreter and pytest read at startup (PYTHONHASHSEED, PYTEST_ADDOPTS, ...: the built-in hash list); other PYTHON* and PYTEST_* variables are removed unless listed.

Any change to uv.lock runs everything.

What is refused

Work the collector cannot see: subprocess and friends, sockets and HTTP clients, ctypes.dlopen, and SQLite databases inside the repository.
Runs that are not repeatable: xdist, reruns, and unseeded random ordering.
Writing inside the repository.
Reading the whole environment (dict(os.environ)).

Laptop to Linux runner

Use a uv-managed Python whose exact patch version uv also ships for Linux (check with uv python list --all-platforms). Homebrew's Python will not match.
Pin the same uv version locally and in astral-sh/setup-uv.
The full set of installed distributions must be equal. A dependency with a platform marker installs on one side only, so everything runs.
The SQLite and OpenSSL versions the interpreter bundles are compared too. To accept other builds of them, set libraries = "any" under [policy] (see macOS to Linux). The Python version and the distributions are still compared.
Delete .DS_Store and __pycache__ from directories your tests list. vci run warns about them.

Go

Needs a Go module and Go 1.26. The unit is a package: every directory with _test.go files. Test ids are package directories relative to the repository root.

Set up

cd your-module
vci init --adapter go --key ~/.ssh/id_ed25519.pub
git add vci.toml .vci/allowed_signers .git-meta \
  && git commit -m "Enable vci"

# default: every package with tests
vci run ./b --key ~/.ssh/id_ed25519
vci plan --base-ref origin/main
vci push

What is recorded

Compile-time inputs, from go list: every file of each in-repo package in the import closure, its directory listing, go:embed targets, and files excluded by build constraints.
Run-time inputs, from a logger overlaid into the standard library for this build only: every file opened or stat'ed, every failed lookup, every env var looked up. This includes package initialisation and TestMain.
External modules as path@version, verified against go.sum.

What is refused

starting a child process;
package net anywhere in the import closure (see the waiver below);
cgo and syscall in non-standard packages;
os.Environ;
a skipped test;
using the local time zone with TZ unset.

Network waiver. Many test helpers import net/http without using it (testify's assert does). Set go_allow_net = true under [policy] to state that these tests do no network I/O. The waiver is recorded in the attestation and only accepted while the base commit's policy still sets it. It is the same as ignore_taints = ["go:net"]; ignore_taints waives the other refusals too (see configuration).

Laptop to Linux runner

Pin the exact Go version in actions/setup-go.
Put set = { TZ = "UTC", CGO_ENABLED = "0" } in [env]: vci sets them on both sides.
A package with platform-specific files (b_linux.go, //go:build darwin) or code that refers to runtime.GOOS is only accepted on the same OS and architecture.
A package with floating-point code is only accepted on the same architecture.

Rust (Cargo)

Needs a Cargo workspace or package with a committed Cargo.lock. The unit is a cargo test target of a workspace member.

Set up

A unit's test id is its package directory, #, and the target:

Test id	What `cargo test` runs
`<package dir>#lib`	the library's unit tests
`<package dir>#doc`	the library's doctests
`<package dir>#bin:<name>`	a binary's unit tests
`<package dir>#test:<name>`	an integration test file `tests/<name>.rs`

cd your-workspace
vci init --adapter cargo --key ~/.ssh/id_ed25519.pub
git add vci.toml .vci/allowed_signers .git-meta Cargo.lock \
  && git commit -m "Enable vci"

# one unit
vci run 'crates/b#test:b' --key ~/.ssh/id_ed25519
# every unit of that package
vci run crates/b --key ~/.ssh/id_ed25519
vci plan --base-ref origin/main
vci push

What is recorded

A weaker guarantee than the other adapters. Rust has no hook for what a test reads at run time. Instead, every file in the package directory of the unit, and of every in-repo crate it builds, is an input.

A test that reads files elsewhere in the repository must declare them in vci.toml:

[[inputs]]
match = ["crates/b#test:*"]
extra = ["testdata/**", "shared/config.json"]

vci scans the source for undeclared paths and refuses the unit when it finds one, but a path built at run time is not seen.

What is refused

Decided by scanning the source text, not by watching the test run:

child processes (Command::new);
sockets and network crates;
FFI and libc;
env::vars();
a unit where no test ran;
the deprecated env mode "loose". The cargo adapter relies on the default policy, where every variable no [env] list names is removed from the test's environment on both sides. That includes RUST_BACKTRACE and RUST_LOG: use set = { RUST_BACKTRACE = "1" } for backtraces.

Laptop to Linux runner

rustc and cargo must be identical, including the LLVM version. Use rustup with rust-toolchain.toml on both sides. Homebrew's rustc can report a different LLVM and then never matches.
Every target-dependent cfg in the unit's code must evaluate the same on both platforms. cfg(unix) is fine between macOS and Linux; cfg(target_os = "linux") makes the unit run.
Code using std::env::consts::OS or platform math functions pins the unit to the attesting OS and architecture.

Rails

Needs Rails 8 and the project's Ruby with its bundle installed (tested with Ruby 3.4.9 and SQLite). The unit is a test file, run in its own process. Two runners are supported:

`runner`	Unit, and how vci runs it
`"minitest"`	a file `bin/rails test` runs by default; run as `bin/rails test <file>`
`"rspec"`	a spec file RSpec's own configuration lists; run as `rspec --options .rspec <file>`

Set runner in vci.toml, or leave it out and vci detects it from the lockfile and the directories present. This section covers Minitest and everything the two share; RSpec has its own section below.

Set up

cd your-app
# so the CI runner can install the same bundle
bundle lock --add-platform x86_64-linux
vci init --adapter rails --key ~/.ssh/id_ed25519.pub
git add vci.toml .vci/allowed_signers .git-meta Gemfile.lock \
  && git commit -m "Enable vci"

vci run test/models/b_test.rb --key ~/.ssh/id_ed25519   # TZ comes from vci.toml's [env] set
vci plan --base-ref origin/main
vci push

The collector is plain Ruby loaded through RUBYOPT; nothing is installed into your project. It sets these run conditions:

the database is the test engine's: the databases config/database.yml configures for the test environment are used as they are, and Rails sets them up as in a plain bin/rails test (maintain_test_schema!, fixtures, transactional tests). Their contents are not an input. Files of a project run one at a time, and vci ci runs the rest in one bin/rails test process;
one process: PARALLEL_WORKERS=1, no Spring, no Bootsnap caches;
a fixed Minitest seed (--seed 0).

What is recorded

every Ruby file loaded, and Zeitwerk's directory listings;
every file read or checked: fixtures, templates, config, the schema;
the gems used, by version, wherever the bundle is installed. With bundler-cache it is in vendor/bundle inside the repository, and nothing there is an input;
native libraries opened through FFI, such as Active Storage's libvips, by file name and hash;
every variable read through ENV.

Any change to Gemfile.lock runs everything.

What Rails reads at boot is not an input

Since 2.7.36, by default (rails_inputs = "loose"), the files Rails reads while it boots are left out of every test file's inputs: config/routes.rb and config/routes/**, config/initializers/**, config/environments/**, config/application.rb, config/boot.rb, config/environment.rb, config/puma.rb, config/spring.rb, config.ru, db/migrate/**, config/locales/**, and the autoload records, so a new file in app/models runs nothing again. Gemfile.lock, db/schema.rb, config/database.yml, test_helper.rb, test/support and every file a test reads after boot (models, services, fixtures, templates) stay inputs.

The trade-off: a change to a boot file no longer runs any test again on a pull request. The default branch's full run (no_skip_refs) runs every test against it. rails_inputs = "strict" records everything, as before; ignore_inputs = ["config/credentials/**"] adds globs to leave out. Both go next to adapter, or in a project's table. vci run says once what it left out, and vci explain shows the mode. See the reference.

What is refused

child processes and forks (except the bin/rails db:test:prepare that Active Record's own schema maintenance starts);
sockets, and a database client a test opens itself;
a SQLite file that is not one of the test environment's databases, ATTACH and SQLite extensions;
writes inside the repository outside log/, tmp/, storage/ and coverage/;
enumerating ENV;
loading or reading files of an installed gem that is not in the bundle (ruby:gem-outside-bundle, ruby:read-outside-bundle);
any failure, error or skip, and a test that made no assertion.

vci does not manage the database. The test database is the test engine's, as in a plain bin/rails test: vci never creates, prepares, purges or loads it, and Rails sets it up (maintain_test_schema!, fixtures, transactional tests). SQLite files, MySQL and PostgreSQL servers configured for the test environment all work with no policy. What the database holds is never an input, so rows left behind change no verdict. vci records the schema file of every test database (db/schema.rb, db/cache_schema.rb, ...), config/database.yml, the fixtures and the code, and the adapter names (sqlite3, mysql2, trilogy, postgresql) are compared; no server version is. It promises what CI promises: the test passed on a database the engine set up, not that it was independent of leftover data.

A database server needs only what a Rails CI job always needs: the same adapter as a service container, and bin/rails db:test:prepare before the tests. Keep SQLite files git-ignored (/storage/*, or /db/*.sqlite3*). Still refused: another SQLite file, ATTACH, extensions, a database client a test opens itself, and Active Record connecting to a server or database that is not the test environment's. rails_allow_db was removed in 1.4.0: it still parses, has no effect, and vci run and vci plan warn. The collector is now 0.5.0, so existing Rails and RSpec attestations run once more in CI.

Laptop to Linux runner

A generated Rails app needs a few changes before anything is skipped across machines:

Credentials. Active Record reads config/master.key at boot. That file is git-ignored, so CI never matches. Give the test environment its own committed credentials (bin/rails credentials:edit --environment test), remove the credentials, or list RAILS_MASTER_KEY in [env] hash and give it the same value on both sides.
CI. The generator's config.eager_load = ENV["CI"].present? makes every test depend on CI. Leave CI out of [env]: it is removed from the tests, so it is unset on both sides and nothing is eager-loaded. (In hash, a laptop never matches a runner; in set, every test loads every application file.)
Time zone data. Use gem "tzinfo-data" for every platform, and keep set = { TZ = "UTC" } in [env] (vci init writes it): vci sets it on both sides.
Lockfile platforms. Gemfile.lock must list both arm64-darwin and x86_64-linux.
Ruby and vendor/. Pin the exact Ruby in .ruby-version, and keep vendor/.keep committed.
Locale. Attest with a UTF-8 locale (LANG=en_US.UTF-8). For Rails, LANG and LC_* are built-in ignore: they reach the tests unhashed, and the default encoding they decide is compared as part of the toolchain. Runners use UTF-8.
libvips and library versions. Homebrew's libvips is a different file from the runner's, so every file that loaded it runs, and so does every file when the SQLite, libyaml or time zone data versions differ. Set libraries = "any" under [policy] to match libvips by name and ignore those versions, and install libvips on the runner. See macOS to Linux.

RSpec

RSpec runs through the Rails adapter, so everything in the Rails section applies: the engine's test database, the single process, what is recorded, and the changes needed for a Linux runner. Tested with rspec-core 3.13 and rspec-rails 8. Plain Ruby projects that use RSpec without Rails work the same way.

Set up

cd your-app
bundle lock --add-platform x86_64-linux
vci init --adapter rails --key ~/.ssh/id_ed25519.pub
git add vci.toml .vci/allowed_signers .git-meta Gemfile.lock \
  .rspec && git commit -m "Enable vci"

vci run spec/models/order_spec.rb \
  --key ~/.ssh/id_ed25519
vci plan --base-ref origin/main
vci push

# vci.toml
project = "."
adapter = "rails"
runner = "rspec"

Option files

RSpec reads options from several places, some outside the repository. vci makes local and CI runs behave the same:

Source	Treatment
`.rspec`	read, and an input of every spec file: editing it runs everything
`~/.rspec`, `$XDG_CONFIG_HOME/rspec/options`	never read
`.rspec-local`	never read (it is usually git-ignored, so CI could not match it)
`SPEC_OPTS`	removed unless listed in `[env]`; in `hash` its value is compared, in `set` vci gives it the same value everywhere

What is recorded

On top of the Rails list:

spec_helper.rb, rails_helper.rb and every file they load;
the listing of directories loaded by a glob, such as spec/support and FactoryBot's spec/factories, so a new support or factory file invalidates the specs that load them;
shared examples, custom matchers, fixtures and file_fixture files;
the versions of rspec-core, rspec-rails, rspec-expectations and rspec-mocks.

Tip: a spec that only requires spec_helper has far fewer inputs than one that loads rails_helper, and survives most changes to the app.

What is refused

a pending or skipped example: pending, skip, xit, xdescribe;
a run where a filter left examples out: :focus, fit, tag filters, --only-failures, --example;
errors outside examples, such as a load error or a failing before(:suite) hook;
a process that exits non-zero after RSpec finished, for example SimpleCov's minimum coverage;
retries that hide a failed first attempt (rspec-retry), and parallel_tests or turbo_tests;
system specs that drive a browser.

Two things to know. A spec generated by rails generate contains a pending example, so it is refused until you replace it with real examples. And an example with no expectations is still attested: unlike Minitest, there is no check for that yet.

Run conditions vci sets for every spec file

vci also fixes the conditions of every run: the seed defaults to 0, the example status file is redirected to a temp directory, and SimpleCov writes outside the repository.

Laptop to Linux runner

Make the same changes as for a Rails app: see Laptop to Linux runner under Rails. The personal option files (~/.rspec, .rspec-local) are never read, so they cannot make your laptop and the runner differ.

Full reference

This page is a summary. The complete lists of recorded inputs, refusals and known limitations for each adapter are in the reference.