Pushing suites

For the complete documentation index, see llms.txt

View Markdown Open in ChatGPT Open in Claude

Tests reach a box in one direction only: you push them. A box holds no deploy key and no token to anybody’s repository and never pulls, so nothing is on it that was not pushed to it.

cd unotest && npx @unotest/web bundle push --box https://tests.example.com --token unob_…

UNOTEST_BOX_URL and UNOTEST_BOX_TOKEN supply the two values instead of flags — a CI job usually takes them from its secret store.

What a bundle is

bundle push packs the unotest/ directory — the suite package: its scenarios, helpers, collections, unotest.config, package.json and lockfile — into one tar.gz and uploads it. The product around it never travels, and neither do its dependencies: the box runs npm ci over the suite’s own manifest.

In a git project, git decides what belongs. The file list is git ls-files --cached --others --exclude-standard, so whatever your .gitignore keeps out stays out. Content is read from the working tree, so uncommitted changes travel — the manifest records dirty: true and the viewer labels the bundle wip.
Secrets never travel. unotest/.env, .env.<name>, .secrets, .secrets.<name> — and any <name>.env / <name>.secrets — are excluded even when committed. A file that looks like one and still reaches the packer stops the push. Environment values live on the box; see Environments & values.
Also excluded: node_modules, .git, run and queue directories, .unotest/, .tmp, .debug, the viewer lock file.
Install hooks are stripped from the packed package.json (postinstall, prepare, …): a box installs the suite, it does not run your project’s hooks. The push reports what it removed.
Your dependencies’ install scripts run on the box, so a native module builds or fetches its binary as usual. They run in a container of their own — the bundle tree is its only writable mount, and none of the box’s state is in reach — which is what makes running somebody’s postinstall on a shared machine affordable.
The id is the content. bundleId is a hash of the packed files, so re-pushing an unchanged suite is an instant no-op (already had this exact bundle), and two people packing the same tree get the same id.
Names of externals travel, values do not. The bundle carries the names your .env / .secrets files (or their .example twins) declare, so the box can answer the push with what an environment cannot supply.

Refused before upload

What would fail on the box is refused on your machine, with the file, the line and the fix:

a scenario that reads a file outside unotest/ (MODULE_NOT_FOUND on the box, hours later);
no lockfile, or no @unotest/web in the suite’s dependencies;
a symlink;
a secrets file inside unotest/;
a suite over 256 MB;
a schedules registry that cannot be read — the box reads it to know what runs on its own.

--dry-run packs and checks without uploading; --out <file> keeps the archive (tar -tzf shows what is inside); --json prints { bundleId, status }.

Refused by the box

The box verifies the archive before writing a single file and answers with a typed refusal; the CLI prints it with what to do next:

Refusal	Meaning
`unauthorized`	The token is unknown, revoked, or for another box.
`min-version`	The bundle pins an `@unotest/web` older than the box runs. Bump it in `unotest/package.json`, commit the lockfile, push again.
`scope-mismatch`	The token and the box disagree about which project this bundle is for: either the token is scoped to a project the box does not serve, or this repository belongs to another project on the box. A token for a project the box does not serve is replaced by `box access`; a repository mapped elsewhere is the box’s own mapping, and a new token does not change it.
`ambiguous-project`	The box serves several projects and the token names none. `box access` mints a pair scoped to one of them.
`too-large`	Over the box’s upload limit (128 MB by default).
`malformed` / `id-mismatch`	The upload did not arrive intact — retry; if it repeats, a proxy is rewriting the body.

The token alone decides which project a bundle lands in. Nothing in the request can name one.

The first line of a successful push says which project that was — ✓ uploaded to <box> · project shop — when the box reports it; an older box or CLI prints the line without it. If the bundle’s repository maps to no project on the box, the push still lands, and a warning: under that line names the project and the token it went in on: a token for the wrong project would have put it there just as quietly. Wrong project? Push with that project’s token. Right one? The bundle is where it belongs: the box just does not list the repository under that project. box create lists the project’s git origin (without credentials) for its project, so pushes from that checkout land without the warning.

Which project that is belongs to the box: a box serves the projects named by the environments it was built with — shop/prod is the environment prod of the project shop — and a push token opens the door of one of them. A box built with environments of a single project has one, which is what the cabinet’s form makes unless you type more.

After a successful push the box may add notes: environments that hold no value for a name the suite declares, with the command that sets it.

Running after a push

cd unotest && npx @unotest/web bundle push --run --env test --collection smoke

--run orders a run of the bundle just pushed, on the same token:

--env <name> names the box environment to run in — cd unotest && npx @unotest/web box envs lists the ones this box serves.
--collection <name> names what to run; repeat for several. Without it the box runs what that environment’s schedules name.
--pr <number> ties the push to a pull request. The queue shows it (PR #412@1a2b3c4), and a newer push of the same pull request withdraws its older runs that are still waiting; a run that already started is never killed.

The box answers 202 as soon as the runs are queued, with their run ids — a suite takes minutes, and a request held open that long is a timeout, not a result. The ids are real from that moment. The command prints each one, with its link into the box’s viewer when the box reports one (a box that does not know the address it is reached at sends none):

✓ queued on test: smoke
  smoke: col-smoke-… — https://tests.example.com/_guard/runs/shop/test/col-smoke-…

--json carries the same (runs: [{ runId, collection, url }], url only when the box sent it, plus superseded when older runs were withdrawn).

A CLI and a box that both support it can also wait for the verdict: --follow asks the box how each run is doing, on the same push token, and exits 0 when every suite passed, 1 when one failed and 2 when there is no verdict to give — a box too old to be followed, a box that restarted and forgot the run, or an hour without an ending. The box shows a run’s state only to the token that ordered it, so a CI job needs nothing beyond the token it pushes with.

From @unotest/web 0.48.0, --follow also prints what each run printed while it goes, so the CI log says why a suite failed, not only that it did:

▶ login
  log: signing in as qa@example.com
  ✗ unotest/e2e/login.js:9 AssertionError: expected the dashboard heading
✗ login — failed
✗ smoke failed (exit 1) — https://tests.example.com/_guard/runs/shop/test/col-smoke-…

Every scenario’s start and end, each failed step with where it is and its error, and the run’s log and note lines — not every passing step; the viewer has those. The box masks the environment’s secrets in them (‹secret:NAME›) before they leave it: a CI log of a public repository is read by everybody. With several collections each line starts with [collection]; with --json the lines go to stderr and stdout stays the one JSON document. A box too old to give lines says so once, and the verdict and link follow as before.

Run requests are rate-limited (60 per minute per box, and per token).

Refusals: unknown-environment, scope-mismatch (the environment belongs to another project), unknown-bundle (the box no longer holds it — push again), no-collections (nothing named and no schedule for that environment), malformed (the box is older than this CLI — push without --run).

Run on every push

Nothing runs the suite on a push by itself: something has to push the bundle and order the run. On GitHub, init writes that workflow:

npx @unotest/web init --ci github

It writes .github/workflows/unotest.yml at the root of your git repository. Its steps run in unotest/ (in a monorepo, the project’s unotest/). On every push to any branch it runs npm ci there, so the @unotest/web version pinned in unotest/package.json is the one used, and then npx @unotest/web bundle push --run --collection examples --json. Replace examples with your collection, or drop --collection to run what the environment’s schedules name; a box with several environments also needs --env <name>.

Recent versions of init give the workflow’s token read access only (permissions: contents: read) and pin actions/checkout and actions/setup-node by commit, with the version in a comment. The file is yours afterwards: Dependabot’s github-actions updates the pins. Every branch runs it on purpose: anyone who can push can edit the workflow in their own branch, so a branch filter does not guard the secrets. To run on fewer branches, add branches: under push.

Then add two repository secrets: UNOTEST_BOX_URL and UNOTEST_BOX_TOKEN. Their values are in unotest/.env and unotest/.secrets after box create. Outside a git repository init writes no workflow. A re-run leaves the file as it is; --force takes the template again and keeps yours as .bak. unotest/package.json keeps its dependencies and your @unotest/web pin under --force: it only gets what the template has and the file lacks.

The box CI pushes to has to outlive the day the cloud gives a box rented without a lifetime: rent it with npx @unotest/web box create <name> --ttl none, or take the lifetime off one you already rented with npx @unotest/web box extend <name> --ttl none. Otherwise it is deleted a day after it was rented, and the workflow’s pushes fail from then on.

With the box’s GitHub App, the box orders runs for pull requests and deployments itself and only needs the bundle: drop --run from the workflow then, or each push is run twice.

A minimal CI job

permissions:
  contents: read
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
        with: { node-version: 22 }
      - run: npx @unotest/web bundle push --run --env test --collection smoke --json
        working-directory: unotest
        env:
          UNOTEST_BOX_URL: ${{ secrets.UNOTEST_BOX_URL }}
          UNOTEST_BOX_TOKEN: ${{ secrets.UNOTEST_BOX_TOKEN }}

Exit codes: 0 pushed (and queued), 1 the box refused or was unreachable, 2 the suite was refused on your machine — nothing left it. With --json, a failed run order after a successful upload still reports the bundle, so the job does not push it again for nothing.

From a GitHub webhook

With the box’s GitHub App installed (see Administration), the box also reacts to your repository’s events. Every mapping lives on the box, never in the repository:

a deployment event’s environment name is mapped to one of the box’s environments; an unmapped name is logged and dropped, never guessed;
pull requests run in a named environment, or each in its own preview environment (below), running the collections the project’s CI configuration names or the environment’s schedules;
the bundle is matched by commit sha against what has been pushed. A commit no bundle was pushed for waits up to 15 minutes for your CI job to upload, then the check fails saying so;
fork pull requests are not run. Their head is somebody else’s branch, and running it would hand it the environment’s secrets. A maintainer re-requesting the check in GitHub is the opt-in.

The box reports one check per collection (unotest / <collection>): queued when the event arrives, in progress when the run takes its slot, completed with a summary and a link into the viewer.

Pull-request environments

A project with preview deployments can give each pull request an environment of its own. unotest configures a template with you:

the target URL shape, with {pr} where the number goes (https://pr-{pr}.preview.acme.dev) — the box substitutes, it never deploys;
variables every preview run gets;
how many may exist at once (5 by default) and how long one survives after its last run (72 hours by default).

The environment is named pr-<n>, created when the pull request is first seen and removed when it closes or goes unused past the TTL. Preview environments learn the suite’s schedules but never arm them: what CI asks for runs, the clock does not. Secrets are set per environment by an administrator; a preview environment that needs one is not one that should be created automatically.

Bundles in the viewer

On a box the viewer shows what it is running:

Bundle badge in the status bar — main@1a2b3c4, or wip for a bundle from an uncommitted tree, with the author, the pinned @unotest/web and who switched the environment last.
Switching. Clicking the badge lists the bundles pushed to this project and switches the environment to one. The switch is a maintenance ticket: a running suite finishes, then the code changes, then the next run starts on the new bundle. An environment with allowSwitch off refuses (follows its own branch — it cannot be switched by hand).
Following the branch. An environment follows the newest bundle of its defaultBranch until somebody selects one by hand; that selection survives restarts and later pushes until the next person switches. Where the menu offers follow <branch> again, it lets go of the selection, and the environment moves to the branch’s newest bundle. Schedules always run the newest bundle of defaultBranch, regardless of what anyone selected — a tester’s switch never redirects tonight’s suite.
Runs remember their bundle. The runs list marks runs that ran on another bundle, and a this bundle only filter narrows the history to the bundle in front of you.
Project / environment switcher in the tree header, where a local viewer shows the checkout directory. Each environment is its own viewer process, so switching is a full page load.

Removing a bundle

From @unotest/web 0.48.0, the token that pushes can also take a bundle back — one pushed with the wrong project’s token, or one nobody wants:

cd unotest && npx @unotest/web bundle rm 1a2b3c4d5e6f7a8b

The id is the one bundle push printed. The box looks for it in the token’s project only, and answers not found in project ”…” the same way whether another project holds it or none does — a token learns nothing about projects that are not its own. To remove a bundle from another project, use that project’s token.

What is in use stays: a bundle an environment runs, or one a waiting or running ticket asked for (a run, or a switch on its way to it), is refused with the names of the environments. For the one an environment runs, switch it away first:

cd unotest && npx @unotest/web bundle rm 1a2b3c4d5e6f7a8b --switch-to 9f8e7d6c5b4a3210

--switch-to starts the same switch the viewer does — recorded as made by the token — and answers as the switches start; the bundle stays. Run the same bundle rm again once they have landed. An environment with allowSwitch off cannot be switched by hand: push a newer bundle to its branch, then remove the old one. A box older than bundle removal refuses and removes nothing; update the box, then retry.

Retention

Each push leaves an archive and, once installed, a node_modules. The box keeps the newest 5 bundles per branch regardless of age and removes older ones after 30 days (both configurable) — never a bundle an environment currently points at or a waiting run has asked for.