inlinedocs
Portal ↗

Automation

The sandbox

Some assurance can’t be done by reading a diff — you have to run the code. inline runs code in ephemeral, isolated microVMs: it clones the PR, runs the existing tests, authors new ones where coverage is thin, and can boot your app for a real browser end-to-end pass — capturing video and screenshots as evidence. The sandbox also backs auto-fix verification and code-graph builds.

It runs off the review critical path. A sandbox run degrades to partial or failed — it never hangs a review.

# Environment

Sandboxes boot from a prebuilt image with common runtimes and browsers preinstalled, so a run starts fast without waiting on a full toolchain install. A warm pool keeps idle boxes ready; each run cleans its clone before reuse.

# What a test run does

  1. Fetch the diff, acquire a sandbox, and clone the PR head (works for forks too).
  2. Probe the project — Node / .NET / Python / Go, lockfiles, existing test files, and monorepo layout — mapping changed files to the package that owns them.
  3. Plan with a fast model: package manager, runner, an ordered step list (install → build → typecheck → lint → test), whether existing tests cover the change, and whether to author new ones. “No test runner” is not a dead end — it plans an install and hands off to authoring.
  4. Execute steps within time and cost budgets. Only an install failure is fatal; output is classified per framework (Vitest/Jest/Mocha/.NET/pytest).
  5. Author tests (optionally) with a Claude loop that can only write test files, when the change isn’t already covered.

# Install resilience

A large share of real-world runs used to end with zero evidence because an install failed. The sandbox has a fallback chain: out-of-memory is surfaced honestly; peer-dependency conflicts retry with legacy resolution; a flaky package feed retries ignoring failed sources; transient network errors retry; a missing test script rewrites to a direct runner invocation; and an “unrunnable” plan is rescued into an install-only plan from the lockfiles so authoring can still produce something.

# Browser end-to-end

When enabled and the PR touches a runnable web app, inline runs a real browser pass:

  1. Boot the app in the background and poll its URL until it answers.
  2. inline writes the browser test config itself (never the model) — video on, screenshot on, trace on failure.
  3. A Claude loop authors specs, runs them, and can read failure screenshots back as images to debug, repairing a couple of times.
  4. If the app needs login and test credentials are configured, it authors a login and reuses the session; without them it runs unauthenticated and reports “login required.”

Evidence — videos, screenshots, traces — is stored (size- and count-capped) and attached to the run.

# Watching it live

Each run has a share-link page at /t/<token> that polls every few seconds and streams step output GitHub-Actions style. Video evidence is served with range support. The portal twin lives under Reviews → (review) → Tests.

# Secrets

Test environment is resolved best-effort from committed .env.test / .env.ci / .env.example (keys only), layered under team secrets you store in the portal. Secret values are never surfaced — only which keys were available, recorded as a report artifact.

# Enabling & limits

Enable Sandbox tests and Browser e2e in review config (per-repo over workspace, both off by default). Runs are time-, cost-, and artifact-budgeted so they always terminate cleanly and never balloon in cost.