Date: 2026-09-17
Status: Reference record — no implementation scheduled
Related: docs/decisions/phase-16-local-first-decision-record.md (Decision 2:
“A deployed SaaS version is explicitly deferred, not cancelled”),
docs/decisions/product-roadmap.md (“Explicitly out of scope”)
Phase 16 deliberately deleted the partial scaffolding that previously implied
TestForge was heading toward a deployed version — .env.production,
.env.staging, DEPLOYMENT_CHECKLIST.md, and the backend-proxy layer that
existed to talk to an external service. That deletion removed the reasoning
along with the code. This document records that reasoning so the next time a
hosted TestForge is actually considered, it starts from an honest list of what
would be required, instead of from the same half-built assumptions Phase 16
just cleared away.
Nothing described below is a plan. Nothing described below is scheduled.
Why local-first doesn’t need it: TestForge runs on a single developer’s own machine, in their own OS session. Whoever can reach the process already has whatever access the process has — there is no second party to authenticate against.
Why a hosted version requires it: A hosted TestForge is reachable by more
than the person who started it. Without authentication, /api/run’s
arbitrary-code-execution surface (see below) is reachable by anyone who can
reach the URL.
Current state: Not implemented. No login, session, token, or identity concept exists anywhere in the codebase.
Why local-first doesn’t need it: /api/run executes caller-supplied
Playwright code with the same privileges as the user running the dev server.
Locally, “the caller” and “the user running the server” are the same person —
this is a tool running its own owner’s code, not untrusted input.
Why a hosted version requires it: Once a request can originate from
someone other than the machine’s owner, /api/run’s current execution model —
write the suite to disk, run it in-process — becomes remote code execution
with no isolation, no resource limits, and no blast-radius containment. A
hosted version needs generated suites to run in an isolated, resource-bounded
environment (a container or a dedicated execution sandbox) per run, not in the
host process.
Current state: Not implemented. src/app/api/run/route.ts and
src/lib/run/executor.ts execute Playwright directly in the Next.js server
process; README.md’s Security Model section states this plainly as the
local-only tool’s intended behavior, not a gap to be closed in place.
Why local-first doesn’t need it: One developer, one machine, one run at a time (or a small number, all trusted). In-memory state disappearing on restart is an inconvenience, not a correctness problem.
Why a hosted version requires it: Run state today lives entirely in
runJobs, a process-global Map (src/lib/run/jobs.ts) — globalThis-scoped
specifically so it survives Next.js’s dev-server module reloading, and
explicitly not designed to survive a process restart or to be scoped per
caller. A hosted, multi-tenant version needs run state that is durable across
restarts/deploys, isolated per tenant, and queryable without holding it in the
serving process’s memory. This is also a stated prerequisite for TestForge’s
own Phase 17 (run persistence and honest coverage) — that phase gives runs
identity and puts them on disk, but scoped to a single local user; per-tenant
isolation of that persisted state is the SaaS-specific extension this
document is naming, not something Phase 17 itself builds.
Current state: Not implemented. runJobs is in-memory, process-global,
and has no tenant or ownership concept.
Why local-first doesn’t need it: Captured evidence (traces, videos, screenshots, HTML report exports) is written to the local filesystem the developer already controls and can browse directly.
Why a hosted version requires it: Evidence generated by someone else’s run needs to be stored somewhere durable, access-controlled per tenant, and servable back to that specific caller — not written to a shared local disk every caller’s request happens to share.
Current state: Not implemented. Evidence capture writes to local disk
paths under playwright.config.ts’s outputDir/trace/video/screenshot
settings, with no remote storage backend, no access control, and no
per-tenant separation.
Why local-first doesn’t need it: There is exactly one tenant: the person running the tool.
Why a hosted version requires it: Every prerequisite above compounds into this one — authentication identifies a caller, but nothing today scopes execution, run state, or artifacts to that caller once identified. Two concurrent hosted users must not be able to see, cancel, rerun, or read the evidence from each other’s runs.
Current state: Not implemented. No concept of a tenant, user, or ownership boundary exists anywhere in the run, report, or storage code paths.
This is not a design for any of the five prerequisites above, not a sizing estimate, and not a commitment to build them. It exists so that if a hosted TestForge is revisited later, the person doing that work reads this instead of re-discovering — or silently re-assuming — the same gaps Phase 16 chose to delete rather than paper over.