Version: 1.0
Date: 2026-09-22
Status: Complete. TFP-065/066 d19c165 (2026-09-22); TFP-067/068 a95027a and TFP-069/070 2bd1b32 (2026-09-23). Reconciled with git 2026-09-29.
An independent agent audit of TestForge’s API/persistence layer, run while
this project was paused, surfaced ten findings. Cowork independently
re-verified every one against the actual repository (not trusted at face
value — two 🔴 findings looked structurally real, and one part of the
audit, the Plane/Docker Compose test-target architecture, looked
suspicious on first read since nothing in this repo’s prior docs had
mentioned it; it checked out as real, docker-compose.yml genuinely
stands up Plane as a local test target). Every finding below was confirmed
against real line numbers, real code paths, or real config — none of this
backlog is taken on the audit’s word alone.
Emil’s call: fix the two 🔴 critical findings now (real defects — a silent-hang failure mode and a path-traversal gap — not feature work to sequence behind Phase 17). Everything else goes on the roadmap rather than being fixed in this pass, so it isn’t lost and isn’t rushed either.
fb905c2) applies here too, and isn’t worth repeating a third time.JSON.parse in finalizeRunJob against a corrupted Playwright reportbug, reliabilitysrc/lib/run/jobs.ts:126 parses Playwright’s JSON
report with no try/catch, while the readFile immediately above it is
wrapped in .catch(() => "") — a real asymmetry. Because
finalizeRunJob runs fire-and-forget (void finalizeRunJob(job) in
executor.ts), a thrown parse error is an unhandled rejection that’s
silently swallowed: job.result never gets set, job.status stays
"running" forever, and useRun’s 800ms poll (which only stops on
snapshot.result) polls indefinitely. The run looks permanently “in
progress” in the UI with no way to recover short of restarting the dev
server.JSON.parse(reportText) call in a try/catch. On failure,
treat it the same as a missing report (parsedReport = null), push a
warning into job.outputChunks so the cause is visible in the run’s
raw output, and let the existing “no report” finalize path run so
job.result and job.status are always set.finalizeRunJob a corrupted JSON
report file and asserts the job reaches a terminal status with a
result, not that it merely “doesn’t throw.”src/lib/run/jobs.ts.node:test, whichever actually
exercises finalizeRunJob — match the existing pattern for that file)
proving a corrupted report can’t hang a run.job.jsonReportFile results in job.result being set and
job.status reaching a terminal value (not "running" forever) —
proven by a real test run, not by reading the diff.test:unit both still pass.runId before any file-path operationbug, securityGET /api/run’s runId comes straight from a
query param with only a truthiness check, then flows into
loadRunRecord/markRunAccessed/runEvidenceDir (persistence.ts),
which build paths via join(RUNS_ROOT, runId, ...) — path.join
resolves ../ segments, so an unvalidated runId is a real (if
locally-scoped, per TestForge’s own local-only threat model)
path-traversal surface. PATCH /api/runs (pin/archive) has the same
gap, and archiveRun’s rm(runEvidenceDir(runId), { recursive: true,
force: true }) makes a crafted runId a real deletion risk, not just a
read risk, if a matching run.json can be placed at the target path.isValidRunId check (real run IDs are randomUUID()
output — validate as a UUID, not a generic “no slashes” blocklist).GET /api/run, DELETE /api/run, and PATCH /api/runs
before any call into persistence.ts.runId (../../x, absolute path,
embedded null byte if relevant to this runtime) must be rejected with
400, not reach the filesystem at all.persistence.ts or
src/lib/run/ — Code’s call on the exact file).runId that isn’t a valid UUID is rejected with 400 before any
fs call, for every affected route.test:unit and
the regression suite both still pass.req.json() error handling to all POST handlersbug, reliabilitysrc/app/api/run/route.ts:27,
generate/route.ts:14, crawl/route.ts:337, introspect/route.ts:58,
and save-suite/route.ts:22 all call await req.json() unguarded. A
malformed body (empty, truncated, wrong Content-Type) throws, and
Next.js turns that into a raw 500 instead of a clean 400 — the correct
pattern already exists in the same codebase (runs/route.ts,
run/route.ts’s own DELETE handler: await req.json().catch(() => (...))).reliability, uxuseRun.ts and useCrawl.ts both abandon
polling on the first failed fetch() (network blip, dev-server
restart) — the backend job keeps running, but the user loses all
visibility and sees an error, with no retry. Separately, both hooks
only stop polling by checking snapshot.result truthy, not by checking
a terminal runStatus as a second signal — currently safe only because
finalizeRunJob always sets result, which is exactly the kind of
single-point-of-failure assumption TFP-065 just showed can break.runStatus values as a second stop condition,
not only result.useRun.ts, useCrawl.ts.unsafe-evalsecurity, technical-debtnext.config.ts’s script-src carries both
'unsafe-inline' and 'unsafe-eval'. 'unsafe-eval' is a dev-mode/HMR
requirement, not a production one; 'unsafe-inline' could move to a
nonce-based approach. The rest of the security header set is already
strong (X-Frame-Options: DENY, frame-ancestors 'none', restrictive
Permissions-Policy) — this closes the one real gap in it.unsafe-eval conditional on NODE_ENV !== "production"
(or the build-mode equivalent); evaluate a nonce-based script-src for
inline scripts as a stretch goal, not a requirement of this ticket.next.config.ts CSP header logic.unsafe-eval; dev mode is unaffected.technical-debt, documentationplaywright.config.ts’s outputDir reads
process.env.TESTFORGE_PLAYWRIGHT_OUTPUT_DIR, which is never set
anywhere in the codebase — dead reference; the executor’s --output
CLI flag overrides it in practice, so this is confusing rather than
broken.http://localhost:3000 (TestForge’s own URL) as
the default base URL when none is provided — correct only when
TestForge is testing itself; for any other target (Plane on :80,
test-assist on :4321) the user must know to override it. Worth a
clearer default or at least a documented callout, not necessarily a
behavior change.docker-compose.yml’s PLANE_SECRET_KEY=change-me-before-production
default is fine for the documented local-only use, but isn’t yet
stated as a hard requirement anywhere if this ever runs beyond a
single local machine.