mr-onion

Phase 24 — Reliability Audit Findings Backlog

Version: 1.0 Date: 2026-09-22 Status: Complete. TFP-065/066 d19c165 (2026-09-22); TFP-067/068 a95027a and TFP-069/070 2bd1b32 (2026-09-23). Reconciled with git 2026-09-29.

Context

An independent agent audit of TestForge’s API/persistence layer, run while this project was paused, surfaced ten findings. Cowork independently re-verified every one against the actual repository (not trusted at face value — two 🔴 findings looked structurally real, and one part of the audit, the Plane/Docker Compose test-target architecture, looked suspicious on first read since nothing in this repo’s prior docs had mentioned it; it checked out as real, docker-compose.yml genuinely stands up Plane as a local test target). Every finding below was confirmed against real line numbers, real code paths, or real config — none of this backlog is taken on the audit’s word alone.

Emil’s call: fix the two 🔴 critical findings now (real defects — a silent-hang failure mode and a path-traversal gap — not feature work to sequence behind Phase 17). Everything else goes on the roadmap rather than being fixed in this pass, so it isn’t lost and isn’t rushed either.

Decisions

  1. TFP-065 and TFP-066 are fixed now, ahead of Phase 17 Group 2 — real defects take priority over new feature work already in flight.
  2. TFP-067 through TFP-070 are recorded as roadmapped backlog (Sprint 3 below), not built in this pass.
  3. Both TFP-065 and TFP-066 ship with real automated test coverage as part of the same commit, not verified once by hand and left uncovered — the same gap Phase 17 Group 1 shipped with and had to close in a follow-up (fb905c2) applies here too, and isn’t worth repeating a third time.

Backlog

TFP-065 - Guard JSON.parse in finalizeRunJob against a corrupted Playwright report

TFP-066 - Validate runId before any file-path operation

Roadmap (not built in this pass)

TFP-067 - Add req.json() error handling to all POST handlers

TFP-068 - Polling hook resilience: retry/backoff, and a terminal-status safety net

TFP-069 - Tighten CSP for production; drop unsafe-eval

TFP-070 - Small cleanups: dead env var, default base URL, hardcoded per-test timeout

  1. TFP-065 and TFP-066 (parallel — no shared files, both P0, fix now)
  2. TFP-067 (mechanical, low risk)
  3. TFP-068 (pairs conceptually with TFP-065)
  4. TFP-069
  5. TFP-070

Suggested Milestone Gates

Gate A - Critical findings closed

Gate B - Reliability audit backlog closed