Date: 2026-10-19 Status: Living document. Every piece of work a decision record deferred, parked or left open, in one place, with its source, its status and what it waits on.
Each item below gets its own decision record, locked with Emil before Code builds it. Phase numbers are given as each record is written.
Build order after Phase 39 (updated 2026-10-20, with R5’s three records placed). Phase 39 runs as locked. R4 comes after it, or fits between its gates: TFP-206, TFP-209, TFP-216, TFP-217 Part A and TFP-222, with TFP-222 before Phase 39’s TFP-155.
benchmarks/2026-10-27-d26-wsl/). The run-time baseline comes from
Gate D’s re-measurement.
Then, before item 2: Phase 47, Mr Onion in a target’s CI pipeline
(added 2026-10-27, Emil: after Phase 39, before Phase 43; numbered here
without renumbering the items below, which the tables’ Order columns cite).
- Standard, read-only, against an app the CI job starts on localhost;
deployed URLs capped until Phase 43.
- The crawl and generate logic moves into src/lib/pipeline/ first
(TFP-263); then the CLI, baselines, a Docker image and a GitHub Action.
- Tickets TFP-263, TFP-245 to TFP-248; TFP-249 after Phase 43 (D31).
- Versioning (TFP-257 steps 1–5) comes before its image.
- Paused at TFP-247 for Phase 48 (Emil, 2026-11-08): TestForge becomes
Mr Onion (D39) on branch phase-48-mr-onion, merged to main in one go,
then tagged 0.3.0. TFP-247 and TFP-248 then ship mr-onion. Tickets
TFP-275 to TFP-281; no measurement session while the branch is open.
- Resumes at TFP-247 (owner decision, 2026-11-12): Phase 48 is
complete (2861a4c, v0.3.0).
- Complete 2026-11-13 at TFP-248 (owner decision, 2026-11-13), Gate A
passed; released as 0.4.0. TFP-249 stays deferred as D31, after
Phase 43.main on 2026-11-14; see its
“Review before building” section.MRONION_ACTIVE switch on, it opens the active gate
and the script-payload test runs again.Not placed yet: D25, a summary email after a site run. Its record is still to write.
Alongside, in Cowork:
r6-skills-relook-2026-10-20.md, tickets
TFP-236 to TFP-239, built with R4. D12 and D20 are settled there.Standing rule (Emil, 2026-10-19): any site Mr Onion hosts locally under Docker keeps its code in WSL’s Linux filesystem, not in a Windows folder mounted into Docker. See D26.
While the Standard sets are built (D1, D2 with D9, D3): test the section, not the whole app (Emil, 2026-10-20).
check:template-unchanged, which shows every other section’s output
is unchanged;The same rule for the Comprehensive work (Emil, 2026-10-20). It covers Phase 44, D10, D7 and D4.
Every new Standard set (D1, D2 with D9, D3): Phase 39’s rule still holds. Each record states the set’s run-time cost against the margin measured after Phase 39 (with-bugs at TFP-208: 1,306 s of 1,800 s).
The rest:
Statuses:
“Order” is the place in the triage result’s build order.
| # | Item | Source | Emil (2026-10-19) | Status | Order | Waits on |
|---|---|---|---|---|---|---|
| D1 | SEO Standard set (the draft set in the Phase 38 proposal) | Phase 38, deferred | Include | Planned | 2 | D8, and Phase 39 complete, so its checks are written once on the shared visit (TFP-212). The record copies the draft SEO set from the coverage proposal (Claude Docs, 2026-10-05) into the repo, and states its run-time cost |
| D2 | Functional checks for the 30 WAI-ARIA APG patterns | Phase 38, deferred | Include | Planned, with D9 | 3 | D1; TFP-214 (the “in place” kind of control). The record decides whether it ships in batches |
| D3 | The rest of the accessibility and security Standard sets | Phase 38, deferred | Include | Planned | 4 | D2. The passive parts need nothing more. Attack-shaped checks sit at the active gate, which D8 opens |
| D4 | Comprehensive SEO, then the highest-risk areas from run history | Phase 35, deferred | Include. Triage: after D7 | Planned | 8 | D7; reuses D1’s checks. The highest-risk areas are chosen from run history at that point |
| D5 | All-areas Comprehensive runs, scheduling, per-area jobs | Phase 35, deferred | Include | Planned | 9 | D4; Phase 39 Gate D (workers) |
| D6 | Model-written tests counting toward the verdict | Phase 35 Decision 5, deferred | “Depends on what these model-written tests find”. Decided 2026-10-20: outside the verdict until measured, per scenario type | Decided in Phase 44 Decision 6: a scenario type joins the verdict when every planted defect of its type is found on the fixture and there’s no unlabelled failure on clean fixture, clean Toolshop or 3 real sites, with Emil’s approval | 9 (with Phase 44) | Phase 44’s measurement |
| D7 | Comprehensive’s negative tests, by ISTQB technique, with a technique-coverage measure | Phase 38, deferred | No problems. Triage: before D4 | Planned | 7 | D10. Includes scenario #8 (scenario-gap-candidates-2026-10-19.md) |
| D8 | The ownership confirmation and the active gate, with the script-payload test. The load-testing run is split out to D24 | Phase 35 Decision 8; Phase 38 | A tester confirms once, not repeatedly. Triage: see “D8, confirming once” below | Planned | 1 | Phase 39 and R4. Locked as Phase 43 (phase-43-ownership-confirmation-decision-record.md, Emil 2026-10-19; TFP-218 to TFP-221). It unblocks D3’s active parts, D10, D24 and Phase 39’s ownership row |
| D9 | INP measured through real interactions | Phase 38, deferred | Needs explaining (below). Triage: with D2, banded findings only | Planned, with D2 | 3 | D2; Phase 41’s GET-only interactions |
| D10 | Authenticated and active security | Phase 35, deferred; GitHub issue #3 | Not sure it should be parked entirely. Triage: the first Comprehensive item | Planned (sequence only; no design yet) | 6 | D8, D3, D19. Overlaps scenarios #3, #4 and #9; R5’s scenario-gaps record says which side owns each |
| D24 | The opt-in load-testing run, designed with the ISTQB Performance Testing syllabus | Phase 38, deferred; split from D8 (2026-10-19) | Split out of D8. Placed last for now; moves up if needed sooner | Planned | 10 | D8’s load-testing tick |
| # | Item | Source | Emil (2026-10-19) | Status | Order | Waits on |
|---|---|---|---|---|---|---|
| D26 | The local Toolshop copies move into WSL’s Linux filesystem, with the README’s start timings corrected | Emil, 2026-10-19, from TFP-211’s slow with-bugs start (about 6 min, composer once 10.5 min) | Wanted, after Phase 39; and the rule for every locally hosted site | Built 2026-10-27, awaiting verification (benchmarks/2026-10-27-d26-wsl/) |
Moved into Phase 39 by Gate D’s ruling 3 (2026-10-26), before the re-measurement | Phase 39 complete: the move also speeds up the Toolshop’s API, so mid-phase it would shift the run times Phase 39 compares. What it covers is below |
| D27 | A comparison benchmark: Mr Onion against the “QA My App” plugin (Anthropic Directory, community) and Playwright’s Test Agents (planner, generator, healer; added 2026-11-02) on the local Toolshop copies | Emil, 2026-10-20 | Wanted: use outside testing tools as yardsticks, not inside Mr Onion | Planned | 8, after D3 and D19, once Standard is complete (Emil, 2026-10-20: earlier, it would mostly point at work already planned) | D26 (both tools run on the WSL setup), D1 to D3, D19. What it covers is below |
| D28 | A security sweep of Mr Onion’s own model-code paths, with the “Verified Security Sweep” plugin (Anthropic Directory, community) or an equivalent review | Emil, 2026-10-20 | Wanted, before the sandbox is built | Planned | 9, after D19, before Phase 45 | Nothing; it informs Phase 45’s design. What it covers is below |
| D29 | A retention policy for saved runs and their evidence | Emil, 2026-10-21, after 278 MB of leftover run folders were cleared | Wanted: stop cluttering with records and evidence nobody opens | Planned | 4, straight after Phase 46 | Phase 46 (the action belongs on its screens). What it covers is below |
| D30 | A solo performance pass, so real sites without the ownership confirmation still get performance judged | Emil, 2026-10-21, after TFP-157’s ruling that timed checks are Not run under the request cap | Wanted | Planned, with D3 | 7, with D3 | TFP-157 (Phase 39). What it covers is below |
| D25 | A summary email after a run against a site, to the tester’s inbox, with the way back to the full report | Emil, 2026-10-19, with TFP-217 (the same for Mr Onion’s own nightly CI) | Wanted: a quick view for whoever is in charge of testing that site | To plan | Not set | A decision record of its own. It is most useful with D5’s scheduled runs. The record decides the questions below |
| D31 | Active and interactive gates against deployed URLs in CI (TFP-249) | Phase 47, deferred (2026-10-27) | Wanted, after Phase 43 | Planned | After Phase 43 | Phase 43, TFP-248 |
| D32 | An npm package for the CI runner | Phase 47 Decision 8 (2026-10-27) | Deferred: the image pins Chromium, npm wouldn’t | Open | - | TFP-247 |
| D33 | Models (gap writer, local model, OpenAI) in CI | Phase 47 Decision 9 (2026-10-27) | Off in CI for now | Open | - | Phase 45’s sandbox |
| D34 | Comprehensive all-areas runs in CI | Phase 47, deferred (2026-10-27) | Wanted, with D5 | Planned, with D5 | With D5 | D5, TFP-248 |
| D35 | Native integrations for other CI systems (GitLab component, Azure DevOps task) and SARIF output for code scanning | Phase 47, deferred (2026-10-27) | Not before someone asks | Open | - | TFP-248 |
| D36 | Workers that back off when a site slows down (adaptive concurrency) | Phase 39 re-measurement ruling 1 (2026-10-27) | Deferred: a fixed limit of 3 holds for now | Open | - | Phase 39 complete |
| D37 | The stale re-check’s re-runs judged for cap starvation (it has no timing reporter, so a reused test it re-runs on a capped host keeps its re-run result) | Phase 39 ruling 4 amended (2026-10-28) | Deferred: rare path, documented as a known limitation | Open | - | TFP-264 |
| D38 | “Comprehensive” becomes “Break Me” once more areas have Comprehensive sets: UI, report and docs wording | Emil, 2026-10-28 | Wanted, when more areas are built | Open | - | More Comprehensive areas (D4, D7) |
| D39 | A new application name (many “testforge” repos exist): repo, package, testforge-skills plugin, data folder (%LOCALAPPDATA%\TestForge, one-time move), TESTFORGE_* settings, test-ID prefixes, TFP- prefix, testforgeapp@gmail.com |
Emil, 2026-10-28; proposed name “Mr Onion” (Emil, 2026-10-30) | Wanted. Tagline “Mr Onion: peel back every layer” (Standard the first layer, “Break Me” the core). Ruled out: “Mr Pickles” (Cucumber’s “pickles”, the Pickles docs tool, a TV brand), “Señor Onion” (could read as a caricature; the ñ breaks technical names), “Allium” (JUXT’s spec language with a Claude Code plugin; Alumnium, an AI testing tool), “Tearjerker”, “Core Sample”, “Shallot” (owner’s preference) | Done 2026-11-12 (2861a4c, Phase 48 merged 2026-11-11; Phase 48 complete 2026-11-12). Locked → Phase 48 (Emil, 2026-11-08): phase-48-rename-to-mr-onion-decision-record.md. Test-ID and TFP- prefixes stay; the mailbox is D44. Nothing is ever created again as emil3663/testforge, so the old URL keeps redirecting (Phase 48 Decision 10) |
Phase 48, after Phase 47’s TFP-246, before TFP-247 | - |
| D40 | An image a later check’s own action brings in (a lazy-loaded image it scrolls to) charged to that check, not to the test that loaded the page, so a check the cap starved that way is Not run, not Failed | Phase 39 rulings on 70a0ad3, item 2 (2026-10-28) |
Deferred: a known limitation, #16 | Open | - | TFP-264 |
| D41 | Runs that really go through a proxy: the browser using it (Playwright’s proxy option) and the lowercase variable names (http_proxy, https_proxy, no_proxy); TFP-209 only passes the uppercase names through |
TFP-209’s verification (2026-11-02) | Deferred, decided with Phase 47 (CI runners behind a corporate proxy) | Open | With Phase 47 | Phase 47’s record |
| D42 | A small System One decision model (Jev first; others benchmarked against it) as a fallback when the deterministic rules can’t settle which Standard tests apply: page type, a form’s purpose, which APG widget a control is, whether a library test fits a component | Emil, 2026-11-04, revisiting Martin’s System One pointer from Phase 34 (only the typed-verdict pattern was adopted then, in TFP-119) | Wanted, with the reasoning below; Jev because most people he has spoken to use it, with a benchmark of the others at the time; packaging looked at closer to the time, with its pros and cons | Open | With D2, when its widget recognition is designed | D2; the labelled set below. What it covers is below |
| D43 | Ending the child-side double setting names: saved site-suite units and library entries that check TESTFORGE_* names are regenerated or rewritten, so runs stop setting both names |
Phase 48 Decision 8 (2026-11-08) | Deferred: until then every run sets both names | Open | Not before 1.0’s removal ticket, and not part of it | Phase 48 merged; the 1.0 record |
| D44 | A Mr Onion mailbox in place of testforgeapp@gmail.com and its +ci and +<site> aliases, if one is wanted |
Phase 48 Decision 4 (2026-11-08) | Deferred: the old mailbox stays; the nightly’s sender name becomes “Mr Onion nightly”; the nightly’s TESTFORGE_SMTP_PASSWORD secret keeps its name until this lands (Phase 48 Decision 9, amended 2026-11-08) |
Open | - | Phase 48 merged |
D39, before the name is committed (2026-10-30). “Mr Onion” is the
proposed name. First checks: no software, app, GitHub project or testing tool
found under it; mr-onion and mronion unregistered on npm (2026-10-30).
Before anything is renamed:
mronion / mr-onion
in the extensions wanted.emil3663/mr-onion (free under the owner’s
account), npm mr-onion (re-check just before publishing), container image
ghcr.io/emil3663/mr-onion (Phase 47, TFP-247).MRONION_*, with the TESTFORGE_* names read as a
fallback for a while), the local data folder and its one-time move, the
plugin name, the test-ID prefixes (TFFU-, TFSE-…), the ticket prefix,
and the testforgeapp@gmail.com addresses.v0.2.0 is tagged under the current name when Phase 39
completes: tags and releases survive a GitHub repo rename, and old links
redirect.D42, a decision model for choosing Standard tests (2026-11-04). Martin pointed at System One models (small, fast models for a single yes/no or pick-one decision: Jev, Kev, Laya, GLiNER2.5-Decide) during Phase 34. Then, only the pattern was adopted: each gate records a typed verdict (TFP-119), and the checks behind it stay deterministic code. Emil asked (2026-11-04) whether such a model could help decide which Standard tests run against a page.
| # | Item | Source | Emil (2026-10-19) | Status |
|---|---|---|---|---|
| D11 | Phase 28, OpenAI path: Gate A open, no live completion ever succeeded | Phase 28 | Park | Parked (2026-10-19). Revisit if a funded key is used |
| D12 | Phase 29, the crawl skill’s drift from dom-snapshot.ts, and its live verification |
Phase 29 (draft); Phase 22 | Needs to be addressed; discuss | Decided in R6 (2026-10-20): the skill describes the method and points to the code for the rules (r6-skills-relook-2026-10-20.md); TFP-237 |
| D13 | Reliability’s element-mapping (local model) | Phase 33, open | Needs explaining (below) | Parked with the local-model path (2026-10-19) |
| D14 | The database test’s skip on pages that save nothing (local model) | Phase 33, open | Needs explaining (below) | Parked with the local-model path (2026-10-19) |
| D15 | Template-versus-local-model comparison | Phases 31–33, deferred | Check it | Closed: measured in Phase 34 (TFP-120, phase-34-option-b-measurement-report.md) and confirmed by Emil 2026-09-28. With page data the model was never called (0 gaps on both targets; 15 of 17 cases identical to the template’s suite) |
| D16 | The hollow cookie/session test | Phase 34, deferred | Not a valid test | Closed: replaced in Phase 36 (TFP-125) by a check on the site’s own session cookies (expiry and scope), skipped with a reason when there are none |
| D17 | Reliability gap-filling (local model) | Phase 34, deferred | Needs explaining (below) | Parked with the local-model path (2026-10-19) |
| D18 | A UI control for gap-filling | Phase 34, deferred | Needs explaining (below) | Closed: decided against in Phase 34 Decision 9 (confirmed by Emil 2026-09-28). Revisit only if a site shows gaps with page data |
| D19 | The failure-triage gate (real defect, test bug, environment, or inconclusive) | Phase 34, deferred (Martin’s review) | Needs explaining (below) | Planned (2026-10-19), order 5: after D3, before D10. It is calibrated on Toolshop’s labelled failures (TFP-178), R1’s 5 false link failures, and the new Standard sets’ failures, so it is in place before Comprehensive and the practice sites multiply the failures |
| D20 | Skills used as rules at run time, and proving the skills | Phase 36, deferred | Needs explaining (below) | Decided in R6 (2026-10-20): no new run-time rules; each plugin skill proven by known-answer cases through its real invocation (TFP-239) |
| D21 | TFP-169, the occasional connection reset in a security test | Known-bug backlog | Can we force it? | Open. TFP-208 (verified) didn’t see it in its 6 runs, nor R1 in its 3; 3b44370’s diagnostics stay. TFP-210’s nightly run watches it alongside D23. Forcing it (below) is a ticket if it recurs |
| D23 | performance-bands.test.ts:69, a timing test, failed once in TFP-208’s check run on 02ed0ef (the fixture page built for an LCP of about 3 s measured over 4 s, with the machine loaded); passed alone 2 of 2, and in the 6 full runs before |
TFP-208’s verification (2026-10-19) | — | Open: known flake. TFP-210’s nightly run will show its rate; a ticket if it recurs. Second timing test, 2026-10-21: timing.test.ts’s “/busy never goes quiet” settle cap (0 capped of 1) failed once in TFP-155’s full unit run (9efbd5f); passed alone 3 of 3 and in that night’s four earlier runs. Likely the same cause: under load the fixture’s polling slows past the 500 ms quiet window. One data point each; a recurrence of either gets a ticket, and the fixture’s /busy is to be made busy by a mechanism that load can’t slow. Third, not timing, 2026-10-21: route.inspect.test.ts’s closed-port test failed once in a full unit run on a969cbf (600 of 601, 1,144 s): closedPort() frees a port, and before the browser tried it something else answered there (ERR_INVALID_HTTP_RESPONSE, not the ERR_CONNECTION_REFUSED it expects). Passed alone 8 of 8, and in that day’s other full runs. A recurrence gets a ticket: a port that stays refused, not one freed and hoped for |
| D22 | The real-site backlog’s stale header (TFP-162 to 164 shown open) | Real-site backlog | Update it | Closed 2026-10-19: header and order list updated (cefa24d, be688d3, f4ed183, 837de4f, verified 2026-10-07) |
D8, confirming once. Proposal for the design:
Triage (Emil, 2026-10-19): the proposal stands, with these calls:
MRONION_INTERACTIVE=1 with
no confirmation (Phase 38 Decision 1).The behaviour doesn’t change. The amendment lands in the same commit as D8’s record.
D9, INP. Interaction to Next Paint is the Core Web Vital for responsiveness: the time from a click, tap or key press to the next frame the browser draws.
Triage (Emil, 2026-10-19): built with D2. It adds no page visits: it times Phase 41’s interactions and D2’s APG checks.
D13, D14 and D17, the local-model items. All three are weaknesses of the optional local model (Ollama), not of the template:
**/email-input/**, or repeated one
test under element names.The template covers reliability and database itself, and since Phase 34 the model is never called when page data exists. Parking all three with the local model costs nothing today.
D19, the failure-triage gate. A step that judges each failure as a real defect, a fault in the test, an environment problem or flake, or inconclusive.
D20, skills at run time. Mr Onion’s own app reads none of the skills
under skills/. Phase 36 asked whether some should become rules the app
applies, for example:
honest-coverage’s “what a green run leaves unknown” added to every
report;dont-stub-what-you-prove run as a check on generated suites.“Proving the skills” is Martin’s idea: test each skill against cases with known answers, as Mr Onion is tested.
D25, what its decision record has to settle. TFP-217 emails Mr Onion’s own CI result from GitHub. A site run is different: it runs in the app, on the tester’s machine.
testforgeapp+<site>@gmail.com.localhost) opens only on the tester’s
machine.D26, moving the local copies into WSL.
Why (as it stood before D26 was built, 2026-10-27). Testsmith’s checkout sits in a Windows folder
(..\practice-software-testing) mounted into Docker. Every file operation
crosses from Windows into Docker’s Linux. The slow steps on each start are:
composer service, with composer install and then
chown -R www-data /var/www over all of vendor/;npm install.with-bugs waits for composer to finish (bug 80’s log folder; see
scripts/toolshop-local/README.md). One start took 10.5 minutes, and
TFP-211’s took about 6. A first install took 20 to 25 minutes. Docker
recommends keeping bind-mounted code in the WSL filesystem on Windows.
What the item covers:
5a6ccc26).start-toolshop.ps1 and stop-toolshop.ps1 point at
it, through a \\wsl$\... path or by calling wsl docker compose. A
start is still one command from PowerShell.COMPOSER_PROCESS_TIMEOUT=0;compose.local.yml;chown. Composer’s chown to uid 82 doesn’t
depend on Windows, so this one probably stays.The rule from now on applies to every site Mr Onion hosts locally under
Docker, such as the-internet’s local copy (Phase 39, TFP-158) and the
practice sites (practice-site-targets-2026-10-19.md). Its code lives in
WSL’s Linux filesystem from the start.
D27, the comparison benchmark.
Why: Mr Onion’s goal is to find every issue on a site. A comparable tool run on the same target, with the same answer key, shows what Mr Onion misses and what it finds that the other doesn’t. That is measured, not claimed.
Not inside Mr Onion: outside tools are yardsticks only. A third-party plugin in a run would bring someone else’s judgement into the verdict, an update cycle Emil doesn’t control, and licensing questions. Most of these plugins also work from source code, while Mr Onion tests from the outside.
When (Emil, 2026-10-20): once Standard is complete, after D3 and D19. Run at step 2 it would mostly point at Standard sets not yet written. Run at the very end it could no longer change the order.
What it covers:
benchmarks/2026-10-06-toolshop-known-bugs/answer-key.md. Every
difference between the two tools is listed, bug by bug.benchmarks/, with what Mr Onion
should take from it, ticketed or added here.write-playwright-test, and “A11y
Enforcer” against D3’s accessibility set, the same way, if D27 proves
useful.D28, the security sweep of the model-code paths.
Why: model-written tests turn text read from a site into code that runs on Emil’s machine (Phase 42, batch 2 ruling 5). TFP-202 hardened that path. A review that proves each finding with a failing test checks the hardening before Phase 45 contains it and Phase 44 puts it to real use.
What it covers:
src/lib/gap-writer/ (the writer’s call, code-check.ts, gap-run.ts);src/lib/candidates/ (binding and applying);src/lib/run/child-env.ts;D29, retention for saved runs.
Why (2026-10-21):
runs/ holds 1,410 saved runs, 684 MB (652 MiB), kept forever..playwright-runs the same day was a different
thing: the evidence folders of runs never saved, none of the 505 a saved
run’s id, and loose run files from before TFP-222’s run folders. Its
remaining leak is TFP-242, in R4.)What its decision record settles (Emil’s choices):
runs/ sits in the repo folder (gitignored).
Phase 42’s data folder (MRONION_DATA_DIR) is where it belongs for a
tester’s install.D30, the solo performance pass.
Why (2026-10-21):
What it covers:
D12, for the R6 discussion. The crawl-and-classify-pages skill is a
written copy of how dom-snapshot.ts classified a page on 2026-09-24:
#/ routes, link inventories.Options:
dom-snapshot.ts changes.D21, forcing TFP-169. Probably, yes. TFP-208’s fix (decbfad) found
that unit tests shared one folder that another run could change mid-start,
which is a likely cause of the same kind of start-up failure. A ticket
could try to reproduce it on purpose:
If none of these reproduces it on the code after TFP-208, close it as
fixed by decbfad, and keep 3b44370’s diagnostics in place.
scenario-gap-candidates-2026-10-19.md: ten scenarios.
practice-site-targets-2026-10-19.md: the practice sites to run against.post-phase-42-review-plan.md: R5’s three decision records (the Docker
sandbox, app screens, scenario gaps) and TFP-206, TFP-209, TFP-210.phase-39-run-concurrency-decision-record.md: locked, next after TFP-208
and TFP-210.