mr-onion

Phase 18 — Skills Taxonomy & Reliability Backlog

Version: 1.0 Date: 2026-09-16 Status: Shipped — TFP-039 landed (code-structure and refactor-audit calibrated/trial-run, docs/WORKING_AGREEMENTS.md now exists in this repo, skills/engineering/CALIBRATION.md has the full output); skills/ROADMAP.md marks both “Status: shipped.” This doc’s header was never updated when the work landed. Corrected 2026-09-23 (Cowork verification pass).

Context

skills/ currently holds one shipped skill (write-playwright-test, all testing-judgment), four recorded candidates that are also testing-judgment (honest-coverage, dont-stub-what-you-prove, find-the-hidden-platform, plan-to-specs), and two drafts under skills/proposed/ (code-structure, refactor-audit) that are a different kind of thing — engineering discipline, not testing judgment. Emil confirmed these should be split into two explicit categories rather than left flat, and asked to look at the Google SRE book (https://sre.google/sre-book/table-of-contents/) as a source for a third kind of judgment: identifying where reliability issues are likely to occur, as part of the test harness itself.

Since only one skill exists so far, restructuring now is cheap — later, with more skills and cross-references, it wouldn’t be.

Standing rule, added after this backlog was first drafted: a skill’s instructions must resolve inside this repo. If a skill’s format, examples, or tooling come from another one of Emil’s repos, that material gets copied into testforge as part of the ticket that builds the skill — a skill that only works if another repo happens to be checked out next to this one is not actually usable. This was found the hard way once already (TFP-039, WORKING_AGREEMENTS.md) and a full pass turned up more of the same, so it’s recorded here rather than left to be rediscovered one candidate at a time:

Candidate What it needs Where it actually is
honest-coverage the two example write-ups it generalizes test-assist/DESKTOP_STABILITY_MATRIX.md, cah-pwa/tests/README.md
dont-stub-what-you-prove TA-227’s write-up test-assist/docs/ta227-228-e2e-testing-brief.md
find-the-hidden-platform TA-230’s write-up one entry inside test-assist/TESTASSIST_BACKLOG.md — worth extracting to its own short doc rather than porting the whole backlog file
plan-to-specs the working tool it generalizes cah-pwa/scripts/generate_playwright_tests.js + generate_test_stability_matrix.js (built once); the --check version REIMAGINING.md describes is on a generator-improvements branch not yet located — that’s a gap, not just a copy

Also worth a decision before more skills get built: code-structure and refactor-audit both follow cah-pwa/skills/’s own template (What this skill is for / Inputs / Output format / Rules / Worked example / How Emil invokes this), but write-playwright-test — the one skill actually shipped — doesn’t use that shape. Converging on one format is cheap now and gets more disruptive the more skills exist.

Scope (this phase)

In scope

  1. Split skills/ into skills/testing/ and skills/engineering/.
  2. Record three new engineering-concern candidates grounded in specific SRE book chapters — reliability risk, not general reliability philosophy.
  3. Review and fix the two existing engineering-concern drafts (skills/proposed/) before any decision to promote them.

Out of scope

Backlog

TFP-037 - Split skills/ into testing/ and engineering/

TFP-038 - Record SRE-informed and ISTQB-informed candidates

TFP-039 - Review, fix, and trial skills/proposed/ before promotion

  1. TFP-037
  2. TFP-038
  3. TFP-039

Suggested Milestone Gates

Gate A - Phase 18 taxonomy checkpoint