This is ISTQB’s structural (white-box) testing category: code coverage
of the paths a suite’s execution actually reaches, as distinct from
black-box testing’s question of whether the assertions on the paths it does
reach are the right ones. A suite can have excellent assertions and still
have this problem — the two questions are independent. “Coverage” already
has a precise, checkable meaning in that vocabulary (statement, branch,
condition coverage), and this skill reuses it rather than inventing a new
category: a skipif-guarded test, a default-off flag, or a permanent
monkeypatch each produce the same structural fact — a real code path exists
that the suite’s own execution graph never touches, on any run, regardless
of what the tests that do run assert.
test-assist’s TA-230 (TA-230.md, beside this skill) is the
concrete case. FloatingLauncher’s register_global_hotkeys parameter
defaults to False, for a sound reason — the True path claims real,
process-wide OS state (RegisterHotKey) that concurrent tests would fight
over. That single flag cost real time three times in two days, in three
genuinely different ways:
_apply_hotkey_labels() runs only when the flag is
True. A launcher rebuild removed two widgets it referenced. The app
died on startup with an AttributeError — while 350 tests stayed green,
because every one of them avoided the only path that would have caught
it.skipif-guarded to Windows. Every run on a Mac reported
green by construction — not because the code was fine, but because
the only tests that could have failed never ran there. It surfaced on
someone else’s Windows machine, the worst place to find it.TestAssist.exe from an
unrelated manual check still held the real Win32 hotkeys mid-release,
so the guarded tests’ own RegisterHotKey calls failed for a reason
that had nothing to do with the code change being gated. Not a defect —
the OS call was behaving correctly — but the failure read as a
regression, under time pressure, during a release.None of the three announces itself in a green suite. Each needed someone to go looking. This skill is that looking, generalized past this one flag.
For each path found, report it as a path the suite cannot fail for — not as a generic “consider adding a test”:
## Path
The guard, and exactly what it hides — file, line, condition.
## What runs it
Which tests, if any, execute this path at all, and under what conditions
(a specific CI job, a specific platform, an opt-in flag no default run
sets).
## What's unverified while it's skipped
The specific thing that could be wrong here and the suite would still
report green. Not "this might have bugs" — the actual shape of a defect
this gap would hide, if one exists.
## Consequence if it breaks silently
What a real break here looks like from the outside — a crash on startup,
a feature quietly absent, a security check that never ran.
## Provenance
Found by a find-the-hidden-platform audit on <date>, scope <what was
covered>.
Then a summary table, ranked by consequence, so the reader can triage
without opening each one — same discipline as refactor-audit’s.
Three signals to audit for, each a different mechanism for the same structural fact:
| Signal | What it hides | Ask |
|---|---|---|
skipif (or equivalent) guard |
A whole test, on some platforms/environments, all the time | Does any CI job or local run actually execute this test, or does every run this project has satisfy the skip condition? |
| Default-off flag gating real shared state | An entire code path, everywhere, unless something opts in | How many tests run with the flag flipped on? If it’s a handful, whatever those tests assume about the machine they run on is now an assumption the whole path inherits silently. |
| A monkeypatch that permanently stands in for the thing under test | The real implementation, forever, in every run | Does any test in the suite ever exercise the real function this patches, anywhere — or does every path to it go through the substitute? |
Distinguish this from dont-stub-what-you-prove. That skill asks
whether a single test’s assertion targets the same thing its own mock
replaces — a claim-correctness question, scoped to one test. This skill
asks whether the real implementation is ever executed by anything in
the suite, on any run — a reachability question, scoped to the whole test
suite’s execution graph. A test can pass dont-stub-what-you-prove’s check
(its assertion targets something other than what it mocks) while the thing
it mocks is never exercised for real anywhere else either — that’s this
skill’s finding, not that one’s.
A default-off flag is a hazard independent of what it guards. Even if
the True path is itself fully covered by the handful of tests that flip
it, the flag being off everywhere else means every other test’s
assumptions about the machine — that this OS resource is free, that this
state is untouched — go unchecked by anything. Name that separately from
whether the guarded path itself has coverage.
“Partly mitigated” is not “closed.” TA-230’s own coverage and
cross-platform fixes existed on an unmerged branch for a period before
landing on main — a mitigation that hasn’t shipped protects nothing yet.
State what’s actually running against the default branch, not what exists
somewhere.
Rank by what a silent break looks like, not by how obscure the guard
is. A skipif-guarded test for a rarely-touched debug flag matters less
than a default-off flag gating the one code path a startup crash lives in.
Name what you could not check. Whether a guarded path actually gets exercised in any real usage — production traffic, a manual QA pass, another team’s fork — isn’t visible from reading the suite. Say so.
Applying the three signals to the flag TA-230 itself is about
(register_global_hotkeys, before its fixes landed):
## Path
FloatingLauncher(register_global_hotkeys=True) — every function this flag
gates, including _apply_hotkey_labels() and the real RegisterHotKey/
UnregisterHotKey calls. The flag defaults to False; roughly 20+ tests
construct a launcher for unrelated reasons and never touch it.
## What runs it
A handful of TA-211 tests explicitly pass register_global_hotkeys=True.
Three of those are additionally skipif-guarded to Windows — so on any
non-Windows run, even that handful shrinks further, to whatever remains
unguarded.
## What's unverified while it's skipped
Every other test's assumption that constructing a launcher has no
side effect on real OS state goes unchecked outside the guarded handful.
Separately, on a non-Windows run, nothing at all exercises the Windows-only
subset — a regression there is invisible until it reaches a Windows
machine, which is exactly what happened (surfaced on someone else's
Windows machine, not caught locally).
## Consequence if it breaks silently
Demonstrated, not hypothetical: a removed-widget AttributeError crashed
the app on startup while 350 unrelated tests stayed green, because the
only path that would have called the missing method was the one every
test defaults away from.
## Provenance
find-the-hidden-platform audit (worked example), applying TA-230's own
findings (TA-230.md, beside this skill) to its own subject rather than a new codebase.
Not measured here: whether this specific flag's mitigations (coverage for
the widget-existence check, a cross-platform-safe attribute scan) have
landed on the default branch at the time this audit runs — check TA-230.md's
own closing note for the latest status.
Summary table:
| Finding | Consequence | Signal |
|---|---|---|
register_global_hotkeys=True path |
High — demonstrated startup crash, real | Default-off flag gating real shared state |
| The three Windows-only TA-211 tests | High — regression invisible off Windows until it reaches one | skipif guard |
“Run find-the-hidden-platform over this repo — what paths can the suite never fail for?”
Or narrower, once a suspicious flag or guard is already spotted: “is this
skipif/default-off flag hiding anything real?”