mr-onion

Phase 35 Measurement Report — Comprehensive Testing, First Slice

Date: 2026-10-01 Ticket: TFP-136 (Acceptance Criteria 2, real site, and 6) Evidence: benchmarks/2026-10-01-phase35-first-slice/ — every run’s full result as /api/run returned it, plus the generate answer for Comprehensive runs. Setup: my next start on 127.0.0.1:3100 (production build of 198dbec), no local model configured unless stated, Emil’s machine (Windows), one run per row.

Bottom line

Durations (Acceptance Criteria 6)

“Run” is the Playwright run as /api/run timed it. “Total” adds page inspection and generation, measured from the client.

Target Run Pages Tests Run Total Per page
Fixture Standard security 3 inspected 3 3 s 10 s —
Fixture Comprehensive security (passive) 3 + probe 13 5 s 8 s ~1.7 s
Fixture Standard accessibility 3 inspected 7 4 s 10 s —
Fixture Comprehensive accessibility 3 6 5 s 9 s ~1.7 s
the-internet Standard security 2 inspected 4 11 s 21 s —
the-internet Comprehensive security (passive) 20 (cap) + probe 81 224 s 231 s ~11 s
the-internet Standard accessibility 2 inspected 42 131 s 140 s —
the-internet Comprehensive accessibility 20 (cap) 40 155 s 160 s ~7.8 s

Passive security on the-internet (Acceptance Criteria 2, real site)

Before the fix, run aeb33b15 (before-fix-security-the-internet.json): 5 failures, all from the “page responds” assertion, all filed under A02.

Page Answer Read by hand Verdict
/basic_auth, /digest_auth, /download_secure HTTP 401 Behind HTTP authentication. Refusing is the server working. Template error (T13 to T15)
/add_remove_elements HTTP 404 The site graph normalises the path. The real page is /add_remove_elements/ (200). Template error (T16)
/authenticate HTTP 404 /login’s POST target, known only from a form-submit edge. Not a page. Template error (T17)

Fix (198dbec):

After the fix, run 12e385fb (security-none-the-internet.herokuapp.com.json):

Accessibility on the-internet

Run 84094801 (accessibility-none-the-internet.herokuapp.com.json), as corrected at Gate B:

Corrected at Gate B (2026-10-01). This section first reported the run as 17 passed, 20 failed, 3 not run. On the three pages answering 401 (/basic_auth, /digest_auth, /download_secure), TFP-121’s field-name check had passed, with no page to check. It now follows 198dbec’s rule and is Not run there, as the axe scan already was.

Every failure was confirmed against the page:

Failure Pages Evidence Labelled
color-contrast (serious, WCAG 1.4.3) 17 Every flagged element is rgb(43,166,203) with white, at 2.83:1. That is under 4.5:1 for normal text, and under 3:1 even for the 32px “Login” heading. R4
image-alt (critical, 1.1.1) /broken_images, /dynamic_content <img> with no alt R5
label (critical, 4.1.2), and TFP-121’s check /checkboxes Text beside the box, no label R6
select-name (critical, 4.1.2), and TFP-121’s check /dropdown No <label> on the page R7
label (critical, 4.1.2), and TFP-121’s check /dynamic_controls Checkbox and text input unlabelled R8
list (serious, 1.3.1) /disappearing_elements A <ul> directly inside a <ul> R9

The Standard accessibility run on the same site (98cfc51f) passed all 42 of its tests: its hand-written checks look at roles, names and focus of the inspected elements. It is not a WCAG scan, and on this site it missed all of the above.

Fixture results (from TFP-133 and TFP-134)

Caveats

Roadmap

product-roadmap.md’s Phase 35 “Still open: timing” bullet is replaced with these measurements.