2.10 Accessibility (WCAG) Scanning

How the A11Y_SCAN step works, the three gating modes (OFF/RATCHET/STRICT) that let you decide whether findings can block a build, and where scan results show up in reports and the Health Center.

This page covers testing web pages, and the browser an agent used. For the wider picture of accessibility testing for AI agents (operability, conversation checks, how to add checks to your agent tests and how to choose whether they fail a run), see 1.19 Accessibility Testing for AI Agents.

What It Does

An A11Y_SCAN step runs an axe-core accessibility scan against the page in its current state – wherever you place it in a case, mid-flow after a modal opens, on a fresh page load, and so on. It returns a structured list of violations, each with a rule id, an impact level (critical/serious/moderate/minor), help text and a link to the underlying WCAG success criterion, and the specific DOM nodes that failed. CertiQA helps you run these scans and report the results toward WCAG, ADA and Section 508 – a scan result is a testing signal, not a compliance certification, and CertiQA does not represent it as one. (Note: Authoring A11Y_SCAN steps directly from the visual canvas action picker is scheduled for Phase 6; steps can currently be imported via JSON/YAML or defined via the API).

{
  "action": "A11Y_SCAN",
  "a11yTags": ["wcag2a", "wcag2aa"],
  "a11yRuleDenylist": ["color-contrast"]
}

Verifying Scan Results

Two verification types read the most recent A11Y_SCAN in the same case: A11Y_NO_VIOLATIONS fails the step if the scan found any violation at all, and A11Y_VIOLATION_COUNT_UNDER fails if the violation count is at or above a threshold you set. Both throw a clear error if no A11Y_SCAN ran earlier in the case – there is no implicit scan-on-demand.

Choosing How Findings Affect Your Build

Accessibility findings are first-class in reporting from the moment a scan runs, but whether they can block a build is entirely your call, set per CertiQA account via certiqa.a11y.gating-mode. This is deliberate: CertiQA is not an enforcement platform, and a customer partway through remediating a large surface needs findings visible in every report long before they can afford to gate on all of them.

Mode	Behavior
OFF	Findings are captured and shown in every report, but never affect gate status.
RATCHET (default)	Only findings that are new since the case’s accepted baseline – and rated critical or serious – can block. Findings already present when the baseline was accepted don’t re-block on every run.
STRICT	Any finding at or above a configured severity threshold blocks, regardless of baseline.

Accepting a Baseline (RATCHET)

RATCHET compares each scan against a case’s accepted baseline – the set of findings you’ve explicitly signed off on – identifying each one by its rule id plus the affected nodes’ selectors (not by its help text, which can shift between axe-core versions). Accept the current findings as the new baseline from the Accessibility Health panel, or via the accessibility baseline/accept API endpoint; either way it requires Approver-level project access, the same bar as triggering auto-quarantine, since it changes what can block future runs.

Where to See Results

Accessibility Results Show Up In

• The JSON, JUnit (as a block) and HTML reports certiqa-engine produces for every run, plus a one-line console summary.
• The Accessibility Health panel (/analytics/accessibility) – cross-linked from Test Health – showing every scanned case, its violations by impact, and its current gate status.
• The Run Report screen in the app, with a per-finding “File Ticket” button that opens the same Jira integration used for failed-test tickets.

Auditing Keyboard Operability

The Keyboard Audit step (KEYBOARD_AUDIT) presses Tab through the page, records each place keyboard focus lands, and reports problems as accessibility findings alongside any axe-core results. It never presses Enter or Space, so it does not activate anything, and it audits the page as it stands, so close any open dialog first. The optional step value is the maximum number of Tab presses (default 200).

Finding	What it means
kbd-unreachable	An interactive control (including a button or link built from a plain element with an ARIA role) that Tab never reaches
kbd-trap	Focus went round in part of the page while links, buttons or fields elsewhere were never reached
kbd-no-focus-indicator	No visible change was detected when the control got focus
kbd-unnamed-stop	Focus landed on a control with no accessible name
kbd-positive-tabindex	A tabindex above 0 changes the order away from the page order

Controls that are correctly outside the tab order are not reported: disabled or hidden controls, tabindex="-1", and all but one radio button in a group. Limits to know: the focus-indicator check looks for a change in outline, shadow, border, colour or text decoration, so an indicator drawn some other way is reported as missing; the inside of cross-origin iframes is not audited and is listed as such; it is designed for Chromium, because Tab behaves differently in other browsers; and it checks keyboard behaviour, not how any screen reader announces the page. Findings help you test toward WCAG 2.1.1, 2.1.2, 2.4.3 and 2.4.7. They are not a conformance result.

Running an Agent With Only the Keyboard

For agent and hybrid cases that drive a browser, the keyboard-only profile removes the pointer and direct targeting. Tick “Keyboard-only agent browsing” in the run dialog, pass --keyboard-only, or set keyboardOnly on trigger_run. The agent then has no browser_click, browser_type or browser_select (a call to one is refused and recorded as refused). It has browser_press_key, which accepts a key or a combination such as Shift+Tab and returns where keyboard focus went (role, accessible name and states, never a field’s value), browser_type_focused to type into whatever has focus, browser_focus_state, and the read and screenshot tools. It is how a keyboard user experiences the page, as far as an automated agent can.

To compare, use “Compare keyboard-only” in the run dialog, or start_operability_comparison with profile: "keyboard". It runs the scope normally and keyboard-only and reports each case’s result in both. Treat a difference as a lead, not a verdict: each side is one run of an LLM agent, so it can be variance, and a keyboard-only run that fails may mean the agent is poor at keyboard navigation rather than that the page is. For a deterministic check of the page itself, use the Keyboard Audit step above.

Scanning the Page an Agent Ends On

For agent and hybrid cases that drive a browser, you can have CertiQA scan the page the agent’s browser session ended on, with no A11Y_SCAN step to write. Tick “Scan the page the agent ends on (axe-core)” in the run dialog, pass --a11y-scan-after-agent to the engine, or set a11yScanAfterAgent on trigger_run. The violations are reported as that case’s accessibility findings, so they appear in the run report and in the accessibility analytics like any other scan. It is off by default because new findings can affect accessibility gating. A case where the agent never opened the browser has no page to scan and reports no scan.

In the case’s transcript drawer the scan sits beside any AI accessibility review (a judge using the ACCESSIBILITY rubric), in two separate panels. The scan is rule-based and repeatable. The review is an AI’s opinion: it can flag risks such as unhelpful alt text, and it cannot certify WCAG conformance. They are never combined into one score.

Wiring It Into CI (Optional)

A separate, opt-in script (check-accessibility-gate.sh in certiqa-integrations) queries the Accessibility Health API after a run and exits 0/1/2 following the same CI exit-code contract as the rest of CertiQA – see CI/CD Exit Codes. It is never added automatically to a generated pipeline; you add it as its own step only once you’ve decided accessibility should participate in your build gate.

Before You Rely On This

• A11Y_SCAN checks what axe-core can detect automatically – keyboard-only navigation, screen-reader phrasing and similar judgment-based criteria still need manual review. Treat a clean scan as a strong automated pass, not a full audit.
• Reports say a case “has no detected violations” or “helps test toward WCAG 2.1 AA” – never that a page “is WCAG compliant” or “is ADA certified.”
• The gating mode and severity threshold are account-wide settings today, not yet configurable per project from the UI – the Accessibility Health panel shows the active mode as a read-only badge.

• 🩺 How to Quarantine Flaky Tests — The other opt-in, customer-decides-enforcement mechanism in CertiQA – a useful comparison for how gating without forced enforcement works elsewhere in the platform.
• ⚙️ CI/CD Local Runner & Exit Codes — The 0/1/2/3 exit-code contract that the optional accessibility CI gate script follows.