Model Context Protocol (MCP) Reference

The CertiQA MCP server connects AI coding assistants (Claude Desktop, Cursor, Copilot) directly to CertiQA test suites, runs, and failure diagnostics.

# Connect directly via stdio
npx -y @certiqa/mcp-server

Available MCP Tools (29 Tools)

list_projects

List all test automation projects. A project groups suites, scenarios, and cases that are run together in a pipeline.

create_project

Create a new test automation project. Provide a short key (slug for URLs/CLI), a display name, and optionally a description, owner team, and default base URL for tests.

create_suite

Create a test suite. A suite groups scenarios and cases that are executed together. Optionally assign it to a project.

list_suites

List all test suites belonging to a project.

create_scenario

Create a test scenario. A scenario is a logical grouping of related test cases within a suite (e.g. ‘Login flow’, ‘Checkout’).

create_case

Create a test case with its steps defined as IR (Intermediate Representation). Steps are the atomic actions: LAUNCH, CLICK, ENTER_VALUE, ASSERTION, etc. Each step has an action, optional selector (CSS/XPATH/TESTID), and value.

add_case_to_suite

Add an existing test case to a suite with a given execution order. Links the case so it runs as part of the suite.

trigger_run

Trigger a test run for a project. This queues execution of all suites under the project. Returns a run ID for status polling. Note: actual execution happens in CI — this endpoint signals intent.

For agent and hybrid cases it also takes four accessibility options: accessibilityConstrained (browser tools target only by visible text or role and accessible name), keyboardOnly (browser tools limited to the keyboard), a11yScanAfterAgent (axe-core scan of the page the session ended on) and a11yMode (off, report, ratchet or enforce: how steps marked as accessibility checks count in this run; ratchet fails a case only for a check that passed in one of the last 20 earlier runs). The first three are refused if the run has no agent or hybrid case. See guide 1.19.

get_run_status

Get the status and details of a specific test run by its ID. Returns gate status (PASS/FAIL), totals, and per-case results.

start_operability_comparison

Run the same agent or hybrid scope twice, once normally and once accessibility-constrained (the agent’s browser tools may only target visible text or role and accessible name, never CSS selectors), and link the two runs. Takes the same scoping options as trigger_run, counts as two runs, and is refused when the scope has no agent or hybrid case. Returns a pairId.

get_operability_comparison

Compare the two runs from start_operability_comparison: state is WAITING until both finish, then COMPLETE, with per-case outcomes (SAME_PASS, SAME_FAIL, PASSED_WITH_INTERACTION_ERRORS, FAILED_ONLY_VARIANT, FAILED_ONLY_NORMAL, NOT_COMPARABLE), browser tool activity in each mode, and targets associated with failed click/type/select calls. A failed call can have causes other than accessibility; inspect the transcript. This is an operability test, not a WCAG conformance check: each mode is one run of an LLM agent, so a difference can also be variance.

build_conversation_checks

Experimental, and not reviewed by an accessibility specialist. Lists (mode: "catalog") or builds (mode: "build") configurable conversation checks for an AI agent under test: confirm before acting, accept a correction, ask instead of guessing, repeat on request, recover from a rejected interpretation, route to a person, and consistent choices. Every assertion can be switched off, every wording pattern replaced and every count set; the defaults reproduce the original behaviour, and anything switched off is recorded in the case description as not checked. Makes no API call: build returns cases whose steps you pass unchanged to create_agent_case. Not an accessibility conformance result; see the Conversation Checks page.

insert_agent_case_steps

Insert steps into an existing AI agent case without touching its other steps. Use it to add accessibility checks from build_conversation_checks (mode: "append") to a case you already have. Do not use update_case_steps on an agent case: it replaces every step and only understands UI and API steps. It refuses a case that is not an agent case, a position outside the case, and a check group the case already has. Added steps keep their markers (checkCategory, checkGroup, checkMode), so a step can be skipped (off), recorded without failing the case (report), or enforced. afterStep is the 1-based step to insert after (omit for the end); dryRun previews without saving. May be routed to a pull request if the project requires change review.

get_run_report

Retrieve the structured test execution report and error logs for a run. Supports summary (default), detailed json, junit xml, or ctrf format. Returns step-level failure messages, error logs, and execution traces for AI log evaluation.

get_project_runs

Get recent test runs for a project. Returns a list of runs with gate status, totals, and timestamps.

get_run_manifest

Get the run manifest for a project — a flattened list of all test cases with their quarantine flags. Useful to understand what will run and what’s excluded from gating.

quarantine_case

Quarantine or lift quarantine on a test case. Quarantined cases still execute but their failures don’t block the CI pipeline gate. Use this for known flaky tests or expected failures.

get_flake_stats

Get flake analytics — per-case flake scores sorted by severity. Helps identify unreliable tests that may need quarantine or fixing.

list_schedules

List all automated CI scheduled runs configured for a project, including cron expressions, target environment, and active status.

create_schedule

Create a recurring scheduled test run for a project. Triggers automatically via cron expression (e.g. “0 0 * * *” for daily midnight runs).

delete_schedule

Delete an existing scheduled test run by ID.

create_agent_case

Create an E2E AI Agent test case. Steps test conversational turns, expected tool calls, mock tool responses, and LLM judge rubric evaluations. Pass tags (for example @a11y-conversation) to tag the case for trigger_run filters and reports. ASSERT_OUTPUT (deterministic matchers on the agent’s last message, a tool’s arguments or a variable) and EXPECT_TOOL_CALL_COUNT (how many times a tool was called, optionally with given arguments) are available for checks that need no LLM judge.

run_agent_suite

Generate execution configuration and command to execute an AI Agent test suite against a target agent endpoint via the certiqa-agent-runner CLI.

generate_synthetic_data

Generate realistic synthetic test data (emails, names, phone numbers, UUIDs, dates, addresses, random values) for authoring automated test cases.

list_certiblocks

List all reusable CertiBlocks (reusable test sub-routines) in the platform or scoped to a project.

get_certiblock

Retrieve full details of a CertiBlock by ID, including its steps and parameterized variable schemas.

create_certiblock

Create a new reusable CertiBlock with optional parameterized input/output variables.

add_step_to_certiblock

Append an atomic test step to a reusable CertiBlock.

poll_tenant_events

Poll the tenant event queue for lightweight (<1KB) gate failure, flake spike, or run completion events. Returns event pointers for on-demand triage.

claim_tenant_event

Atomically claim an event from the tenant event queue to avoid racing with other autonomous agents in the pool.

resolve_tenant_event

Mark a claimed tenant event as RESOLVED with diagnostic resolution notes and root-cause analysis.

triage_run_failure

Fetch a run failure summary with per-case boolean flags (hasScreenshot, hasTrace, hasVideo, hasMessage). Always call this before requesting heavy media artifacts to keep LLM context minimal.

get_case_artifacts

Fetch on-demand presigned URLs for specific test case artifacts. Keeps agent context small by only fetching what is required.