5.3 GitLab CI/CD Pipeline Template & JUnit Artifacts

How to configure GitLab CI pipelines to run CertiQA suites and publish JUnit test reports.

Turning a CI Stage Into a Release Gate

A GitLab pipeline is only as strict as the exit code of its slowest job, so the CertiQA stage below is written as an ordinary script: step – GitLab doesn’t need a special concept of a “quality gate” any more than GitHub does; a non-zero exit on this job simply fails the pipeline, which in turn blocks a merge request from showing a green check, or blocks the merge outright if you’ve turned on “Pipelines must succeed” in the project’s merge request settings.

test:certiqa:
  stage: test
  image: alpine:latest
  before_script:
    - apk add --no-cache curl jq bash
  variables:
    CERTIQA_API_URL: "https://api.certiqa.app"
    CERTIQA_PROJECT_ID: "$CERTIQA_PROJECT_ID"
    CERTIQA_SUITE_ID: "$CERTIQA_SUITE_ID"
    CERTIQA_TOKEN: "$CERTIQA_API_TOKEN"
  script:
    - ./certiqa-run.sh
  artifacts:
    when: always
    reports:
      junit: certiqa-junit.xml
    paths:
      - certiqa-junit.xml
    expire_in: 30 days

The reports: junit: key under artifacts: is doing more work than a plain paths: upload would – GitLab parses that file and renders it directly inside the merge request’s Test Summary widget, so a reviewer sees which specific test cases failed without leaving the MR page or downloading anything. when: always matters just as much as the report key itself: the default on_success skips artifact collection entirely on a failed job, which is exactly the run you actually want evidence from. Pair that with expire_in sized to how long your team actually needs to look back at a run – GitLab deletes expired artifacts automatically, and a pipeline that never sets this keeps every report forever, quietly consuming your project’s storage quota.

Where the Variables Come From

CERTIQA_PROJECT_ID, CERTIQA_SUITE_ID, and CERTIQA_API_TOKEN are defined once under Settings → CI/CD → Variables at the project (or group, for shared credentials across repos) level, not hard-coded into the YAML. Mark CERTIQA_API_TOKEN both Protected and Masked: Protected restricts it to pipelines running on protected branches or tags so a feature-branch pipeline from a fork can’t read it, and Masked keeps GitLab from ever printing its value in job logs, even if a script accidentally echoes it.

Exit Code	What GitLab Shows	What It Means
0	Job passes, stage green	Every non-quarantined test passed.
1	Job fails, pipeline red, MR check blocked	A real regression – treat it as a merge blocker.
2	Job fails; pipeline red	The run timed out waiting for results (default 30 minutes). Retry the job and check the run in the app.
3	Job fails with no useful test evidence in the JUnit report	The runner itself couldn’t start – bad project/suite ID, unreachable API URL, or an expired token.

GitLab Doesn’t Know About Exit Codes 2 and 3 Either

• Like a plain GitHub Actions step, a script: block treats any non-zero exit as a job failure. A timeout (exit 2) or an infrastructure error (exit 3) is not a test regression, so configure a retry for those exit codes using your CI system’s retry mechanism. Do not remap exit code 2 to 0: it means the run did not finish, and doing so turns an unfinished run green. Quarantined failures need no special handling, because they do not fail the gate.

• 🐧 CLI Runner & Headless Execution — The exit-code contract (0/1/2/3) and BYOC networking model this pipeline job relies on.
• 🐙 GitHub Actions Quality Gate Workflow — The same runner and exit-code story, wired into a GitHub Actions workflow instead.
• 🩺 How to Quarantine Flaky Tests — How a quarantined test is kept from failing the gate.