Run Oobee in CI/CD

What this guide covers

The Oobee CLI can run unattended in a continuous integration and continuous delivery (CI/CD) pipeline. This guide shows how to run a scan from the Oobee Docker image in a GitLab CI/CD job and keep the reports as job artefacts. The CLI writes reports but does not fail a job on the issues it finds. To fail a build on issue counts, refer to Gate a CI build on accessibility thresholds.

Prerequisites

A GitLab project with CI/CD runners that can run Docker images.
The URL of a site or sitemap that the runner can reach and that you are allowed to scan.
The Oobee Docker image, either from GitHub Container Registry or built from the Oobee Dockerfile.
A name and email address for the -k option, or OOBEE_DISABLE_TELEMETRY=1 in the job.

Step 1: Get the Oobee image

The Oobee image is based on the Microsoft Playwright image and includes Google Chrome. It runs as a non-root user and keeps the application in /app/oobee. Use the published image:

docker pull ghcr.io/govtechsg/oobee:latest

To use your own copy, build the image from the repository Dockerfile and push it to your GitLab container registry. Set DOCKER_IMAGE in the next step to that image.

Step 2: Add the scan job

Add a job to .gitlab-ci.yml. The job starts from the Oobee image, runs one scan and moves the newest results folder to an artifacts folder.

variables:
  DOCKER_IMAGE: ghcr.io/govtechsg/oobee:latest
  OOBEE_SCAN_URL: 'https://example.com'
  OOBEE_SCAN_TYPE: 'intelligent'
  OOBEE_SCAN_DEVICE: 'Desktop'
  OOBEE_SCAN_MAX_NUM_PAGES: 200
  OOBEE_MAX_CONCURRENT_PAGES: 5
  OOBEE_INCLUDE_SCREENSHOTS: 'none'
  OOBEE_SCAN_SAFE_MODE: 'yes'
  OOBEE_SCAN_ARTIFACT_NAME: 'oobee-scan-results.zip'
  OOBEE_SCAN_NAME_EMAIL: '<YOUR_NAME>:<YOUR_EMAIL>'

oobee-scan:
  stage: test
  image:
    name: $DOCKER_IMAGE
    entrypoint: ['/bin/bash', '-lc']
  script:
    - cd /app/oobee
    - npm run cli -- -b chromium -c "$OOBEE_SCAN_TYPE" -d "$OOBEE_SCAN_DEVICE" -o "$CI_PROJECT_DIR/$OOBEE_SCAN_ARTIFACT_NAME" -u "$OOBEE_SCAN_URL" -p "$OOBEE_SCAN_MAX_NUM_PAGES" -t "$OOBEE_MAX_CONCURRENT_PAGES" -a "$OOBEE_INCLUDE_SCREENSHOTS" -f "$OOBEE_SCAN_SAFE_MODE" -k "$OOBEE_SCAN_NAME_EMAIL" || true
    - results_directory=$(find results -mindepth 1 -maxdepth 1 -type d -print -quit)
    - if [ -n "$results_directory" ]; then mv "$results_directory" "$CI_PROJECT_DIR/artifacts"; fi

The || true at the end of the scan command keeps the job passing when a scan ends with an error, so the later lines can still collect partial results. Remove it if you want a failed scan to fail the job.

Step 3: Tune the scan for CI

Set these variables to suit the runner and the site:

Variable	Effect
`OOBEE_SCAN_TYPE`	`sitemap`, `website` or `intelligent`. Use `intelligent` for the fullest coverage.
`OOBEE_SCAN_MAX_NUM_PAGES`	Maximum pages scanned. Maps to `-p`.
`OOBEE_MAX_CONCURRENT_PAGES`	Pages scanned at once. Maps to `-t`. Lower it to `1` if the site blocks the scan with 403 errors.
`OOBEE_INCLUDE_SCREENSHOTS`	`none` leaves out element screenshots and speeds up the scan. `screenshots` includes them.
`OOBEE_SCAN_SAFE_MODE`	`yes` stops Oobee clicking buttons and links to find more links. Maps to `-f`.

For runner sizing and expected duration, refer to How Oobee scans.

Step 4: Keep the reports as artefacts

Add an artifacts block to the job to keep the reports. Keeping only the summary files saves storage.

  artifacts:
    paths:
      - artifacts/report.csv
      - artifacts/report.html
      - artifacts/scanDetails.csv
      - artifacts/summary.pdf

To keep element screenshots, add artifacts/elemScreenshots/. To keep the whole results folder, add artifacts/. To keep the zip file, add the value of OOBEE_SCAN_ARTIFACT_NAME. Refer to Scan report files for what each file holds.

Constraints and limitations

Constraint	Detail
No build failure on issues	The CLI does not return a failing exit code because of accessibility issues. Use the npm module with `thresholds` to fail a build.
Browser	The image includes Google Chrome and Playwright. The example sets `-b chromium` to use Playwright’s Chromium.
Scan blocked by the site	Sites that rate-limit or block automated requests return 403 errors. Lower concurrency and page limits, or ask the site owner to allow the runner.
Storage	Large scans produce large reports. A 5,000-page scan needs at least 20 GB of free disk space.
Memory	The CLI starts Node.js with a 10,000 MB heap limit (`--max-old-space-size=10000`). Give the job enough memory.

Run a scan — scan types and options.
Upload results to Amazon S3 — store results outside the pipeline.
Configure a proxy — scan from runners behind a proxy.
CLI options, environment variables and error codes — every option and default.

Scan PDFs and local files

CLI options, environment variables and error codes