CI Integration

Blop is designed CI-first. The default reporter writes JUnit XML and a JSON result that any pipeline can pick up, and blop test exits non-zero when a test fails, so branch protection works with no extra wiring.

GitHub Actions (recommended)

# .github/workflows/blop.yml
name: Blop E2E
on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - run: npm ci
      - run: npx playwright install --with-deps chromium

      - name: Run Blop
        run: npx @blopai/cli test --base-url ${{ vars.APP_URL }} --reporter all
        env:
          BLOP_AGENT_PROVIDER: openrouter
          BLOP_AGENT_MODEL: anthropic/claude-sonnet-4-6
          BLOP_AGENT_API_KEY: ${{ secrets.BLOP_AGENT_API_KEY }}

      - name: Upload artifacts
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: blop-results
          path: .blop/

      - name: Publish JUnit report
        if: always()
        uses: dorny/test-reporter@v1
        with:
          name: Blop tests
          path: .blop/report.xml
          reporter: java-junit

Key points:

• Install browsers explicitly. npx playwright install is required on fresh runners.
• Use repository secrets for API keys. Never inline them.
• if: always() on artifact upload. You want the bundle even when tests fail; that is when it matters most.
• Pin the model. Floating model versions can change agent behavior between runs.

CI metadata captured automatically

On GitHub Actions, Blop populates BlopCiMetadata on every test result:

{
  "provider": "github-actions",
  "runId": "1234567890",
  "jobId": "test",
  "branch": "feat/checkout-flow",
  "commitSha": "deadbeef...",
  "pullRequest": "42"
}

This is included in results.json. Auto-detection currently covers GitHub Actions only; on other providers BlopCiMetadata.provider is null.

Other CI providers

The same shape works on any provider that supports JUnit ingestion:

• GitLab CI: artifacts.reports.junit: .blop/report.xml
• CircleCI: store_test_results: path: .blop
• Buildkite: buildkite-agent artifact upload .blop/**/*

Parallelization

Blop runs tests serially within a single process so the agent keeps predictable control of the browser. To parallelize, shard at the workflow level:

strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: npx @blopai/cli test "tests/shard-${{ matrix.shard }}/**/*.blop.ts" --report-dir .blop/${{ matrix.shard }}

Pair with actions/upload-artifact and merge the JUnit reports downstream.

Cost control

Each test costs up to maxSteps model calls. To keep CI bills sane:

• Pin a --max-steps budget that is tight but realistic.
• Run the full suite on main; on PRs, run a smoke subset.
• Cache ~/.cache/ms-playwright to skip re-downloading browsers.