Documentation

Getting Started

Install, write your first test, and run it locally in a few minutes.

This page takes you from a blank project to a passing agent test in a few minutes. Once it runs, branch into Writing Tests for depth on the DSL.

Prerequisites

  • Node.js 20+ (or the Bun runtime).
  • An AI provider API key. Blop's agent loop speaks the OpenAI chat-completions protocol, so OpenRouter, OpenAI, Groq, xAI, Mistral, Cerebras, and NVIDIA all work directly; Anthropic and Google route through OpenRouter.
  • A target app running locally or at a reachable URL.

Install

npm install --save-dev @blopai/cli
# or: pnpm add -D @blopai/cli
# or: bun add -d @blopai/cli

The package ships a blop binary. You can also run it without installing:

npx @blopai/cli init

Point the agent at a model provider (OpenRouter is the default):

export BLOP_AGENT_PROVIDER=openrouter
export BLOP_AGENT_MODEL=anthropic/claude-sonnet-4-6
export BLOP_AGENT_API_KEY=sk-or-...

Provider-native variables (OPENAI_API_KEY, GROQ_API_KEY, …) also work. See the CLI reference for the full list.

Scaffold a test

npx @blopai/cli init

This writes tests/homepage.blop.ts:

import { agentTest, describe } from "@blopai/cli";

describe("homepage", () => {
  agentTest("loads", async ({ agent }) => {
    await agent.goto("/");
    await agent.goal("Verify the homepage loads and the primary user action is visible.");
  });
});

Run it

npx @blopai/cli test --base-url http://localhost:3000

What happens:

  1. Blop discovers every **/*.blop.ts spec under the current directory.
  2. It launches a Playwright browser (Chromium by default).
  3. The agent receives your goal and a curated set of browser tools.
  4. The agent navigates, inspects, asserts, and finally calls finish_test.
  5. Blop writes a structured run result to .blop/.

On a fresh machine you may need the browser binaries once:

npx playwright install chromium

Output

After a run, the report directory (default .blop/) contains:

File Description
.blop/results.json Structured BlopRunResult with every test outcome
.blop/events.jsonl One agent event per line (useful for debugging)
.blop/report.xml JUnit-compatible XML for CI consumers
.blop/screenshots/ Screenshots captured during the run

Configure once

Drop a blop.config.ts at your project root to stop passing flags every run:

import type { BlopConfig } from "@blopai/cli";

export default {
  baseUrl: "http://localhost:3000",
  browser: "chromium",
  reporter: "all",
  maxSteps: 100,
  include: ["tests/**/*.blop.ts"],
} satisfies BlopConfig;

CLI flags always override config values. Full options live in Configuration.

Watch mode

While iterating on a spec:

npx @blopai/cli watch tests/

Reruns affected tests on save.

Next steps