Crawler

API Status

Loading status...

Crawl a Webpage

Crawl Result

Copied!

API Reference

Base URL: /v1. Send your key in the X-API-KEY header. Endpoints tagged ADMIN require the admin key instead. Request/response bodies are JSON unless noted.

Browser

POST /v1/browser/crawl API Key

Render a page and extract content (text, markdown, html, metadata, jsonLd, socials, contacts, screenshot, pdf).

curl -X POST http://localhost:3000/v1/browser/crawl \
  -H "X-API-KEY: your-key" -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "options": {
      "extract": {
        "text": true,
        "markdown": { "raw": false },
        "metadata": true
      }
    }
  }'
POST /v1/browser/evaluate API Key

Run a custom JavaScript snippet in the page context and return its result. Body: { url, script, options? }.

POST /v1/browser/tags API Key

Return the sorted list of unique HTML tag names present on the page. Body: { url, options? }.

Batch

POST /v1/batch API Key

Queue a crawl across many URLs. Returns 202 with a jobId; results are delivered to webhookUrl if provided. Body: { urls: string[], options?, webhookUrl? }.

Jobs

GET /v1/jobs API Key

List recent jobs. Query: ?limit= (default 20, max 100).

GET /v1/jobs/:id API Key

Get the status and results of a single job by numeric id.

PDF

POST /v1/pdf/extract API Key

Extract text from an uploaded PDF. multipart/form-data with a pdf file field (max 10 MB).

POST /v1/pdf/markdown API Key

Convert an uploaded PDF to Markdown. multipart/form-data with a pdf file field (max 10 MB).

Admin

POST /v1/admin/keys Admin

Create a new API key. Body: { name }. The full key is returned once — save it.

GET /v1/admin/keys Admin

List all API keys (prefixes and metadata only).

DELETE /v1/admin/keys/:id Admin

Revoke and delete an API key.

POST /v1/admin/keys/:id/regenerate Admin

Regenerate the secret for an existing key. Returns the new key once.

GET /v1/admin/usage Admin

Aggregate usage statistics across all keys.

GET /v1/admin/history Admin

Recent request history across all keys.

GET /v1/admin/keys/:id/usage Admin

Usage statistics for a single key.

GET /v1/admin/keys/:id/history Admin

Request history for a single key.

POST /v1/admin/purge Admin

Purge stored usage/history data.

Health & Monitoring

GET /v1/healthz Public

Liveness probe. Returns { "status": "ok" }.

GET /v1/status Public

Service status and timestamp (secrets masked).

GET /v1/metrics Public

Prometheus metrics exposition.