Fetch API

Fetch a single public URL through the lightweight public v1 API.

Quick start

Send one URL; Fetch starts with the configured identity/proxy path and can retry direct retrieval after proxy or network transport failure.

curl -X POST "$SPIDER_API_BASE/v1/fetch" \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Idempotency-Key: fetch-example-01" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com"
  }'

Authentication

Use a bearer token.

Create and manage product API keys in Product Console settings. The docs page itself is public.

Response shape

Content

Normalized HTML and markdown outputs for the fetched page.

Metadata

Final URL, title, status, canonical URL, and page metadata.

Audit fields

Engine attempt, quality, fallback, and cost fields for explainability. body_size_limit and redirect_limit are non-retryable guard outcomes. The current server runtime default is 10 MiB (10485760 bytes), not caller-configurable in public v1; the response body and internal transport error text are never exposed.

Diagnostics

Use result.diagnostics.adaptive_extraction for selector memory / adaptive extraction diagnostics, result.diagnostics.browser_debug for sanitized browser debug capture counts, result.diagnostics.browser_failure for sanitized browser failure category/stage diagnostics, and result.diagnostics.content_quality for content quality diagnostics.

Reference

Endpoint reference

OpenAPI-backed details for request bodies, responses, errors, and field types.

OpenAPI endpoint details

Interactive schemas, request bodies, response objects, and public error contracts load here.