Fetch API

Fetch a public web page by URL and read its returned content with SpideyData's Fetch API. Includes authentication, a curl example and response fields.

On this page

Fetch API endpoints

Find the task you need. Each endpoint includes its parameters, request example, and response reference.

Fetch web page content by URL

Submit one public URL and read the returned page content using the Fetch endpoint.

Quick start

Send one URL; Fetch starts with the configured identity/proxy path and can retry direct retrieval after proxy or network transport failure.

API base https://kieapi.com

curl -X POST "https://kieapi.com/v1/fetch" \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Idempotency-Key: fetch-example-01" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com"
  }'

When a web data request needs a browser — Work through content acceptance, explicit browser escalation, and result provenance.

Map the data exits before solving the CAPTCHA — Identify the data source and available collection path before adding browser work.

Authentication

Use a bearer token.

Team administrators manage API keys in Team Settings. Each key keeps its Team and billing scope when its creator leaves or switches Teams. Keys cannot manage members, billing or other keys. Public /v1 requests use Authorization: Bearer $SPIDER_API_KEY only; the Key's fixed Team is the payer. An optional X-Spider-Team-Id must match that Team or the request is denied. Creating, listing, rotating or revoking Keys through the product console (and other non-identity console business routes) requires X-Spider-Team-Id when the Team model is enabled; omitting it returns HTTP 409 team_context_required.

Response shape

Content

Delivered page outputs: 1 Credit/page, shadow by default. Read /v1/runs/{run_id}/billing for actual charged Credits; cost_units are diagnostics.

Metadata

Final URL, title, status, canonical URL, and page metadata.

Audit fields

Engine attempt, quality, fallback, and cost fields for explainability. body_size_limit and redirect_limit are non-retryable guard outcomes. The current server runtime default is 10 MiB (10485760 bytes), not caller-configurable in public v1; the response body and internal transport error text are never exposed.

Diagnostics

Use result.diagnostics.adaptive_extraction for selector memory / adaptive extraction diagnostics, result.diagnostics.browser_debug for sanitized browser debug capture counts, result.diagnostics.browser_failure for sanitized browser failure category/stage diagnostics, and result.diagnostics.content_quality for content quality diagnostics.