Docs
Everything you need to start scraping with DreamScrape.
Quickstart
Get your first scrape in 30 seconds.
1. Get an API key
Contact us or use the demo key on the homepage to test.
2. Make your first request
curl
curl -X POST https://api.dreamscrape.app/scrape \
-H "Authorization: Bearer ds_pk_your_key" \
-H "Content-Type: application/json" \
-d '{"url": "https://news.ycombinator.com", "renderMode": "auto"}'Node.js
const res = await fetch('https://api.dreamscrape.app/scrape', {
method: 'POST',
headers: {
'Authorization': 'Bearer ds_pk_your_key',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: 'https://news.ycombinator.com',
renderMode: 'auto',
}),
});
const data = await res.json();
console.log(data.markdown); // Clean markdown
console.log(data.engineTier); // "http", "stealth-playwright", etc.
console.log(data.timing.fetchMs); // Latency in msPython
import requests
res = requests.post('https://api.dreamscrape.app/scrape',
headers={'Authorization': 'Bearer ds_pk_your_key'},
json={'url': 'https://news.ycombinator.com', 'renderMode': 'auto'})
data = res.json()
print(data['markdown']) # Clean markdown
print(data['engineTier']) # Which engine tier was used
print(data['timing']['fetchMs']) # Latency3. Read the response
Every response includes:
- •
markdown— Clean extracted content - •
engineTier— Which engine served the request ("http", "stealth-playwright", "camoufox") - •
timing.fetchMs— Request latency in milliseconds - •
metadata— Page title, description, language - •
links— All extracted links from the page
API Reference
POST /scrape
Scrape a URL and get clean markdown + metadata.
| Parameter | Type | Default | Description |
|---|---|---|---|
| url | string | (required) | URL to scrape |
| renderMode | "auto" | "http" | "browser" | "auto" | How to fetch the page |
| engine | "auto" | "stealth-playwright" | "camoufox" | — | Force a specific engine |
| timeout | number | 30000 | Request timeout in ms |
| async | boolean | false | Return 202 + job ID for polling |
| impersonateBrowser | "chrome" | "firefox" | — | TLS fingerprint impersonation |
| tlsFingerprint | "ja3" | "ja4" | — | JA3 (legacy) or JA4 (recommended) |
| acceptContentType | "html" | "json" | "any" | "html" | Accept JSON API responses |
| solveCaptcha | boolean | false | Auto-solve CAPTCHAs |
| useProxy | boolean | false | Use residential proxy |
| headers | object | — | Custom request headers |
| extract | object | — | Structured extraction schema |
GET /jobs/:id
Poll for async scrape results (when async: true).
GET /usage
Get your current month's usage by engine tier.
GET /health
Check API status, engine availability, and dependency health.
Error Taxonomy
Every error includes a blockReason that tells you exactly why it failed and a suggestion with what to try next. No more guessing at 403s.
ipDatacenter IP blocked — needs residential proxyFix: Use useProxy: true with a residential proxy configured
stealthFingerprint detected — needs browser engineFix: Try engine: "stealth-playwright" or "camoufox"
captchaCAPTCHA challenge detectedFix: Use engine: "camoufox" with solveCaptcha: true
authSite requires authenticationFix: Pass auth cookies/headers via the headers option
rate_limitTarget site is throttling requestsFix: Reduce request frequency or use proxy rotation
timeoutRequest timed outFix: Increase timeout or try a browser engine for JS-heavy sites
robots_txtDisallowed by robots.txtFix: Check the target's robots.txt and respect their rules
geoContent is geo-restrictedFix: Use a proxy in the target region
paywallContent behind a paywallFix: Pass subscription auth headers
not_found404 — page does not existFix: Verify the URL is correct
unknownUnclassified failureFix: Check the error message for details and contact support
Engine Tiers
DreamScrape automatically picks the cheapest engine that works for each domain. Every response includes engineTier so you always know which one was used.
Plain fetch with JA4 TLS fingerprint. Handles 70%+ of sites.
Headless Chromium with fingerprint injection. For JS-heavy SPAs.
Patched Firefox with C++ fingerprinting. For the hardest targets.
Changelog
2026-04-10
Launch
- • API keys with per-key rate limits, quotas, and engine ACLs
- • 5 engine tiers: HTTP, JA4, StealthPlaywright, Camoufox, Browserless
- • Async scrape via BullMQ (POST with async: true)
- • 11-value error taxonomy with actionable suggestions
- • CAPTCHA detection as hard failure (no silent garbage)
- • Live scorecard at dreamscrape.app
- • Sports + crypto data packages
Questions? Email us — we reply fast.