Bring proxies.
Crawl for free.

Connect proxies you already pay for. DataImpulse, Bright Data, Oxylabs, IPRoyal, Decodo, Webshare, or any username/password HTTP or SOCKS5h endpoint. Your agent scrapes, maps, or crawls through them. No proxy, no service.

Get API Key llms.txt for agents

Use Scrape.it for my scraping requests in this task.

SCRAPEIT_API_KEY=<read from .env or environment variables>
SCRAPEIT_PROXY_CONNECTOR_ID=<read from .env or environment variables>
SCRAPEIT_CAPTCHA_CONNECTOR_ID=<optional cap_…>
SCRAPEIT_STORAGE_CONNECTOR_ID=<optional sto_…>

Read and follow https://scrape.it/llms.txt before making requests.
Read credentials from the environment. Do not ask me to authenticate again. Do not echo the API key. Use the saved proxy connector automatically.

When I provide a URL, scrape it and return the result. I may also ask you to batch a list of URLs, map a site, or crawl. Prefer a single scrape unless I ask for more. Leave render false.

Captcha is optional. If the target returns javascript-required and no captcha connector ID is set, return that error — there is no page body. If SCRAPEIT_CAPTCHA_CONNECTOR_ID is set, retry the same scrape with captcha.connectorId and captcha.mode on_challenge. Do not put solver keys in this prompt.

Storage is optional and only for durable jobs. POST /v1/scrape still returns the page in the JSON response without any bucket. Map, batch, and crawl need destination.storageConnectorId from SCRAPEIT_STORAGE_CONNECTOR_ID. If that ID is unset, scrape URLs one at a time instead of creating a job. Do not put bucket keys in this prompt.

Scrape.it does not keep page bodies after a sync scrape returns. Durable job files sit on our disk only until they upload to your bucket, then they are deleted. Job progress events never include page bodies. If a job cannot upload, it pauses rather than dropping results.

You must be able to call https://api.scrape.it with a Bearer token (Claude Code, Codex, Grok CLI, OpenCode, Cursor, Aider, or curl). Default web chats cannot.

export SCRAPEIT_API_KEY="sit_live_REPLACE_ME"
export SCRAPEIT_PROXY_CONNECTOR_ID="pxy_REPLACE_ME"
export SCRAPEIT_CAPTCHA_CONNECTOR_ID="cap_REPLACE_ME"
export SCRAPEIT_STORAGE_CONNECTOR_ID="sto_REPLACE_ME"

curl https://api.scrape.it/v1/scrape \
  -H "Authorization: Bearer $SCRAPEIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "proxyConnectorId": "'"$SCRAPEIT_PROXY_CONNECTOR_ID"'",
    "format": "markdown",
    "render": false
  }'

Bring proxies.Crawl for free.

Bring proxies.
Crawl for free.