# Web Scraping Language (WSL) — Scrape.it legacy

Scrape.it introduced WSL as a one-line DSL for crawl and extract jobs. The public editor lived at `/edit/...` and `/wsl`. npm packages `wslang` and `webscrapinglang-cli` pointed here.

WSL is not accepted by the current API. There is no WSL runtime, no visual editor, and no plan to rebuild either. This page exists so old Medium, npm, Stack Overflow, and `/edit` links land on a truthful note instead of a 404.

## What WSL looked like

Actions ran left to right, separated by `>>`. Selectors were CSS or XPath. Extracts were JSON maps of column name to selector. These examples are copied from the published 2019 `wslang` README.

```
GOTO github.com/search?p=[1-3]&q=[cms, chess, minecraft] >> EXTRACT {title: h3}
```

```
GOTO en.wikipedia.org/wiki/List_of_Dexter_episodes >> CRAWL .summary a >> EXTRACT {title: h1, code: //tr[7]/td}
```

```
GOTO news.ycombinator.com >> CRAWL [.morelink,4] .hnuser
```

```
GOTO en.wikipedia.org/wiki/List_of_Dexter_episodes >> EXTRACT { title: h1, aired: //table[2]//tr[2]/td[5] } _IN_ .wikiepisodetable
```

```
GOTO github.com/search?q= >> TYPE input[@name="q"] ["time", "security", "social"] >> TYPE ["KEY_ENTER"] >> EXTRACT {"search url": ".text-gray-dark.mr-2"}
```

## Equivalent work today

Bring an authenticated proxy you already pay for. Scrape.it fetches. You keep the route.

| Old WSL idea | Current API |
|---|---|
| `GOTO url >> EXTRACT {...}` | `POST /v1/scrape` with `url` and a saved `proxyConnectorId` |
| `CRAWL selector` over one site | `POST /v1/map` then `POST /v1/batch/scrape`, or `POST /v1/crawl` |
| Paginated `EXTRACT [.morelink, n]` | A bounded crawl or an explicit URL list |
| Cloud workers / rotating IPs from us | Your proxy. We do not sell or pool proxies |
| Visual `/edit` playground | `https://api.scrape.it/auth/start` then curl or an agent with `llms.txt` |

```bash
export SCRAPEIT_API_KEY="sit_live_REPLACE_ME"
export SCRAPEIT_PROXY_CONNECTOR_ID="pxy_REPLACE_ME"

curl https://api.scrape.it/v1/scrape \
  -H "Authorization: Bearer $SCRAPEIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","proxyConnectorId":"pxy_REPLACE_ME","format":"markdown","render":false}'
```

Start at https://api.scrape.it/auth/start. Contract: https://scrape.it/openapi.json. Proxy paste formats: https://scrape.it/connectors.md.

The old Show HN is https://news.ycombinator.com/item?id=8564250. `/tour2` from that thread now redirects to the homepage.