Core concepts

Errors

Every error has the same shape and a stable code. Branch on the code; show the message.

Error shape

{
  "error": {
    "code": "INVALID_PARAM",
    "message": "ids[1] (\"12\") is not a TCIN: expected the numeric Target item number (TCIN), 6 to 10 digits, e.g. 83887639.",
    "param": "ids[1]",
    "docs": "/docs/errors#invalid_param"
  }
}

Field	Description
`code`	Stable and machine-readable. Branch on it.
`message`	For humans. May be reworded.
`param`	The parameter at fault, when there is one.
`docs`	A link to the code's entry below.

Coming-soon platforms, marketplaces and endpoints add "coming_soon": true to a 501.

Codes

Code	HTTP	Meaning
`MISSING_PARAM`	400	Missing parameter
`INVALID_PARAM`	400	Invalid parameter
`UNAUTHORIZED`	401	Unauthorized
`DEMO_READ_ONLY`	403	Demo workspace
`INSUFFICIENT_CREDITS`	402	Insufficient credits
`NOT_FOUND`	404	Not found
`GONE`	410	Gone
`CONFLICT`	409	Conflict
`CONCURRENCY_LIMIT`	429	Concurrency limit reached
`RATE_LIMITED`	429	Rate limited
`TARGET_FAILED`	502	Target failed
`UNAVAILABLE`	503	Temporarily unavailable
`NOT_IMPLEMENTED`	501	Not implemented
`INTERNAL`	500	Internal error

MISSING_PARAM

HTTP 400 · Missing parameter. A required parameter was not sent. param names it.

What to do

Add the parameter. JSON bodies need Content-Type: application/json.

HTTP 400 · Invalid parameter. A parameter was sent but its value is not accepted: wrong type, out of range, an unsupported marketplace, a private webhook URL, an entry in the id list that is not a valid id on that platform (param is then the list and its index, e.g. ids[i], asins[i] on Amazon), or two options that cannot be combined. param names it.

What to do

Read message: it says exactly what is expected.

UNAUTHORIZED

HTTP 401 · Unauthorized. No API key was sent, or the key is unknown or revoked.

What to do

Send your key as Authorization: Bearer <key>, X-API-Key: <key> or ?api_key=<key>. Keys are in the dashboard under Developers → API keys.

DEMO_READ_ONLY

HTTP 403 · Demo workspace. The key belongs to the public, read-only demo workspace. It can be seen in the dashboard but cannot make requests or spend credits.

What to do

Create an account (5,000 free credits) and use your own API key.

INSUFFICIENT_CREDITS

HTTP 402 · Insufficient credits. Your credit balance cannot cover this request. Nothing was charged. A call is all-or-nothing: every item in it must be affordable.

What to do

Buy a credit pack (credits never expire), or enable auto top-up in Billing.

NOT_FOUND

HTTP 404 · Not found. The endpoint, watchlist, webhook endpoint or structured endpoint does not exist, or belongs to another account.

What to do

Check the id and the path. The API reference lists every endpoint.

GONE

HTTP 410 · Gone. The endpoint was retired and will not come back. The generic URL scraper (/v1/scrape, /v1/jobs, /v1/batches) answers this: Scrapely serves retailer data only.

What to do

Use the product API, POST /v1/products, with a platform, a marketplace and item ids; watchlists (/v1/watchlists) run it on a schedule.

CONFLICT

HTTP 409 · Conflict. The request conflicts with the resource's current state: a request with the same Idempotency-Key is still in progress, a watchlist that is already running was asked to run again, or a per-account limit was reached.

What to do

Retry idempotency and run conflicts after a few seconds; for limits, delete something first.

CONCURRENCY_LIMIT

HTTP 429 · Concurrency limit reached. All of your concurrent request slots are busy. The request was refused immediately and nothing was charged.

What to do

Retry with backoff. Concurrency grows with lifetime credits purchased.

RATE_LIMITED

HTTP 429 · Rate limited. Too many API calls with this key in a short window. Separate from concurrency: this counts calls, including cheap ones such as listing watchlist changes.

What to do

Wait Retry-After seconds. Prefer webhooks to polling in a tight loop.

TARGET_FAILED

HTTP 502 · Target failed. We could not get a usable answer from the retailer after our retries: it blocked us, rate-limited us, errored, or never responded. You were not charged.

What to do

Retry with backoff. Items that fail inside a successful call come back unavailable and are refunded on their own.

UNAVAILABLE

HTTP 503 · Temporarily unavailable. A dependency on our side is down or overloaded. Nothing was charged.

What to do

Retry with exponential backoff. Status updates at /status.

NOT_IMPLEMENTED

HTTP 501 · Not implemented. The platform, marketplace, endpoint or option is announced but not available yet. It answers with "coming_soon": true (structured endpoints also with "message": "Coming soon"); nothing is charged.

What to do

Use a live platform and marketplace (GET /v1/platforms lists them) with POST /v1/products. Contact support if you need something sooner.

INTERNAL

HTTP 500 · Internal error. Something failed on our side. Nothing was charged, and the failure was logged with your X-Request-Id.

What to do

Retry. If it persists, contact support with the X-Request-Id header value.

Retrying

Retry with exponential backoff on CONCURRENCY_LIMIT, RATE_LIMITED (honour Retry-After), TARGET_FAILED, UNAVAILABLE and INTERNAL. Do not retry 400, 401, 402 or 404 without changing something.

Item-level failures are not HTTP errors: retry unavailable items in your next call. For creates, send an Idempotency-Key so a retry can never double-charge.

import os
import time
import requests

HEADERS = {"Authorization": f"Bearer {os.environ['SCRAPELY_API_KEY']}"}
RETRYABLE = {"CONCURRENCY_LIMIT", "RATE_LIMITED", "TARGET_FAILED", "UNAVAILABLE", "INTERNAL"}

def call(method, path, attempts=5, **kwargs):
    for i in range(attempts):
        r = requests.request(method, f"https://api.scrapelyserp.com/v1{path}", headers=HEADERS, timeout=300, **kwargs)
        if r.ok:
            return r.json()
        code = r.json()["error"]["code"]
        if code not in RETRYABLE or i == attempts - 1:
            raise RuntimeError(code)
        time.sleep(float(r.headers.get("Retry-After", 2 ** i)))

data = call("POST", "/products", json={"platform": "walmart", "ids": ["5129928603"]})