Automation

Watchlists & pipelines

Hand us up to 50,000 item ids and a schedule. We read them on every run, compare with the last read, and push what changed to your webhook.

Overview

A watchlist is a list of item ids on one platform and marketplace, read on a schedule. Each run compares what it reads with the previous run and sends signed webhook events for what moved.

Out of the box a list runs its platform's default pipeline, the cheapest one that catches every change. Send your own pipeline to decide which reads run, what a change or a match leads to, and what reaches you.

Events are signed with the receiving endpoint's secret, so webhook_url must be one of your registered endpoints. The same data is always available to pull with GET /v1/watchlists/{id}/items.

Create a watchlist

curl -X POST "https://api.scrapelyserp.com/v1/watchlists" \
  -H "Authorization: Bearer $SCRAPELY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "name": "Walmart MAP",
    "platform": "walmart",
    "ids": ["5129928603", "931920073"],
    "every": "15m",
    "webhook_url": "https://example.com/hooks/scrapely"
  }'

Field	Required	Description
`name`	yes	Up to 100 characters.
`platform`	no	Default `amazon`. Any platform with live products: `amazon`, `walmart`, `target`, `bestbuy`.
`marketplace`	no	Default `us`. Must be live for products (a coming-soon one is a `501`).
`ids`	yes	1 to 50,000 item ids of the platform. Duplicates are dropped. A comma- or newline-separated string also works. Amazon also accepts `asins`.
`every`	one of	A fixed cadence, start to start: `5m`, `15m`, `1h`, `6h`, `24h`, or any whole number of minutes, hours or days from 5 minutes to 7 days. See Schedules.
`cron`	one of	A 5-field cron expression, instead of `every`. See Schedules.
`timezone`	no	With `cron`: the IANA time zone it runs in. Default `UTC`.
`webhook_url`	no	One of your registered webhook endpoints, by URL: this list's events go there only. Omitted: every endpoint subscribed to the event.
`webhook_endpoint_id`	no	The same, by endpoint id (`we_…`). Send one or the other.
`pipeline`	no	What each run does. See Pipelines. Omitted: the platform default.
`refresh_every_runs`	no	Amazon default pipeline only: every Nth run reads every item at `lite`. Default 8; `0` turns it off; at most 1,000.

Creating, editing and deleting lists is free. Up to 100 lists per account. The first run starts right away and records a baseline; it reports no changes.

Schedules

Send exactly one schedule: every for a fixed cadence, or cron (with an optional timezone) to run at set times. Sending both, or timezone with every, is a 400.

Schedule	Example	Runs
`every`	`"every": "15m"`	Every 15 minutes, start to start.
`cron` + `timezone`	`"cron": "0 8 * * 1-5", "timezone": "America/New_York"`	At 08:00 New York time, Monday to Friday.
`cron` + `timezone`	`"cron": "/30 * * *", "timezone": "UTC"`	Every 30 minutes, on the hour and half hour.

cron has five fields: minute, hour, day of month, month, day of week. Runs must be at least 5 minutes apart.
timezone is an IANA name such as Europe/London; the cron fields are read in that zone. Default UTC.
A list answers its schedule as schedule: { "type": "every", "every": "15m", "every_seconds": 900 } or { "type": "cron", "cron": "0 8 * * 1-5", "timezone": "America/New_York" }, plus next_run_at. On a cron list, every is null.
PATCH with every or cron to switch between the two; on a cron list, timezone alone changes the zone.

Default pipeline

On Walmart, Target and Best Buy, every run reads each active item once at standard and reports the tracked fields that moved. Amazon's default is cheaper for large lists:

New items: one full read, as the baseline.
Every run: a pulse scan of every item (offer count and lowest new price, the cheapest tier).
On a change: a lite read of the items whose scan moved, plus those whose last buy box was above the lowest offer (the scan cannot see that buy box move). Changes found there are reported.
Every 8th run (refresh_every_runs): a lite read of every item instead of the scan. It catches a buy-box seller change at the same price; you never pay for both on one run.

Item status	What happens
`ok`	New snapshot. Any tracked field that moved is a change.
`not_found`	The item is delisted: reported once, kept on the list as `delisted`, and not read again. Add it again to give it another chance.
`partial`, `unavailable`	Nothing is charged or reported; the item is retried on the next run.

Tracked fields

Platform	A change in any of these is reported
Amazon	`price`, `inStock`, `availabilityType`, `isFba`, `offerListingId`, `merchant`, `condition`, `newCount`, `newMinPrice`, `usedCount`, `usedMinPrice`
Walmart	`price`, `listPrice`, `inStock`, `seller`, `byPlatform`, `thirdPartySeller`, `offerId`, `clearance`
Target	`price`, `regularPrice`, `inStock`, `shipping`, `pickup`
Best Buy	`price`, `inStock`, `state`, `regularPrice`, `purchasable`

Prices within half a cent are equal. Amazon compares merchant by seller id.

Pipelines

A pipeline is up to 10 steps. Each step runs one scraper over the items sent to it and compares what it reads with the last read of the same kind.

Each step has up to 8 rules. A rule picks items by outcome (optionally one field, or a value test) and sends them to another step (run), to your webhook (notify), or both. Chains cannot loop.

Price drops (Target): notify only when the price goes down

{
  "entry": "check",
  "baseline": null,
  "refresh": null,
  "steps": [
    {
      "id": "check",
      "scraper": "target.standard",
      "routes": [
        {
          "when": "changed",
          "field": "price",
          "op": "decreased",
          "notify": true
        }
      ]
    }
  ]
}

Deal hunter (Amazon): a cheap scan, every seller when the lowest price drops

{
  "entry": "scan",
  "baseline": null,
  "refresh": null,
  "steps": [
    {
      "id": "scan",
      "scraper": "amazon.pulse",
      "routes": [
        {
          "when": "changed",
          "field": "newMinPrice",
          "op": "decreased",
          "run": "sellers"
        }
      ]
    },
    {
      "id": "sellers",
      "scraper": "amazon.offers",
      "routes": [
        {
          "when": "always",
          "notify": true
        }
      ]
    }
  ]
}

Scrapers

Scraper	Reads	Credits per item	Fields rules can test
`amazon.pulse`	Offer count and lowest new price. The cheapest way to notice that something moved.	0.25	`newCount`, `newMinPrice`
`amazon.lite`	Buy box, stock, seller, offer counts, rating and delivery dates.	1	`price`, `listPrice`, `inStock`, `availabilityType`, `merchant`, `isFba`, `offerListingId`, `condition`, `newCount`, `newMinPrice`, `usedCount`, `usedMinPrice`, `deliveryDate`, `rating`
`amazon.full`	Everything in prices & delivery, plus images, description and attributes.	5	`price`, `listPrice`, `inStock`, `availabilityType`, `merchant`, `isFba`, `offerListingId`, `condition`, `newCount`, `newMinPrice`, `usedCount`, `usedMinPrice`, `deliveryDate`, `rating`
`amazon.offers`	All offers on the listing: who holds the buy box, the lowest delivered price, how many sellers.	1–3	`offerCount`, `buyboxSeller`, `buyboxPrice`, `lowestPrice`
`amazon.offer_check`	Checks the exact buy-box offer the last product step found. Cheap, and catches a pulled offer.	0.5	`buyable`
`walmart.standard`	Price, stock and the rest of what the retailer shows.	1	`price`, `listPrice`, `inStock`, `seller`, `byPlatform`, `clearance`
`target.standard`	Price, stock and the rest of what the retailer shows.	1	`price`, `regularPrice`, `inStock`, `shipping`, `pickup`
`bestbuy.standard`	Price, stock and the rest of what the retailer shows.	1	`price`, `regularPrice`, `inStock`, `purchasable`, `state`

amazon.offer_check checks the exact buy-box offer an earlier product or offers step found, so one of those must run first; items without a known offer are skipped. GET /v1/watchlists/scrapers?platform=…&marketplace=… lists what can run in a marketplace, with ready-made templates.

Rules

Key	Meaning
`when`	The branch: `new`, `changed`, `unchanged`, `ok`, `failed`, `not_found`, `always`. `new`: first read. `changed` / `unchanged`: a tracked field moved, or did not. `ok`: found (use with `field` and `op` to test a value on every run). `failed`: no answer this run. `not_found`: gone (also delisted). `always`: every item the step answered.
`field`	With `changed`: only a change to this field counts. With `ok`: the field `op` tests.
`op`, `value`	`increased`, `decreased`, `below`, `above`, `is`, `is_not`. `increased` and `decreased` compare with the last read; the others compare with `value`.
`run`	A step id: matching items go through that step next, in the same run.
`notify`	`true`: send the matching items. A `changed` rule sends `watchlist.product.changed`; any other branch sends `watchlist.product.matched`.

Pipeline keys

Key	Meaning
`steps`	Each `{ id, scraper, routes, condition? }`. `condition` (`all`, `new`, `used`) is for `amazon.offers`.
`entry`	The step every run starts with. Default: the first step.
`baseline`	Where items never read before start instead (for example a `full` read). Default: `entry`.
`refresh`	`{ "every_runs": N, "entry": "<step>" }`: every Nth run starts at that step instead.

PATCH a list with "pipeline": null to go back to the default.

Billing

A run costs exactly what its reads would cost as API calls: each step is billed at its scraper's rate, rounded up once per call, and partial and unavailable items are refunded. GET /v1/watchlists/{id} shows estimated_credits per run.

Example: 2,000 Walmart items on 15m cost about 2,000 credits per run. A 10,000-item Amazon list on the default pipeline costs about 2,500 credits per scan run plus 1 per item that moved, and 10,000 on each refresh run.

When the balance runs out

A run reserves its first step up front. If your balance cannot cover it, the run is skipped: nothing is read or charged, and watchlist.run.completed arrives with "status": "insufficient_credits". After 3 skips in a row the list is paused (paused_reason: "insufficient_credits"): top up, then PATCH it to "status": "active". If only a later step cannot be covered, its items wait for the next run (stats.deferred).

Events

Each run sends, in order: watchlist.product.changed, watchlist.product.matched and watchlist.product.delisted (each only when there is something to say), then watchlist.run.completed. Item events are batched, at most 500 items per delivery (part of parts).

Every entry carries id, the item id. Amazon entries also carry asin, the same value, for older receivers. changes holds only the fields that moved, as { before, after }; snapshot is every tracked field after the change.

watchlist.product.changed

{
  "id": "evt_1a2b3c4d5e6f7a8b9c0d",
  "type": "watchlist.product.changed",
  "created_at": "2026-10-16T09:15:04.120Z",
  "data": {
    "watchlist_id": "wl_9a8b7c6d5e4f3a21",
    "watchlist_name": "Walmart MAP",
    "platform": "walmart",
    "marketplace": "us",
    "currency": "USD",
    "run_id": "wlrun_0d1e2f3a4b5c6d7e",
    "run_number": 7,
    "part": 1,
    "parts": 1,
    "changes": [
      {
        "id": "5129928603",
        "changed_at": "2026-10-16T09:15:02.871Z",
        "tier": "standard",
        "step": "check",
        "scraper": "walmart.standard",
        "changes": {
          "price": {
            "before": 24.97,
            "after": 21.48
          },
          "inStock": {
            "before": false,
            "after": true
          }
        },
        "snapshot": {
          "price": 21.48,
          "listPrice": 29.99,
          "inStock": true,
          "seller": "Walmart.com",
          "byPlatform": true,
          "…": "…"
        }
      }
    ]
  }
}

watchlist.product.matched (data)

{
  "watchlist_id": "wl_7c2e9a1b3d4f5e60",
  "watchlist_name": "Back in stock",
  "platform": "target",
  "marketplace": "us",
  "currency": "USD",
  "run_id": "wlrun_2a3b4c5d6e7f8091",
  "run_number": 12,
  "part": 1,
  "parts": 1,
  "products": [
    {
      "id": "83887639",
      "matched_at": "2026-10-16T09:15:03.512Z",
      "step": "check",
      "scraper": "target.standard",
      "branch": "ok",
      "status": "ok",
      "values": {
        "price": 399.99,
        "inStock": true
      }
    }
  ]
}

watchlist.product.delisted (data)

{
  "watchlist_id": "wl_5e6f7a8b9c0d1e2f",
  "watchlist_name": "Headphones",
  "platform": "bestbuy",
  "marketplace": "us",
  "currency": "USD",
  "run_id": "wlrun_9f8e7d6c5b4a3921",
  "run_number": 31,
  "part": 1,
  "parts": 1,
  "products": [
    {
      "id": "6418599",
      "delisted_at": "2026-10-16T09:15:01.002Z",
      "detected_by": "standard",
      "step": "check",
      "scraper": "bestbuy.standard",
      "last_snapshot": {
        "price": 129.99,
        "inStock": false,
        "…": "…"
      }
    }
  ]
}

watchlist.run.completed (data)

{
  "id": "wlrun_0d1e2f3a4b5c6d7e",
  "object": "watchlist_run",
  "watchlist_id": "wl_9a8b7c6d5e4f3a21",
  "run_number": 7,
  "trigger": "schedule",
  "kind": "pulse",
  "status": "completed",
  "credits": 1998,
  "stats": {
    "tracked": 2000,
    "entry": "check",
    "new_items": 0,
    "steps": {
      "check": {
        "scraper": "walmart.standard",
        "requested": 2000,
        "ok": 1996,
        "partial": 0,
        "not_found": 2,
        "unavailable": 2,
        "credits": 1998,
        "skipped": 0
      }
    },
    "matched": 0,
    "deferred": 0,
    "changed": 14,
    "delisted": 2,
    "retry_next_run": 2
  },
  "error": null,
  "started_at": "2026-10-16T09:15:00.004Z",
  "finished_at": "2026-10-16T09:15:21.101Z",
  "watchlist": {
    "id": "wl_9a8b7c6d5e4f3a21",
    "name": "Walmart MAP",
    "platform": "walmart",
    "marketplace": "us",
    "status": "active",
    "paused_reason": null,
    "next_run_at": "2026-10-16T09:30:00.004Z"
  }
}

Missed a delivery?

GET /v1/watchlists/{id}/items?changed_since=<ISO time> returns every item that changed or was delisted since then, with its current snapshot and last_change.

Endpoints

POST/v1/watchlistsCreate a list. Supports Idempotency-Key.
GET/v1/watchlistsYour lists.
GET/v1/watchlists/{id}One list, with item_count, items_by_state and estimated_credits.
PATCH/v1/watchlists/{id}name, the schedule (every, or cron and timezone), webhook_url or webhook_endpoint_id (null clears), pipeline (null: the default), refresh_every_runs, status (active, paused).
DELETE/v1/watchlists/{id}Delete the list with its items and runs.
POST/v1/watchlists/{id}/items{ "add": […], "remove": […] }. Added items get a baseline read on the next run.
GET/v1/watchlists/{id}/itemsItems with their last snapshot. ?changed_since=, ?state=, ?limit= (up to 1,000), ?starting_after=.
POST/v1/watchlists/{id}/runRun now (202). Replaces the next scheduled run; 409 if one is in progress.
GET/v1/watchlists/{id}/runsRun history with per-step counts and credits.
GET/v1/watchlists/scrapersScrapers and templates for ?platform= and ?marketplace=. Free.

Add and remove items

curl -X POST "https://api.scrapelyserp.com/v1/watchlists/wl_9a8b7c6d5e4f3a21/items" \
  -H "Authorization: Bearer $SCRAPELY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"add":["1000000404"],"remove":["931920073"]}'

What changed since yesterday

curl -G "https://api.scrapelyserp.com/v1/watchlists/wl_9a8b7c6d5e4f3a21/items" \
  -H "Authorization: Bearer $SCRAPELY_API_KEY" \
  --data-urlencode "changed_since=2026-10-15T09:00:00Z"

Every schema is in the API reference. POST /v1/watchlists/{id}/asins (old name of /items) and the asin_count / asins_by_state fields still work.