Automation

Webhooks

Register HTTPS endpoints, receive watchlist events as signed callbacks, and verify that each one really came from Scrapely.

Endpoints

Each endpoint has its own signing secret (whsec_…). It is returned once, when you create the endpoint or rotate its secret, and never again: lists show only its last four characters.

curl -X POST "https://api.scrapelyserp.com/v1/webhook_endpoints" \
  -H "Authorization: Bearer $SCRAPELY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/hooks/scrapely",
    "description": "Production",
    "events": ["watchlist.product.changed", "watchlist.product.delisted"]
  }'

Field	Description
`url`	Required. An `https://` URL on a public address (see Allowed destinations).
`description`	Up to 120 characters, for you.
`events`	`null` (the default): every event. A list: only those. `[]`: none, except deliveries that name this endpoint.
`enabled`	Default `true`. A disabled endpoint receives nothing.

GET/v1/webhook_endpointsYour endpoints. Never their secrets.
POST/v1/webhook_endpointsCreate one. Returns secret once.
PATCH/v1/webhook_endpoints/{id}Change url, description, events or enabled.
DELETE/v1/webhook_endpoints/{id}Remove it. Deliveries still naming it are recorded as failed.
POST/v1/webhook_endpoints/{id}/rotateA new secret, returned once. See Rotating a secret.
POST/v1/webhook_endpoints/{id}/testSend one signed webhook.test now and report how your endpoint answered.

Events

Event	Sent when
`watchlist.product.changed`	A notify rule fired on a change (the default pipeline: a tracked field moved).
`watchlist.product.matched`	A notify rule fired on any other branch, such as a value test.
`watchlist.product.delisted`	An item came back `not_found` and was delisted.
`watchlist.run.completed`	A run finished, was skipped or failed. Always sent, last.
`webhook.test`	You press Send test or call `/test`. Only to that endpoint.

A watchlist's events go to the endpoint its webhook_url (or webhook_endpoint_id) names. A list without one sends to every enabled endpoint subscribed to the event. Payloads are on Watchlists.

Delivery

Every delivery is a POST with a JSON body: an envelope with the event id, type, created_at and its data.

{
  "id": "evt_1a2b3c4d5e6f7a8b9c0d",
  "type": "watchlist.product.changed",
  "created_at": "2026-10-16T09:15:04.120Z",
  "data": {
    "watchlist_id": "wl_9a8b7c6d5e4f3a21",
    "watchlist_name": "Walmart MAP",
    "platform": "walmart",
    "marketplace": "us",
    "currency": "USD",
    "run_id": "wlrun_0d1e2f3a4b5c6d7e",
    "run_number": 7,
    "part": 1,
    "parts": 1,
    "changes": [
      {
        "id": "5129928603",
        "changed_at": "2026-10-16T09:15:02.871Z",
        "tier": "standard",
        "changes": {
          "price": {
            "before": 24.97,
            "after": 21.48
          }
        },
        "snapshot": {
          "price": 21.48,
          "…": "…"
        }
      }
    ]
  }
}

Header	Value
`Scrapely-Signature`	`t=<unix seconds>,v1=<hex HMAC-SHA256>`. Two `v1=` while a rotation overlaps.
`Scrapely-Endpoint`	The receiving endpoint's id (`we_…`): which secret to verify with.
`Scrapely-Event`	The event type.
`Scrapely-Delivery`	The event id, stable across retries. Deduplicate on it.
`Scrapely-Attempt`	`1` for the first try, then `2`, `3`, …

Responding and retries

Answer with any 2xx within 10 seconds, then do the work. Anything else (a 4xx or 5xx, a timeout, a redirect) is retried after 10 s, 1 min, 5 min, 30 min and 2 h. Redirects are never followed.

Make handlers idempotent

Delivery is at least once: a retry can arrive after a response we did not see. Record the Scrapely-Delivery ids you have processed and skip repeats.

Verifying signatures

Read t and every v1 from Scrapely-Signature.
Compute HMAC-SHA256 of "<t>.<raw body>" with the endpoint's secret.
Accept if it equals any v1, compared in constant time.
Refuse timestamps more than 5 minutes old. That stops a captured delivery being replayed.

Use the raw body

Verify the exact bytes you received, before parsing JSON. Re-serialising changes whitespace and the signature will not match.

import { createHmac, timingSafeEqual } from 'node:crypto';

export function verifyScrapely(rawBody, header, secret, toleranceSec = 300) {
  let t = 0;
  const sigs = [];
  for (const part of header.split(',')) {
    const [k, v = ''] = part.split('=');
    if (k === 't') t = Number(v);
    else if (k === 'v1') sigs.push(Buffer.from(v, 'hex'));
  }
  if (!t || Math.abs(Date.now() / 1000 - t) > toleranceSec) return false;
  const expected = createHmac('sha256', secret).update(`${t}.${rawBody}`).digest();
  // Two v1 values while a rotation overlaps: either may match.
  return sigs.some((s) => s.length === expected.length && timingSafeEqual(s, expected));
}

// Express: keep the raw body for this route.
app.post('/hooks/scrapely', express.raw({ type: 'application/json' }), (req, res) => {
  const raw = req.body.toString('utf8');
  if (!verifyScrapely(raw, req.get('Scrapely-Signature') ?? '', process.env.SCRAPELY_WEBHOOK_SECRET)) {
    return res.sendStatus(400);
  }
  res.sendStatus(200);
  queue.add(JSON.parse(raw)); // work after acknowledging
});

Rotating a secret

Rotate from the endpoint's row in the dashboard, or with POST /v1/webhook_endpoints/{id}/rotate. The new secret is returned once and signs every delivery from that moment.

Body	What happens
`{ "overlap": true }` (default)	For 24 hours each delivery carries two `v1=` signatures, one per secret. Deploy the new secret within that window; nothing fails meanwhile.
`{ "overlap": false }`	The old secret stops at once. Use it if a secret may have leaked; deliveries fail verification (and are retried) until your receiver has the new one.

An endpoint URL must be https://, on port 443 or 1024–65535, and resolve to a public address. Loopback, private, link-local and cloud metadata addresses are refused when the endpoint is registered and again at every delivery.