newsfilter

Receiving webhooks

Verify the signature, answer fast, and expect duplicates and reordering.

newsfilter sends every event on your tickers to your webhook URL as it happens. Webhooks are signed with Standard Webhooks, which has libraries for most languages.

The request

Each event is a POST with a JSON body, the event envelope:

{
  "event_id": "evt_01JC4ZQ8M6W7Y3T2K9P5R1N0BX",
  "type": "story.updated",
  "schema_version": 1,
  "occurred_at": "2026-11-03T14:05:00Z",
  "origin": "live",
  "ticker": "ACME",
  "data": { "story": { "id": "st_…", "version": 3 }, "changed": ["articles"] }
}

and three headers:

HeaderValue
webhook-idThe event_id. The same on every retry.
webhook-timestampWhen this attempt was sent, in Unix seconds. New on every retry.
webhook-signatureThe signature of this attempt.

Each type's data is in the reference.

Verify the signature

Verify before you trust or parse anything. The library checks the signature and rejects attempts whose timestamp is too old, which stops replayed requests. Verify the raw body: parsing and re-serialising the JSON first changes the bytes and the check fails.

import { Webhook } from 'standardwebhooks';

const wh = new Webhook(process.env.NEWSFILTER_WEBHOOK_SECRET!); // whsec_…

export async function POST(req: Request) {
  const body = await req.text();
  try {
    const event = wh.verify(body, Object.fromEntries(req.headers));
    await saveForLater(event);
  } catch {
    return new Response('bad signature', { status: 401 });
  }
  return new Response(null, { status: 204 });
}

Answer fast

Answer with any 2xx within 10 seconds. Anything else counts as a failure: another status, a timeout, a network error, or a redirect (redirects are not followed). Store the event and answer; do the work afterwards, in a queue or a background job.

If your endpoint times out or can't be reached, the rest of your queued events wait for the next attempt, about a minute later, so one slow response delays the others.

Retries

A failed delivery is retried with the same webhook-id and a fresh timestamp and signature, about:

AttemptAfter the previous one
230 seconds
31 minute
42 minutes
54 minutes
68 minutes
716 minutes
832 minutes
9 onwards1 hour

Each delay varies by up to 20%. newsfilter stops trying once the next attempt would fall more than 24 hours after the delivery was queued, which is when the event was published. A delivery that runs out of retries is not sent again; the event is still in catch-up for 30 days.

Duplicates

Delivery is at least once. You may get the same event twice, for example when newsfilter can't record that your endpoint accepted it. Two things handle it:

  • Write your copy of the data so that applying an event twice changes nothing. Keeping a copy in sync shows how.
  • For anything that must happen once, such as a notification, claim the event_id (or the story id) with a single insert before acting; see Notifications.

Order

Events go out in the order they were published, but a failed one is retried on its own schedule and doesn't hold back the others. Your own handlers may also run several at once. So a retried story.created can arrive after the same story's story.updated. Don't rely on arrival order:

  • Compare a story's version with the one you have, and keep the higher.
  • For one fixed order, use catch-up: it returns events in the order they were published.

Keeping a copy in sync shows two ways to handle this.

Notifications and origin

origin says why an event exists:

  • live: news as it arrives.
  • backfill: news loaded for a ticker that was just added. Don't notify users for these.
  • reprocess: an article judged again, for example with a newer model.

On this page