Stories and versions
How articles become stories, how stories change, and what each field promises.
Articles
newsfilter reads every article the news provider lists for your tickers and judges it once per ticker: an article about two companies is judged twice and appears once for each. Each judgement says whether the article reports something new about that company, and why.
Articles that pass are shortlisted. GET /v1/articles lists them by
default; status=all lists every article with its pipeline_status, which
says where it is in the pipeline:
pipeline_status | Meaning |
|---|---|
shortlisted | Judged and kept. It joins a story when its ticker is next grouped; until then story_id is null. |
dropped | Judged and not kept. judgement.reason says why. |
duplicate | A copy of an article already seen for this ticker. |
skipped | From a source newsfilter does not fetch. |
pending | Not judged yet. |
failed | Judging failed. |
Stories
Shortlisted articles about the same event are grouped into a story.
A story has a headline, a summary, its articles and a strong flag.
- Its id stays the same as articles join it. When a regrouping splits or merges stories, the id stays with the group that keeps most of the original articles.
versiongoes up by one on every change you can see: its articles, headline, summary orstrong. Two snapshots of the same story can always be ordered byversion; keep the highest.- The summary comes later. A new story's
summarycan benull; it arrives in astory.updatedevent withchanged: ["summary"]. - Strong is one-way. A story flagged strong stays strong, and
story.became_strongis sent once per story.strong_reasongives the reason the story is strong, or why it is not. - Stories close. A story closes when its articles move into another
story (
merged) or when none of them is shortlisted any more (no_articles), for example after they are judged again. A closed story is gone: lists skip it andGET /v1/stories/{id}answers404. Apps hear about it throughstory.closed, whoseversionis higher than any earlier snapshot of the story.
Currently a story stops changing once none of its articles is from the last 7 days.
Events
Every change is recorded as an event, sent as a webhook and kept for 30 days for catch-up.
| Event | When |
|---|---|
article.shortlisted | An article passed the filter and was placed in a story. Once per article and ticker. |
story.created | A new story. |
story.updated | A story changed; changed lists what. |
story.became_strong | A story was flagged strong. Once per story. |
story.closed | A story merged into another or has no articles left. Once per story. |
Every event carries a snapshot taken when it was created, so its data is
the story or article as it was then, not as it is now.