The pipeline
Six steps from a pricing page to the record.
Machines checking machines — and when they disagree, or a number moves absurdly far, the line stops and a person decides.
01
Capture
MarkupBot fetches each tracked pricing page on an adaptive schedule — recently changed pages several times a day, quiet ones weekly. A plain fetch first; a real browser only when the page demands it. robots.txt is honoured and nothing behind a login is ever collected — see /bot.
ADAPTIVE SCHEDULE · ROBOTS HONOURED
02
Archive
Every capture that differs from the last one is stored raw — the page text and a rendered image, hashed and immutable. Published figures link to the exact capture they came from, so anyone can check the source we saw.
RAW HTML + IMAGE · CONTENT-HASHED
03
Extract
A language model reads the capture into structured plans. Then the machine checks the machine: every extracted price must appear verbatim in the page text, or the whole extraction is discarded. Models suggest; evidence decides.
EVIDENCE-CHECKED · DISCARDED ON DOUBT
04
Diff
Yesterday’s plans against today’s, field by field: price, billing interval, seat minimums, included units, limits, features that moved tier. The diff is deterministic code, not model output, and a suite of golden test cases referees every change to it.
DETERMINISTIC · GOLDEN-TESTED
05
Gate
Two machines must agree before anything ambiguous publishes: a second, independent model re-reads the capture and audits the first extraction figure by figure. Agreement publishes, contradiction rejects, and anything either side is unsure about — plus every price move beyond ±200% — waits for a human. The queue size is published above.
INDEPENDENT AUDIT · EXTREMES WAIT FOR A HUMAN
06
Publish
Changes go live with an effective date labelled either “stated by vendor” or “detected by us” — never implied precision. Alerts fan out the same day. A published change is never silently edited; fixes go through public corrections.
DATED · SOURCED · NEVER SILENTLY EDITED