Stage 2

Ingestion and order-book reconstruction

Turn the live websocket / REST feed into a continuously valid order book, or refuse to trade against it.

Challenge we are solving

Feeds drop. Sequence numbers gap. Snapshots disagree with deltas. If you trade against an invalid book you mis-price and over-fill. The challenge is detecting the bad state fast enough to stop downstream stages.

What this stage does

Consumes the market feed, applies deltas to a local order book, validates sequence numbers, detects stale books, triggers resyncs, and emits a fresh order_book_snapshot at every tick.

Why this stage exists

Every downstream stage — signals, fair value, execution — reads from the book. The ingestion stage is the only place that knows whether the book is currently trustworthy.

Flow

WS feedpolymarket book

Sequence
checkseq_t = seq_{t-1}+1

Apply
deltasdepth_t = depth_{t-1}+Δ

Validatefresh · monotonic

SnapshotOrderBookSnapshot

What the backend should expose

feed_status (connected · degraded · disconnected)
last_seq, expected_seq, gap_count_60s
stale_duration_ms (since last accepted update)
book_validity (valid · stale · resyncing · invalid)
resync_jobs (last 1h)
last_snapshot_ts, last_delta_ts

Maths we expect here

Every formula below is implemented in packages/polytraders-bots/ or packages/polytraders-runner/. Treat the worked example as the unit-test sanity check you should be able to reproduce locally.

Apply a depth delta

\[depth_t = depth_{t-1} + \Delta_t\]

Symbol	Meaning	Units / range
\(depth_t\)	Order-book depth vector at tick t	shares per level
\(\Delta_t\)	Delta payload from the websocket	signed shares per level

worked example\[depth_{t-1} = [120, 80, 40],\; \Delta_t = [+10, 0, -20] \;\Rightarrow\; depth_t = [130, 80, 20]\]

A book is rebuilt by replaying the last accepted snapshot plus every delta since. A single missed delta invalidates the chain.

Sequence-gap detection

\[seq_t = seq_{t-1} + 1 \;\;\text{else}\;\; \text{invalidate and resync}\]

Symbol	Meaning	Units / range
\(seq_t\)	Sequence number of latest update	monotonic integer
\(seq_{t-1}\)	Previous sequence number	monotonic integer

worked example\[seq_{t-1} = 992,\; seq_t = 994 \;\Rightarrow\; \text{gap}=1 \;\Rightarrow\; \text{resync from snapshot}\]

Gaps trigger book_validity = invalid until a fresh REST snapshot lands and the local replay matches the new seq.

Update-age (staleness)

\[freshness\_ms = now - last\_update\_ts\]

Symbol	Meaning	Units / range
\(now\)	Current time	ms epoch
\(last_update_ts\)	Timestamp of last applied update	ms epoch

worked example\[now - last\_update = 4{,}200\,\text{ms} > 2{,}000 \;\Rightarrow\; \text{mark stale}\]

Top-of-book validity

\[ask > bid \;\wedge\; ask - bid \le max\_spread\]

Symbol	Meaning	Units / range
\(bid\)	Best bid	0..1
\(ask\)	Best ask	0..1
\(max_spread\)	Per-market sanity cap on quoted spread	configurable; defaults 0.30

worked example\[bid=0.62,\; ask=0.64 \;\Rightarrow\; spread=0.02 \le 0.30 \;\Rightarrow\; \text{valid}\]

How a developer codes this stage

Reference TypeScript implementation lives in packages/polytraders-* at the repository root. Stage owners maintain these files — read them before writing new code.

packages/polytraders-contracts/src/OrderBookSnapshot.tsThe shape every consumer downstream receives. Includes seq, ts, bids[], asks[], hash.
packages/polytraders-mock-clob-v2/src/orderbook.jsThe reference CLOB V2 book builder. Applies deltas, computes hashes, exposes freshness.
packages/polytraders-synthdata/src/generator.jsReplay-deterministic synthetic book generator — same delta protocol as live, used for tests.

See it in the platform mock

The platform mock is the source of truth for what each stage's UI exposes. Open these alongside the code references.

↗Live book viewer/mock-app/data-book.html ↗Feed status + gaps/mock-app/data-registry.html ↗Stages 2-3 in depth/pipelines/data-flow.html

Reason codes emitted at this stage

INTEL_*INTEL — ingest, freshness, signals

Hover or tap any reason code on this page (or anywhere on the site) to see its canonical short description. Full registry: /standards/reason-codes.