Skip to content

Data layer

Order book history that can be reconstructed, not merely replayed.

Expreci is designed around sequence-aware capture, explicit data-quality state, and deterministic reconstruction of Polymarket order books.

Why another data layer

A file of updates is not necessarily a book

Prediction-market data is often assembled from public endpoints, ad hoc websocket collectors, and periodic snapshots. Those sources can be useful, but the resulting archive may conflate event time with receive time, omit sequence gaps, duplicate events after reconnects, or preserve deltas without enough state to establish a valid starting book.

Those defects are most consequential during volatility—the periods when microstructure research is most sensitive to event ordering. Expreci's intended data layer treats capture provenance and reconstruction state as first-class data rather than cleanup details.

Intended pipeline

Preserve evidence before producing convenience

  1. 01

    Regional collectors

    Independent capture processes retain source timestamps, local receive timestamps, raw payloads, and source identifiers. The architecture supports region-aware capture and failover.

  2. 02

    Sequence and gap control

    Events are ordered by source sequence where available. Duplicates are idempotently rejected; discontinuities create explicit gap records rather than being silently interpolated.

  3. 03

    Immutable event archive

    Normalized deltas, snapshots, and capture metadata are intended to be persisted as append-only records so reconstruction can be repeated and audited.

  4. 04

    Deterministic materialization

    A versioned reconstruction engine applies one defined transition rule set to produce queryable books and derived depth observations.

Event model

Sequence, source, and time remain explicit

A normalized delta is intended to remain traceable to its raw source while carrying enough information for deterministic application.

Open the full schema reference

Core event shape

Book delta
{
  "type": "book_delta",
  "market_id": "mkt_01K2F7Y9AT4B",
  "asset_id": "ast_01K2F7Z1CN8Q",
  "sequence": 18422091,
  "exchange_timestamp": "2026-06-01T14:02:09.481Z",
  "received_timestamp": "2026-06-01T14:02:09.497Z",
  "changes": [
    { "side": "bid", "price": "0.6200", "size": "1450.00" },
    { "side": "ask", "price": "0.6300", "size": "980.00" }
  ]
}

Data quality contract

What we optimize for

DimensionTarget propertyIntended evidence
CompletenessEvery expected sequence is present or the missing interval is explicitly represented.Per-stream gap ledger; no hidden fill-forward.
OrderingEvents are applied according to a documented source order, not arrival order alone.Source sequence plus deterministic tie-break rules.
ReproducibilityThe same snapshot, deltas, and engine version produce the same book state.Content hashes and reconstruction version identifiers.
Time fidelityExchange and receive time remain distinct throughout the pipeline.UTC timestamps at source precision; no invented event time.

Fit assessment

Tell us which failure modes matter to your research.

Register data interest