Market Data Backfill Strategies for Prediction Markets

A good backfill loads old data into the same schema and the same code path as your live feed. It never overwrites history with today’s view of it. For prediction markets, that means keeping market identity, revisions, and markets that later settled or disappeared, then proving the result is continuous before any signal or order logic touches it.

This guide is for developers, quants and bot builders who pull data from more than one venue. It covers the order of operations, the schema, the failure modes specific to Polymarket, Kalshi and Limitless, and the validation checks that tell you the backfill is done.

Assymetrixdata.assymetrix.comBackfill With Cross-Venue ContextAssymetrix unifies historical and real-time prediction market data from Polymarket, Kalshi, and Limitless through one Data API.Explore the Data API

TL;DR

• Open and acknowledge the live subscription first, buffer its events, then page history and replay the buffer. Doing it the other way round leaves a hole at the seam.

• Store immutable raw events plus a normalized layer. Keep exchange-event time and local-receipt time as separate columns.

• Retain every revision and every market that later settled, expired or was renamed. Current-only tables create survivorship bias and lookahead.

• Match the data type to the strategy: bars cannot reproduce a model that depended on quotes or depth.

• Declare a backfill finished only after checks for gaps, duplicates, finality and cross-venue disagreement pass.

Table of Contents

• What a backfill has to get right

• Decide the scope before you page anything

• The backfill sequence that avoids a seam

• A schema that survives corrections

• Point-in-time correctness

• Cross-venue backfill: where the three venues differ

• Validate before you trust it

• Perspective: why we backfill at the venue-neutral layer

• Build it yourself or start from a unified feed

• A worked example: backfilling a divergence study

• FAQ

• Sources

What a backfill has to get right

A backfill is not a bulk download. It is a promise that your historical table and your live table describe the same world, with no seam between them. Most of the pain comes from three places.

Identity. Polymarket, Kalshi and Limitless each name markets, outcomes and events differently, and their lifecycles differ. A market that looks like one row today may have been several things over its life. Engineering guidance on vendor disagreement over IDs, timing and corrections calls out historized identifier mapping as a core control, and that applies directly here.

Time. There is the time the venue says something happened, and the time your system learned of it. If you collapse them, you cannot later answer what you could have known at a given moment.

Truth over time. Venues correct trades, change metadata, and resolve markets. A table that stores only the latest value quietly rewrites the past.

The same engineering guidance lists what a robust backfill schema needs: immutable raw events, canonical normalization, historized identifier mapping, correction handling, gap detection, and separate exchange and receipt timestamps. We treat that list as a checklist, not a suggestion.

Decide the scope before you page anything

Backfills fail early when scope is vague. Write down four things before the first request.

1. Venues and market universe. Which of the three venues, and which markets: all of them, a category, or a specific set of events?

2. Date range. The start you actually need for the research question, not the earliest date available.

3. Data family. Trades, quotes, candles, order-book depth, or holder and wallet data.

4. Coverage boundary. The earliest and latest points each provider can actually serve.

The data family decision is the one people skip. Feed quality is strategy-dependent: trades are needed for execution and flow analysis, quotes for spread work, bars for slower signals, and depth for liquidity-sensitive models. Backfilling candles cannot reproduce a strategy that historically relied on quotes or depth. If your live bot reads the book, a candle-only history will flatter or mislead it.

For Smart Money work the data family is trades with wallet attribution. For divergence work it is aligned price snapshots across venues. Pick the family from the signal, not from what is easiest to fetch.

Record the coverage boundary

Some venues split data between a live window and an archive that sits behind a separate path, and the boundary moves over time. Check each venue’s current documentation for a cutoff or retention statement, and query it on every run instead of hard-coding a date. Store the boundary you observed alongside the job, so a later gap can be explained rather than rediscovered.

The backfill sequence that avoids a seam

The order matters more than the tooling. A practical sequence, adapted from a published historical and live data pipeline guide, looks like this.

1. Define venue and market identity.

2. Record the provider’s coverage boundary.

3. Open the live subscription and wait for its acknowledgment.

4. Buffer raw live events as they arrive.

5. Page historical markets, trades and candles.

6. Normalize history through the same path as live events.

7. Deduplicate.

8. Replay the buffered live events.

9. Validate continuity and finality.

10. Only then enable signals or order logic.

The reason for step 3 before step 5 is simple. If you page history first and subscribe afterward, anything that happens between the end of your last page and the start of your subscription is lost. Subscribing first and buffering means the overlap is duplicated rather than missing, and duplicates are cheap to remove.

Use one normalization path

History and live events should pass through the same normalizer. Two code paths produce two subtly different tables: different rounding, different outcome ordering, different handling of null fields. Those differences show up months later as a phantom edge in a backtest.

Resynchronize after every reconnect

A socket that reconnects has not necessarily restored your application state. The same pipeline guidance says to repeat a scoped resynchronization after reconnects rather than assuming transport recovery fixed things. In practice: note the last event you processed per market, fetch history from just before it, deduplicate, and continue.

A schema that survives corrections

A schema built for backfills separates what arrived from what you believe.

Two habits pay for themselves. First, keep raw events immutable, so a normalization bug is fixed by replaying, never by patching rows. Second, give the identity map validity intervals. A relabeled or merged market should not retroactively change what a past row meant.

If you store candles, a schema with one row per symbol, interval and bar-open time in UTC, as shown in this Postgres OHLCV walkthrough, makes gap detection a simple query. Derive candles from trades where you can, and keep the trades.

For metadata specifically, see our schema guide to market metadata in prediction markets.

Point-in-time correctness

The question a backtest must answer is what the model could have known then, not what the database says now. Point-in-time lessons for quants frame it exactly that way. In prediction markets the distinction touches wallet rankings, market availability, resolution status and cross-venue divergence calculations.

The method is straightforward. Preserve every revision with its availability timestamp and query the latest version available at the simulated decision time. Retain markets that later disappear, so the historical universe is not made only of survivors.

The scale of the survivorship problem is easy to underestimate. In one published equity example, 3,359 of 8,039 companies, or 41.8%, in a 2003 universe did not exist in the database in 2012. That is an equity figure, not a prediction market statistic, but the mechanism is the same: join history only to what is active today and you test on a flattering subset.

Four concrete leaks to avoid in prediction market research:

• Resolution lookahead. Using the final outcome to label markets in a universe that the strategy could not have identified in advance.

• Wallet rank lookahead. Ranking wallets by lifetime profit, then testing whether they would have been profitable earlier. Rank each wallet using only trades settled before the decision time.

• Pruned markets. Dropping markets that were delisted, renamed or merged.

• Silent metadata edits. Using today’s title, category or close time for a past decision.

If you want to see how point-in-time handling looks inside a strategy-testing tool, backtesting with historical data shows one approach to unbiased strategy research. Our own walkthrough, backtesting prediction market strategies with the Data API, applies it to price snapshots.

Cross-venue backfill: where the three venues differ

Backtesting a single venue hides most of these problems. The moment you compare Polymarket, Kalshi and Limitless, you meet them all, because identifiers, market lifecycles and update semantics differ between venues.

Identifiers. The same real-world event can appear as different IDs, with different outcome structures. Build a canonical event ID and map every venue ID to it, with dates. Our guide to event deduplication with canonical IDs goes into the matching rules.

Lifecycles. Markets open, trade, halt, resolve and settle on different schedules. Store the lifecycle state as an event with a timestamp, not as a column you overwrite.

Update semantics. One venue may push a full snapshot, another an increment. Normalize to events, and record which kind the source gave you.

Clocks. Venue timestamps are not on one clock. Keep exchange-event time and receipt time apart, and compare venues on exchange time only after you have measured typical skew.

Before building divergence signals on top, read how we detect price divergence between Polymarket and Kalshi. The signal is only as good as the alignment underneath it.

Validate before you trust it

A backfill is finished when checks pass, not when the last page returns. The metrics worth tracking, drawn from the same market-data engineering guidance, are feed liveness, latency, missing intervals, duplicate rate, correction rate and cross-venue disagreement rate.

Turn them into a gate:

Run the job twice on the same range. If the second run changes anything, your key or your normalizer is unstable. Idempotence is the cheapest correctness test there is.

Expect to find gaps that are real

Some gaps are the venue’s, not yours: a market with no trades for hours is not missing data. Mark quiet intervals as quiet by checking the venue’s reported activity, and log what you decided. A gap list with explanations is a deliverable.

Perspective: why we backfill at the venue-neutral layer

We built the Assymetrix Data API around one view: the backfill is the product, and the live feed is the easy part. Anyone can subscribe to a socket. The work is making history from Polymarket, Kalshi and Limitless agree with itself, with identifiers that stay stable when venues change how they name things.

That is why our platform rests on roughly 1.5 terabytes of historical data spanning nearly one billion rows of trading activity, and why we normalize across venues once instead of asking every team to do it again. A quant team that rebuilds three feeds spends its first quarter on plumbing, and the results of that plumbing quietly shape every backtest afterward.

Our rule of thumb is to distrust any historical table that cannot tell you what it knew and when. If you cannot reproduce last month’s result from last month’s data, the backfill is not finished. We would rather you find that in a test than in a live account.

— Dean

Build it yourself or start from a unified feed

Building in-house makes sense when you need one venue, a narrow date range and full control of the raw layer. It stops making sense when you need three venues, canonical IDs, wallet-level history and reproducible reruns, because the identity map and the validation gate become permanent maintenance.

The Assymetrix Data API gives developers, quants and AI agents cross-venue prediction market data through a single integration. For the shape of the feed, see the real-time and historical API guide, and for large historical pulls, the institutional backfill overview. For reproducible research, start with the historical data access guide.

A worked example: backfilling a divergence study

Suppose you want to test whether price gaps between Polymarket and Kalshi on the same event predict short-term moves.

1. Scope. Pick one category and a six-month window. You need trades and price snapshots, not depth.

2. Identity. Build the canonical event map with validity dates, including markets that have since resolved.

3. Live first. Start recording live events now, so the end of the window is covered by your own capture.

4. Page history for each venue and normalize through the live path.

5. Align clocks. Compare on exchange-event time, after measuring skew from receipt times.

6. Gate. Run gap, duplicate and finality checks. Rerun the job and diff the output.

7. Test with availability times. At each simulated decision time, use only rows and metadata that were available then.

The result is a study you can repeat. A later change in your findings then traces to the data or the model, not to a table that shifted underneath you. Our execution-aware divergence guide covers what to do once the signal exists.

FAQ

How far back should I backfill?

Only as far as the research question needs, plus a warm-up period for any rolling features. Older data often reflects different market structure and fewer participants, so more history is not automatically better. Record the start you chose and why.

Should I backfill before or after starting the live feed?

Start the live subscription first and buffer its events, then page history and replay the buffer. This turns the overlap into duplicates you can remove, rather than a gap you cannot see.

Do candles replace trades in a backfill?

Not if the strategy ever depended on trades, quotes or depth. Bars are fine for slower signals, but they cannot reproduce flow or liquidity-sensitive behavior. Keep trades where you can and derive candles from them.

How do I know my backfill has no survivorship bias?

Check that markets which later settled, expired or were renamed are still in your historical universe, with their original identifiers and metadata. Compare the count of markets per month against what the venue listed at the time, where you can. A universe that contains only currently active markets is biased.

Sources

• Market data vendors disagree on IDs, timing and corrections

• Historical and Live Market Data, Siebly AI Agent Guide

• Back-Testing Data Methodology, Portfolio123

• Point in time financial data and sampling

• Market Data Engineering and Feed Quality

• How to store OHLCV data: a Postgres schema that survives gaps, corrections, and adjustments

• Backtesting, Darkbot Documentation

Prediction Market Edit History: Tracking Market Revisions

A prediction market’s edit history is the record of everything that changed about a market after it opened: its question text, its resolution rules, its status, and any moderator action taken during resolution. If you build backtests, bots or research datasets, that record decides whether your results mean anything. This guide covers where revision data comes from, what the public records show, and how to store it so a backtest sees only what was knowable at the time. It is about prediction markets, not the history of stock exchanges.

AssymetrixBuild Better Prediction Market BacktestsAssymetrix brings together prediction market data across Polymarket, Kalshi, and Limitless through one Data API for research and trading tools.Explore the Data API

TL;DR

• A market is a living record, not a fixed row. Rules, status and resolution can change after trading starts, so store changes as timestamped events, never as overwrites.

• Polymarket’s open resolution subgraph creates a resolution entity when a question is initialized, with a status, a dispute flag and a last-update timestamp. Those fields are the backbone of an edit history.

• Moderator revisions are separately queryable through tooling built on that data, and they show when and how moderators stepped in during resolution.

• For backtests, the rule is point-in-time: use the version of a market that existed at the timestamp of each trade.

• Cross-venue work needs canonical market IDs, so a revision on one venue can be matched to the same event on another.

Table of Contents

• Why market edits matter more than most builders expect

• What the public on-chain records actually contain

• Moderator revisions: the intervention layer

• A schema for storing edit history

• Building the history step by step

• Point-in-time backtesting with revisions

• Revisions and resolution data together

• Cross-venue edits and canonical IDs

• Our perspective

• Where a data API fits

• Common mistakes

• FAQ

• Sources

Why market edits matter more than most builders expect

Most prediction market datasets arrive as a table: one row per market, with a question, an end date, an outcome and a final price. That shape hides a problem. The row you see today is the last version of the market. The market a trader faced three weeks earlier may have carried different wording, a different status, or an unresolved dispute.

If you train a model on the final version, you leak information. Suppose a clarification narrowed what “yes” meant on a given date. A model that sees the clarified text while replaying trades from before it will look smarter than it was. Prices before the clarification reflected a wider, vaguer question, and traders priced that ambiguity.

There are three kinds of edits worth separating:

1. Descriptive changes. The question text, description or supporting data shown to traders.

2. Lifecycle changes. The market moves between states such as posed, proposed, disputed and resolved.

3. Intervention changes. A moderator or a similar authority steps into resolution and changes the course of the outcome.

Each needs a different capture method, which is why a single “last updated” column is not an edit history.

What the public on-chain records actually contain

Polymarket publishes an open-source subgraph for market resolution. Its UMA CTF adapter handler shows how the first record is built. When a question is initialized, the handler logs the question ID and creates a MarketResolution entity keyed by that ID. It stores the creator as the author, saves the ancillary data, sets the last update timestamp from the request timestamp, marks the status as “posed”, and sets the disputed flag to false.

For anyone designing a schema, that handler teaches four things.

• The question ID is the stable key. Everything about a market’s life hangs off it, so your revision table should be keyed on it too.

• Ancillary data is part of the record. It carries the details that resolution depends on. Treat changes to it as rule changes, not cosmetic ones.

• Status is a state machine. The entity starts as “posed” and moves from there. Each move is an event with a time, so each deserves a row.

• The dispute flag starts false. A flip to true is one of the most informative events a market can have, and a history that only stores the current value loses it.

The handler shown is the initialization step only. Later handlers in the same project update the entity as the market moves through its lifecycle. When you build your own pipeline, read each handler and ask what it overwrites. Anything it overwrites is something you must snapshot yourself, because the entity will show only the latest value.

Moderator revisions: the intervention layer

The third kind of edit is the one traders worry about most. A public tool in the graph-polymarket MCP project, get_market_revisions, is described as retrieving moderator revisions for Polymarket markets, to see when and how moderators intervened in market resolution. Its input schema is small: an optional count of revisions to return and an optional question ID.

Two design lessons follow from that interface.

Query by question ID. Revisions only make sense in the context of one market. Pulling them across all markets without a key gives you a feed, which is useful for monitoring but not for analysis. When you study a particular market, always pass the ID.

Limit by count, then page. A revision list is usually short for a clean market and long for a contested one. The length itself is a signal. A market with many interventions is a different instrument from one that resolved without any.

We would not treat any single tool as the system of record. Use it to discover revisions, then confirm against the underlying on-chain or subgraph data and store your own copy with the fetch time.

A schema for storing edit history

The simplest durable design is an append-only event table next to a current-state table. Here is a minimal version.

The two timestamps matter most. effective_at tells you when the change happened in the world. observed_at tells you when you could have acted on it. A backtest should gate on observed_at if it models a live bot, and on effective_at if it models an idealized trader. Mixing them up is the most common source of silent look-ahead bias we see in prediction market research.

Store raw old and new values. If you store only your own classification, such as “minor edit”, you cannot revisit the classification when your definition changes.

Building the history step by step

1. Snapshot the full market list on a schedule. Hash the question text, description and resolution fields. A changed hash flags an edit even when no event was emitted.

2. Subscribe to lifecycle events. Index the resolution contract events through a subgraph or your own listener, and write one row per state transition.

3. Poll revision sources for each active market. Query by question ID and diff against what you stored.

4. Write to the append-only table first. Update the current-state table only after the event row is committed.

5. Backfill carefully. For markets that closed before you started collecting, you can rebuild lifecycle events from chain history, but descriptive edits may be unrecoverable. Mark those markets as incomplete rather than assuming no edits occurred.

6. Version your dataset. When a backfill changes past rows, publish a new dataset version so earlier results remain reproducible.

Step five is where honesty pays. An empty edit history can mean “no edits” or “we were not watching”. Your schema should be able to tell the two apart, for instance with a coverage_start field per market.

Point-in-time backtesting with revisions

Once you have the event table, a point-in-time view is a join. For each trade at time t, select the latest revision row per field where the gating timestamp is at or before t. That gives you the market as traders saw it.

Three checks catch most mistakes.

• Replay a known contested market. Confirm your view of the question at each trade matches the version live at that time.

• Compare price behavior around revisions. Look at spread and volume in the window before and after a descriptive or intervention event. A strategy that appears profitable only after revisions deserves suspicion.

• Exclude markets with incomplete coverage from headline results, and report them separately.

For how order flow and prices react in thin books, our piece on price impact in prediction markets pairs well with revision analysis, since revisions often land when liquidity is least forgiving. The wider structural picture is in prediction market microstructure.

Revisions and resolution data together

A revision history is most useful beside the outcome. Two questions become answerable: do markets with interventions resolve differently from clean ones, and how long do disputed markets stay unresolved? Neither can be answered from final-state tables alone.

Resolution rows need the same discipline as revisions: raw values, timestamps, a source reference. If you want a structured view of that side, see our guide to normalized prediction market resolution data. For on-chain analysis methods that feed both, the Polymarket on-chain analytics guide covers the quant workflow.

Cross-venue edits and canonical IDs

The same real-world event often trades on more than one venue, each with its own wording and its own process for changing rules. A revision on one venue can open a gap against another, and that gap is sometimes mistaken for an arbitrage. Often it is a difference in contract terms.

To reason about this you need a canonical ID that links equivalent markets, plus a field recording how their rules differ. Without it, a spread that looks like free money may be two different questions. Our arbitrage strategy guide goes through why rule comparison belongs before any trade logic, and the cross-venue data guide explains the ID approach. Regulatory context for US-regulated venues is covered in our CFTC rules guide for developers.

Our perspective

We build a cross-venue data layer, and the lesson we return to most is that a market is a sequence, not a snapshot. Our platform aggregates data from Polymarket, Kalshi and Limitless, built on roughly 1.5 terabytes of history across nearly a billion rows of trading activity. At that scale, the cost of an unrecorded change compounds: one silent rule edit contaminates every model trained across the affected window.

Our view is simple. Store events, not states. Keep two clocks. Mark what you did not observe. And keep the venue-neutral key separate from every venue’s own ID, because neutrality is what lets you compare how different venues handle the same event. We would rather a dataset say “coverage unknown” than imply a clean history it cannot prove.

— Dean

Where a data API fits

If you would rather not run the indexers, snapshot jobs and diffing yourself, our Data API gives developers unified access to cross-venue prediction market data through one integration. For Python workflows, start with the Python prediction market data guide. Pricing for the API is not published on our site, so check the Data API page for current plans.

Common mistakes

• Overwriting rows in place. It is the fastest way to destroy an edit history.

• Using one timestamp. Without separate effective and observed times, latency and look-ahead are indistinguishable.

• Treating the dispute flag as static. It starts false for every market, and its change is the event you care about.

• Assuming no revisions means a clean market. It may mean you were not collecting.

• Joining on question text. Text changes. Join on IDs.

• Ignoring ancillary data. Resolution depends on it, so changes there are rule changes.

FAQ

What does “market edit history” mean for a prediction market?

It is the ordered record of changes to a market after launch: wording, resolution rules, lifecycle status and moderator actions. Each change should carry a timestamp and the old and new values. Without it you can only see the market’s final form.

Where can I find moderator revisions for Polymarket markets?

The graph-polymarket MCP project exposes a tool that retrieves moderator revisions and accepts a question ID and a result count. Use it for discovery, and store your own timestamped copy of anything you rely on.

Why does edit history matter for backtesting?

Because a backtest that uses the final version of a market lets future information leak into past trades. Replaying each trade against the market as it stood at that moment removes that bias. It is the difference between a result you can trust and one that only looks good.

Can I rebuild edit history for markets that already closed?

Partly. Lifecycle events can often be reconstructed from chain history, as the resolution subgraph code suggests by tracking status and timestamps on-chain. Descriptive edits may be lost if nobody captured them, so flag those markets as incomplete.

Sources

• Polymarket resolution-subgraph: uma-ctf-adapter.ts

• get_market_revisions, graph-polymarket-mcp on Glama

Market Resolution Data: What to Capture and How to Use It

Market resolution data is the record of how a prediction market ended: which outcome won, who or what declared it, when, under which rules, and whether anyone disputed it. For a quant or a bot builder, it is the label on every backtest and the trigger for every payout. Treat it as a first-class dataset with its own schema, not as a single winner column.

This guide covers how resolution works on the two largest venues, which fields matter, where cross-venue joins break, and how to turn settlement events into usable signals.

AssymetrixBuild With Unified Resolution DataAssymetrix brings prediction market data across Polymarket, Kalshi, and Limitless together through one API for research and trading tools.Explore the Data API

TL;DR

• Resolution is a lifecycle, not a flag: market creation, close, proposal, challenge window, optional dispute, final outcome, payout. Store each stage with its own timestamp.

• Polymarket markets mostly settle through the UMA Optimistic Oracle. Kalshi settles centrally against named source agencies in its contract terms.

• Resolution is source-dependent. Two markets on the same event can settle differently if their rules or data sources differ, so keep the rule text next to the outcome.

• Joining venues requires identifier mapping and timestamp normalization, because block time and exchange-server time are not the same clock.

• Dispute and challenge windows are tradable information. Model them as states, not noise.

Table of Contents

• What resolution data actually contains

• How Polymarket resolution works, as data

• How Kalshi resolution works, as data

• Source-dependent, not exchange-dependent

• Joining resolution data across venues

• A minimal resolution schema

• Using resolution data in research and bots

• Our perspective

• Working with resolution data through an API

• Pre-launch checklist

• FAQ

• Sources

What resolution data actually contains

A useful resolution record answers six questions about every market:

1. What was the question? The exact title and the full rule text, versioned, since clarifications can be posted after launch.

2. What counted as evidence? The named resolution source: an agency, a data release, an official result page, or a free-text description.

3. When did it close and when did it settle? These are different timestamps, and the gap between them is where most of the interesting behavior lives.

4. Who proposed the outcome? On oracle-based venues, a proposer address. On centralized venues, the exchange itself.

5. Was it disputed? A boolean is not enough. Keep the number of rounds, the timing, and the final verdict.

6. What was the final payout vector? For binary markets, 1 or 0. For multi-outcome or scalar markets, the full vector, including any split outcomes.

Researchers at HKUST and NYU describe the same need from the academic side. Their suite of prediction market datasets links market metadata, fill-level trades and oracle-resolution events into one lifecycle, and reports that the main obstacle to algorithmic trading in this space is data fragmentation across on-chain and off-chain sources. As of March 2026, the paper counts more than 770,000 market records and 943 million fill records in its aggregated data.

How Polymarket resolution works, as data

Polymarket does not decide its own winners. Most markets resolve through the UMA Optimistic Oracle. According to the UMA documentation, the sequence runs like this:

• A proposer posts an outcome and a bond (the documentation lists $750).

• A challenge window opens, which is 2 hours.

• If nobody disputes, the proposal becomes final and payouts execute.

• If someone disputes, the request moves to a debate period of roughly 24 to 48 hours, then a vote by UMA token holders through the DVM, which takes about 48 hours.

UMA reports that 99.8% of requests resolve without dispute. That number is a useful prior, but it is an average over every request type. Ambiguous political or legal questions are overrepresented among the disputed ones, so do not apply the 0.2% to every market you track.

The states worth storing

For a bot or a research pipeline, the oracle flow maps cleanly onto a small state machine:

The on-chain events behind each transition carry block numbers and transaction hashes, which makes them auditable. That is a real strength of this venue’s data, and a reason to keep the raw event next to your parsed state.

Why the challenge window matters to a model

The UMA design rests on economic incentives: staking and slashing create a quantifiable cost of corruption. For anyone consuming this data, the practical consequence is that the challenge window acts as a buffer zone. Prices on a proposed-but-unfinalized market often sit just below the payout value, and volatility can cluster around the window as participants weigh the chance of a dispute. Treat that as a hypothesis to test on your own data, not as a guarantee. Label every fill with its distance to the proposal and to the window end, and let the data tell you whether the clustering exists for your market categories.

How Kalshi resolution works, as data

Kalshi uses a centralized model. According to a summary of how contracts settle, each market lists named source agencies in contract terms that are filed with the CFTC through the exchange’s self-certification process. The same summary gives examples by category: governing leagues for sports, government data releases such as BLS and BEA for economics, and NOAA or the National Weather Service for weather.

For your schema, the consequence is that the resolution source is a structured input you can read before the event, not only after. Practitioners who work with Kalshi rule datasets point to two fields in particular: rules_primary and rules_secondary. Read both. The primary rule usually states the trigger; the secondary text often carries the edge cases, such as what happens if a source is revised, delayed or unavailable.

The practical difference between the two venues:

Source-dependent, not exchange-dependent

The most common modeling mistake is assuming that “will X happen” means the same thing everywhere. It does not. Resolution is source-dependent: the same real-world event can settle differently on two venues if one names an official release and the other names a news report, or if the cutoff times differ by hours.

A few places where this bites:

• Cutoff semantics. “By December 31” can mean 11:59 p.m. in one time zone on one venue and a UTC boundary on another.

• Revisions. An economic figure revised after the first release may or may not change the settlement, depending on the rule text.

• Ambiguity clauses. Some rules specify a fallback outcome when the event is unclear or cancelled. These produce 50/50 or void outcomes that break naive win/loss labels.

• Multiple sources. A rule may name a primary source and a backup. Record which one actually triggered settlement.

Store the rule text as a hash plus the raw text, so you can detect when it changes between snapshots.

Joining resolution data across venues

Cross-venue analysis is where resolution data pays off, and where it breaks. Two problems dominate.

Identifier mapping

The same outcome can be represented differently on each venue: a token ID on one, a ticker string on another. The HKUST and NYU paper frames the fix as an abstraction layer that performs identifier resolution, linking market creation, order-level trading and oracle settlement into one coherent timeline. Without it, you cannot tell whether two markets are the same bet, and your arbitrage or divergence work will be built on false pairs. We cover the approach in detail in our guide to event deduplication with canonical IDs.

Timestamp normalization

Polymarket data typically relies on on-chain receipts, so its time resolution is block time. Kalshi uses exchange-server timestamps. The research behind the HKUST and NYU datasets names timestamp standardization as the primary technical hurdle for unified analysis. A workable approach:

1. Convert everything to UTC with millisecond precision.

2. Keep the original timestamp and its source type (block_time, server_time) in separate columns.

3. Never compare a block time and a server time at sub-second precision. Use a tolerance band that matches the coarser clock.

4. For event ordering across venues, prefer the sequence within each venue and join on coarse time buckets.

If you are building on exchange ticker feeds for the market-data side, this documentation on real-time market data from exchange tickers is a useful reference for how streaming feeds are typically structured before you add your own timestamp layer.

A minimal resolution schema

This is a starting point, not a standard. Adjust it to your pipeline.

A few habits keep this table honest. Never overwrite rows; append versions. Keep the raw payload. And keep your labels reproducible: the same query should return the same outcomes next year. Our walkthrough on versioning prediction market datasets shows one way to do that.

Using resolution data in research and bots

Calibration and backtests

Resolution outcomes are the ground truth for calibration curves: bucket final pre-resolution prices, compare to realized frequency, and measure the gap by category and venue. Two cautions. First, exclude void and split outcomes, or handle them explicitly. Second, choose the price snapshot carefully. A price taken after the proposal is not a forecast anymore, it is a bet on dispute risk.

Time-to-settlement features

The gap between close and final settlement is a feature in its own right. Longer gaps can reflect ambiguity, disputes or slow source publication. Capital locked in a market that has already effectively ended has a cost, and a bot that tracks expected time to payout can rank opportunities by annualized return rather than raw edge.

Dispute-aware execution

For an algorithmic agent, a sensible rule set might look like this:

• Flag any market whose rule text contains discretionary language and apply a wider uncertainty haircut.

• Pause or reduce size in the window between proposal and finalization unless your edge was measured inside it.

• Log every dispute event and review whether your model’s prior was too confident.

These are design suggestions for you to test, not performance claims.

Skill scoring

Trader-level analysis needs resolution data too. A wallet’s realized profit and loss is undefined until markets finalize, so skill scores that mix open and resolved positions are biased. Score on resolved positions only, and weight by market difficulty if you can.

Our perspective

We built our platform because resolution data is where venue-specific details leak into otherwise clean analysis. After working through venue data at scale, a few things stand out to us.

First, resolution belongs in the same schema as trades and orderbooks, not in a side table maintained by hand. When a settlement event sits on the same timeline as the fills that preceded it, questions like “who was buying into the challenge window?” become simple queries. Our data layer covers roughly 1.5 terabytes of history across Polymarket, Kalshi and Limitless, and we describe how the resolution side is organized in our guide to normalized resolution data.

Second, neutrality matters. A data layer tied to one exchange will tend to inherit that exchange’s assumptions about what a market is. A venue-neutral layer has to expose the differences, such as source dependence and timestamp types, rather than hide them.

Third, most resolution errors we expect researchers to make are not statistical. They are join errors: two markets treated as one, or one outcome labeled with another venue’s rule. Fix identifiers first, then model.

— Dean

Working with resolution data through an API

If you would rather not maintain three parsers and a mapping table, our Data API exposes cross-venue market, trade and resolution data through a single integration. Pricing is not published on our site, so check the Data API page for current access options. For related build guides, see our notes on orderbook data for developers and OHLCV candles.

Pre-launch checklist

Before you trust a resolution dataset, whether you built it or bought it:

1. Spot-check twenty resolved markets against the venue’s own page, including rule text.

2. Confirm that void, split and cancelled outcomes are represented, not silently dropped.

3. Verify that timestamps carry a type and a time zone.

4. Test that a rule edit creates a new version instead of overwriting.

5. Reconcile the count of markets per venue against a second source.

6. Check that disputed markets show their dispute timing, not only the final answer.

FAQ

What is market resolution data?

It is the structured record of how a prediction market ended: the final outcome, the rules and source used, the timing of each settlement step, and any dispute. It is the ground truth for backtests, calibration studies and payout logic.

How long does Polymarket resolution take?

If no one disputes a proposal, the market finalizes after a 2-hour challenge window, according to the UMA documentation. A dispute adds a debate period of about 24 to 48 hours and a token-holder vote of roughly 48 hours. Always check the specific market, since rules and timing can differ.

Do Polymarket and Kalshi resolve the same event the same way?

Not necessarily. Polymarket mostly relies on an oracle process, while Kalshi names source agencies in contract terms and determines outcomes itself. Even for similar events, cutoffs and sources can differ, so compare rule text before treating two markets as equivalent.

Why is it hard to combine resolution data across venues?

Identifiers and clocks differ. The same outcome can have different IDs on each venue, and on-chain block time is not comparable to exchange-server time at fine precision. Identifier mapping and timestamp normalization are the two steps that make a unified timeline possible.

Sources

• How does UMA work? | UMA Documentation

• Unlocking the Forecasting Economy: A Suite of Datasets for the Full Lifecycle of Prediction Markets (arXiv)

• How Contracts and Dispute Resolution Work at Prediction Markets

• Real-time Market Data with Exchange Tickers | Darkbot Documentation

Market Metadata in Prediction Markets: A Schema Guide

Market metadata is the descriptive layer around a prediction market contract: its identifiers, question text, outcomes, lifecycle status, timestamps, category, and the rules that decide how it resolves. It is not prices or trades. It is the dimension table that tells your code what a contract is, whether it can trade, and how and when it will pay out. If you build bots, research pipelines, or AI agents on Polymarket and Kalshi, getting this layer right matters more than most people expect, because every join, filter, and execution rule depends on it.

Assymetrixdata.assymetrix.comWork with Unified Market DataAssymetrix brings prediction market data from Polymarket, Kalshi, and Limitless into one intelligence layer through a single API integration.Explore the Data API

TL;DR

• Metadata is the dimension table of a prediction market: identifiers, status, timestamps, outcomes, category, and resolution rules. Price and trade data hang off it.

• Venues model the same idea differently. Polymarket keys markets by a condition ID and outcome token IDs. Kalshi keys them by tickers and event tickers, with its own status strings.

• Normalize status into a small shared lifecycle, and keep the raw venue value next to it so you can audit mapping errors.

• Prioritize resolution fields (source, timing, determinacy) over titles and descriptions. Titles are written for people, not programs.

• Store metadata as versioned snapshots, because status, close times, and resolution details change after you first see a market.

Table of Contents

• What counts as market metadata

• How Polymarket and Kalshi describe a market

• Why resolution fields outrank titles

• Normalizing status into a shared lifecycle

• Canonical IDs: the join key that holds everything together

• A minimal normalized schema

• Using metadata in live trading systems

• Our perspective

• Getting normalized metadata without building it yourself

• Common mistakes to avoid

• FAQ

• Sources

What counts as market metadata

A prediction market contract has three kinds of data. Trade data records what happened: fills, quotes, order book depth. State data records what the market is right now: last price, open interest, volume. Metadata records what the market is: a stable description that changes rarely, but matters when it does.

For a binary market, the useful metadata falls into six groups:

1. Identity. The venue’s native IDs and any cross-venue key you assign.

2. Question and outcomes. Title, subtitle or description, and the labels for each outcome.

3. Lifecycle. Status, creation time, open time, close time, expiration, and resolution time.

4. Classification. Category, tags, event grouping, and whether a market belongs to a multi-outcome event.

5. Resolution. Who or what decides the result, the rule text, and the final outcome once known.

6. Mechanics. Price units, tick size, whether the contract can close early, and settlement timers.

Most developers start with titles and categories because those are visible on the venue’s website. That is the wrong priority, and the sections below explain why.

How Polymarket and Kalshi describe a market

The two largest venues in the US conversation structure metadata differently, and your schema has to absorb both.

Polymarket

Polymarket’s market metadata is served through its Gamma API. In one developer walkthrough of the API, a market is described as the basic tradable unit, tied to an on-chain condition identified by a conditionId, with a pair of outcome tokens identified by clobTokenIds for binary markets. That split matters. The condition ID identifies the question. The token IDs identify each side you can actually trade on the order book. Pricing endpoints take the token ID, so a pipeline that stores only the condition ID cannot fetch a quote without a second lookup.

The same walkthrough notes that market listings can be filtered by tag, by a closed flag, and by condition IDs. Note what that implies: the venue’s native status filter is a boolean-style flag, not a rich lifecycle.

Kalshi

Kalshi is organized around tickers. Its Get Markets reference shows a market object carrying a ticker, an event_ticker, a market_type, title and subtitle fields, and a set of timestamps: created, open, close, expiration, and latest expiration. It also exposes a settlement_timer_seconds field, a result field, a can_close_early flag, and price fields in both cents and dollar strings. The listing endpoint documents a status filter with values for unopened, open, closed, and settled markets.

Two details trip people up. First, Kalshi prices appear in more than one unit, so a normalizer must pick one (we suggest decimal probability between 0 and 1) and convert at ingestion. Second, the status values accepted by the filter are not necessarily the full set of values you will see on returned objects, so treat status as an open vocabulary and log anything unrecognized rather than crashing on it.

Side by side

Treat this table as a starting map, not a contract. Venues add and rename fields, so verify against current documentation before you hard-code anything.

Why resolution fields outrank titles

The academic literature has a name for the core problem. The SoK on market microstructure for decentralized prediction markets surveys the history of these markets, and the concept of settlement legibility, meaning how clearly and verifiably a market’s outcome can be determined, is central to how these designs are discussed. For a data engineer, the practical reading is simple: a market is only as machine-usable as its resolution is legible.

Titles and descriptions are written for human discovery. They are fine for a search box. They are poor keys, because they get edited, they are ambiguous across venues (two markets titled almost the same can resolve on different sources or different dates), and they say little about how a dispute would be handled.

So the fields we would rank highest for any trading or research pipeline are:

• Resolution source. Which oracle, data feed, or authority settles the market.

• Resolution rule text. Stored verbatim, hashed, and versioned.

• Close time versus expiration time. These are different things, and the gap is where bots lose money.

• Final outcome and resolution timestamp. Needed for backtests that score predictions.

• Settlement delay. How long between the event ending and funds becoming claimable.

Two markets that look identical on price can have very different risk once you read these fields. A cross-venue spread that looks like free money may exist precisely because one contract resolves on a stricter rule than the other. That is the main reason cross-venue arbitrage is a metadata problem before it is a pricing problem.

Normalizing status into a shared lifecycle

The most common bug in multi-venue prediction market code is a status mismatch. A bot sees one venue’s “closed” and assumes the market is settled, or sees the other venue’s “settled” and does not realize a market can be closed but unresolved for hours or days.

A robust approach is a small shared lifecycle with three or four states, plus the raw value:

Store three columns, not one: status_raw (exactly what the venue sent), status_unified (your mapping), and status_mapped_at or a mapping version. When a venue introduces a new status string, you will see rows where the mapping is null instead of silently wrong values.

A few mapping rules we have found worth writing down:

• Never infer resolved from a timestamp. A close time in the past means trading should be halted, not that an outcome exists. Require a recorded outcome.

• Treat archival as a visibility flag, not a lifecycle state. An archived market may be resolved, or it may be an abandoned one.

• Handle early closure. Where a market can close before its scheduled time, the scheduled close time is a forecast, not a fact. Update it when the venue changes it.

• Keep a terminal state for voided or canceled markets if the venue supports them, so they are not scored as normal outcomes in a backtest.

If you are designing the lifecycle layer as part of a wider system with registries and monitoring, the same discipline appears in model lifecycle management practices for regulated teams: version every artifact, record who changed what, and keep an audit trail you can replay.

Canonical IDs: the join key that holds everything together

A unified metadata table needs one key that survives across venues and across venue-side renames. We call it a canonical ID. The design rules are:

• It is generated by you, not derived from a title.

• It maps one-to-many to venue-native identifiers (condition ID plus token IDs on one side, ticker on the other).

• It is stable when the venue edits the title or description.

• Equivalence across venues is a separate table. Two contracts on the same real-world question get linked by a reviewed relationship with a confidence level and a note on resolution differences, rather than being merged into one row.

That last point is where most naive aggregators fail. “Same question” is a judgment about resolution rules, not about wording. Keep the equivalence link separate and annotate it with the differences you found: source, timing, edge cases. We cover the identifier design in more depth in our guide to canonical IDs for cross-venue prediction market data.

A minimal normalized schema

Here is a compact starting point for a market dimension table. It is deliberately small.

The last two columns are the ones people skip. Metadata changes: close times move, statuses advance, rule text gets clarified. If you overwrite rows in place, your backtest will use information that was not available at the time. Store each version with a validity window and query it point-in-time. Our write-up on reproducible dataset versioning for prediction markets goes through the workflow.

Using metadata in live trading systems

Metadata earns its keep at execution time. Before a bot sends an order, a short pre-trade check against the market dimension table can stop a surprising number of mistakes:

1. Is status_unified equal to open?

2. Is the current time before closes_at, with a buffer for markets that can close early?

3. Has rules_text_hash changed since the strategy was approved?

4. For a cross-venue pair, does the equivalence link still carry an acceptable resolution-difference flag?

5. Is the price in the unit your strategy expects?

The same logic applies when you compare a backtest to live behavior. Gaps between simulated and live fills are well documented in other trading domains, and a verified comparison of backtest versus live trading results is a useful reminder to measure your own slippage rather than assume it. In prediction markets, stale metadata is one of the quieter causes of that gap: the backtest thought a market was tradable when it had already halted.

For AI agents, the metadata layer is even more important. An agent reading a market title may form a confident view about the wrong contract. Give the agent structured fields (resolution source, close time, unified status, outcome definitions) as part of its tool output, not just the headline question. We describe that pattern in how AI agents use prediction market data to trade autonomously.

Our perspective

We index trading activity across Polymarket, Kalshi, and Limitless, and the same lesson keeps repeating: the hard part of cross-venue data is rarely volume. It is meaning. Prices are numbers and they join easily. Metadata is a set of claims about what those numbers refer to, and those claims differ between venues in ways a title will never reveal.

That is why we treat canonical IDs, normalized status, and resolution fields as first-class data instead of decoration. A venue-neutral layer is only useful if it keeps the differences visible. Normalization should compress the mess into a schema your code can use, but it should never hide the raw value or the resolution quirk that makes two similar markets non-equivalent. When a mapping has to be judgment, we want the judgment recorded, versioned, and reversible.

Our advice to anyone building their own pipeline: spend your first week on the dimension table, not the price stream. Every later mistake traces back to it.

— Dean

Getting normalized metadata without building it yourself

If you would rather not maintain venue mappings, our Data API provides cross-venue prediction market data for Polymarket, Kalshi, and Limitless through a single integration, with normalized market schemas and historical depth for backtests. For a Python workflow, start with the Python prediction market data guide.

Common mistakes to avoid

• Joining on titles. They change and they collide.

• Collapsing status into a boolean. Closed and resolved are different states with different risk.

• Overwriting metadata in place. You lose point-in-time correctness.

• Merging equivalent-looking markets. Link them and annotate the rule differences instead.

• Mixing price units. Convert once at ingestion and record the raw unit.

• Ignoring early-close flags. A scheduled close time can move.

FAQ

What is market metadata in a prediction market?

It is the descriptive data about a contract rather than its trading activity: identifiers, question text, outcomes, status, timestamps, category, and resolution rules. Prices and trades reference it through keys such as a condition ID, a token ID, or a ticker.

Why can’t I just use market titles to match Polymarket and Kalshi contracts?

Titles are written for people and are often edited, and similar wording can hide different resolution sources or dates. Match markets through reviewed equivalence links between canonical IDs, and record any differences in resolution rules next to the link.

How many status values should a unified schema have?

Three or four is enough for most trading systems: upcoming, open, closed, and resolved. Keep the venue’s raw status in a separate column so you can audit the mapping and catch new values when a venue adds them.

Which metadata fields matter most for arbitrage?

Resolution source, resolution rule text, close and expiration times, and settlement timing. These determine whether two contracts are truly the same bet and how long capital stays locked, which is what separates a real spread from a mismatched pair.

Sources

• The Polymarket API: Architecture, Endpoints, and Use Cases

• Get Markets, Kalshi API documentation

• SoK: Market Microstructure for Decentralized Prediction Markets (DePMs)