Truth Tracking vs. Metaculus and Prediction Markets

Truth Tracking, Metaculus, and prediction markets all deal with uncertainty about the future. But they answer different questions and collect different kinds of evidence.

• Truth Tracking asks how public commentators performed across their historical predictions.
• Metaculus asks what a forecasting community currently estimates about defined future questions.
• Prediction markets use trading activity or market positions to produce a live probability or price signal.

These systems should not be compared as if they were competing versions of the same product. A person may reasonably use all three: a market or forecasting platform to understand current expectations, and a tracking site to evaluate the historical record of commentators who discuss those events.

Short version

Truth Tracking: evaluating what was already said

Truth Tracking begins with public media rather than a newly created forecasting question.

Its workflows collect statements from videos and other published material, transcribe or inspect the source, identify predictions with a subject, direction or target, and time horizon, then evaluate the result after the relevant deadline.

For stock content, the system records instruments such as stocks, ETFs, indexes, crypto, or commodities; direction or target; horizon; source attribution; and market-data outcomes. It separates short, medium, and long horizons and can report simulated P/L relative to SPY. Truth Tracking stock methodology

For politics, the system focuses on dated, observable claims from public commentary and preserves source links and timestamps where available. The result categories include successful, partial, failed, pending, and difficult-to-verify cases. Truth Tracking politics methodology

What Truth Tracking measures well

• whether a commentator’s own public claim matched a later outcome;
• whether a channel repeatedly made specific, time-bounded calls;
• how performance varies by channel, topic, asset, or horizon;
• whether memorable public successes are supported by a broader record;
• what was actually said in the original source context.

What it does not primarily measure

• the current crowd probability of an event;
• the probability calibration of a forecaster who gives explicit numerical probabilities;
• the price of a tradable contract;
• the forecast of a community that answers platform-defined questions.

Truth Tracking is best understood as an audit layer over public commentary.

Metaculus: structured probabilistic forecasting

Metaculus is an online forecasting platform focused on questions about the future. Its public site currently presents millions of predictions, tens of thousands of forecasting questions, a large set of forecasters, and topics including AI, geopolitics, economics, science, and current events. Metaculus

Its questions are deliberately structured. Participants may submit probabilities, numerical estimates, or distributions depending on the question. The platform can aggregate forecasts into a community estimate and evaluate forecasters using scoring and calibration concepts.

What Metaculus does well

• makes the forecast object explicit before people answer;
• captures numerical uncertainty instead of only yes/no outcomes;
• provides a community estimate for many questions;
• supports tournaments and structured forecasting programs;
• is well suited to calibration and proper scoring rules;
• creates a shared question space where forecasts can be compared more consistently.

How it differs from Truth Tracking

Metaculus generally starts with a defined question on its own platform. Truth Tracking starts with a statement made elsewhere, often in a video or broadcast.

That difference affects the evidence:

• Metaculus knows the question, resolution criteria, and forecast format in advance.
• Truth Tracking must determine whether a public statement qualifies as a prediction and reconstruct its criteria from the source.

Metaculus therefore has an advantage in formal probability measurement. Truth Tracking has an advantage when the research question concerns people who did not participate in a forecasting platform.

For example, Metaculus can answer:

“What probability does the community assign to this event?”

Truth Tracking can answer:

“What did this commentator say about the event, and how did that commentator’s comparable calls perform over time?”

Prediction markets: live collective signals

Prediction markets produce prices or probabilities through trading or market positions. Participants buy or sell claims about future outcomes, and the market signal changes as information and expectations change.

Manifold

Manifold describes itself as a social prediction-market platform where users can create and trade markets on a wide range of topics. It also publishes calibration analysis comparing market probabilities with resolved outcomes. Manifold documentation Manifold calibration

Manifold’s strength is breadth and participation. A user can create a market on a topic that may not fit a centrally curated forecasting tournament. The resulting probability is a collective market signal, not the historical record of one commentator.

Polymarket

Polymarket presents live markets across politics, sports, crypto, finance, geopolitics, technology, and other subjects. Market pages display outcome probabilities, trading volume, comments, source links, and end dates. Polymarket

Its strength is current-event breadth and market activity. Its output is useful for asking what market participants currently expect, but it does not automatically reveal which individual trader or commentator deserves credit for a forecast over time.

Kalshi

Kalshi provides event contracts tied to measurable future events. Its product is closer to a regulated event-contract market than to a retrospective public-commentary audit. Kalshi

Its strength is a structured market mechanism for event outcomes. Like other prediction markets, its primary output is a live market signal rather than a channel-level record of public pundit predictions.

What prediction markets measure differently

A market price can change continuously before an event resolves. It may incorporate new information, trader positions, liquidity, hedging, and differences in participant beliefs.

A public-commentary record usually captures a statement at a specific time. It asks whether that statement was correct under a defined horizon, not how the speaker would have traded as new information arrived.

This creates several differences:

Neither output should automatically be treated as the “true” probability. They represent different information processes.

Relative strengths and weaknesses

Truth Tracking’s relative strengths

• captures media predictions that formal platforms may never receive;
• preserves the speaker’s original language and context;
• supports long-run channel and pundit records;
• can compare stock, political, and sports commentators;
• exposes selective memory and deleted or overlooked claims;
• evaluates public claims even when no market existed for them.

Truth Tracking’s relative weaknesses

• extraction from natural speech is harder than answering a structured platform question;
• transcript and attribution errors require review;
• many public statements do not include explicit probabilities;
• domain-specific scoring is harder to compare than one standardized Brier framework;
• current records may have uneven sample sizes and coverage;
• it does not produce a continuously updated crowd probability.

Metaculus’s relative strengths

• clearer forecast definitions;
• richer probability information;
• better fit for calibration and tournament scoring;
• larger community forecasting ecosystem.

Metaculus’s relative weakness for this use case

• it does not primarily audit existing video channels or public commentators;
• coverage depends on questions created and maintained on the platform;
• its results describe participants answering platform questions, not necessarily what media personalities said elsewhere.

Prediction markets’ relative strengths

• rapid reaction to new information;
• live consensus estimates;
• incentive structures tied to positions or trading;
• broad participation and current-event coverage.

Prediction markets’ relative weaknesses for this use case

• market price is not the same as an individual’s historical forecast record;
• low liquidity can make prices noisy;
• market rules and resolution criteria can affect the result;
• participation, trading access, and legal availability vary by platform and location;
• market activity may reflect incentives beyond pure forecasting accuracy.

Which tool fits which question?

The strongest product position for Truth Tracking

Truth Tracking should not claim to replace formal forecasting platforms or prediction markets. Those systems have clear advantages for probability elicitation, market aggregation, and calibration.

Its strongest position is narrower:

Truth Tracking makes public commentary auditable after the fact.

That means preserving the original statement, identifying the time horizon, evaluating the outcome, and building a record at the channel or commentator level.

This is valuable because many influential forecasts are made in videos, interviews, podcasts, and broadcasts rather than on forecasting platforms. Without a tracking layer, those claims are often remembered selectively and are difficult to compare.

What Truth Tracking can learn from these platforms

The comparison also suggests several improvements:

• add explicit probability fields when a source provides them;
• report calibration separately from hit rate;
• use standardized scores where the underlying prediction format supports them;
• show confidence intervals or uncertainty for small samples;
• publish machine-readable data for independent analysis;
• make domain-specific scoring rules especially clear;
• distinguish a source’s original call from later updates or revisions.

These improvements would make public-media auditing more rigorous without changing its core purpose.

Conclusion

Truth Tracking, Metaculus, and prediction markets are best viewed as complementary systems.

Metaculus is designed for structured probabilistic forecasting. Prediction markets are designed to aggregate expectations through trading or market positions. Truth Tracking is designed to reconstruct and evaluate the historical claims made by public commentators.

The important question is not which system produces the single best number. It is what evidence the reader needs: a current consensus, a calibrated probability, or an auditable record of what a particular person publicly predicted.

This article is educational and is not investment, political, or sports-betting advice. Historical forecasting performance does not guarantee future results.