How the ratings are scored.

How every wallet in the category gets a public accuracy score you can recompute yourself.

This is the reference for two ratings, each on a 0–1000 scale:

• Foresight Rating — grades forecasts: what you bet and held until the question resolved. The world grades you.
• Trader Rating — grades trades: what you bought and sold before resolution. The market grades you.

They answer different questions and are shown as two separate lenses on a profile. A wallet can be a great forecaster and a mediocre trader, or the reverse. Neither is derived from self-reported anything: both are computed from the public HIP-4 trade tape (which carries buyer/seller on every fill) plus the settled outcome of each question.

Every constant below is quoted from the scoring core, crates/metrics/src/score/ — the six rating/eligibility constants live in mod.rs, and the structural-truth tolerance PARTITION_TOL in truth.rs. That one module set is the single source of truth, so the doc, the batch, and the anti-gaming tests never drift: where a number appears here it is the literal value in that code.

Part 1 — Foresight Rating (the forecaster's score)

The unit of scoring: an observation

We never look at PnL. A forecast is a position you were still holding when the question resolved. To recover positions from a public tape of fills we net each wallet's trades:

• Per (wallet, coin), buys push FIFO lots (sz @ px); sells consume the oldest lots first. Each surviving lot keeps its own entry price and time — no cost-basis averaging, so a wallet that scaled in at 30¢ and again at 55¢ contributes two separate observations.
• A position fully closed before resolution leaves nothing to grade. That is deliberate: market makers and wash traders net flat and simply fall out of the forecaster score (they are measured elsewhere — see Known limitations).
• A surviving lot at the moment its coin resolves is one observation:

> { p = lot entry price, o ∈ {0,1} realized outcome, w_stake = p·sz notional }

p is read as a probability: paying p per share for a coin that pays 1 is a p-confident bet that the outcome happens. o is what actually happened.

Ground truth (o)

The hard part is knowing o, because Hyperliquid drops a settled coin from both allMids and candleSnapshot, and outcomeMeta says only which outcomes settled, never to what. We recover the 0/1 value from three sources, most-authoritative first:

1. Curated (TD_RESOLUTIONS): hand-entered winners for markets the venue no longer serves.
2. Structural: in a single-winner question, the active non-fallback Yes-mids form a distribution that sums to ≈1 (World-Cup live: 0.418 + 0.580 ≈ 1.0, tolerance PARTITION_TOL = 0.06). Any already-settled sibling of an in-progress winner has therefore resolved No — a team resolves No the instant it is eliminated. The champion needs source (1).
3. Settlement journal (watcherd): records which outcomes settled after the live catalog drops them, so the offline batch can still grade dead coins; graded via source (1).

Fallback ("void") outcomes are never graded — a void is not a forecast.

Weighting: not every forecast is worth the same

Each observation carries a composite weight w = w_stake · w_unc:

• Stake weight w_stake = p · sz — the notional you put behind the call. Skin in the game. A $5,000 conviction bet counts more than a $10 dabble.
• Uncertainty weight w_unc = p·(1 − p) — how informative the call was. A coin-flip (p = 0.5) carries the maximum w_unc = 0.25; a near-certainty (p = 0.98) carries almost none (0.0196). Being right about a 98¢ foregone-conclusion is nearly free information, so it barely moves your score — this is also what defuses the "farm a thousand near-certainties" attack (see Anti-farming).

The wallet aggregate: weighted Brier

A wallet's raw score over a window is the weighted Brier score:

B_w = Σ w·(p − o)²  /  Σ w          w = w_stake · w_unc

The Brier score is just mean squared error between forecast probability and outcome. Lower is better; 0 is perfect. It is a strictly proper scoring rule: your expected score is minimized only by reporting your true probability, so it cannot be gamed by shading forecasts toward the extremes.

Effective sample size — because observations are weighted, "how many forecasts is this really worth?" is not the raw count. We use Kish's effective sample size:

n_eff = (Σ w)²  /  Σ w²

A wallet whose weight is concentrated in one giant bet has n_eff ≈ 1; a wallet with many comparable bets has n_eff near its observation count. n_eff drives both eligibility and shrinkage.

Why 0.25 is the line

Answer 0.5 to every question and every term becomes (0.5 − o)² = 0.25, so B = 0.25 exactly. That is the coin-flip baseline (BASELINE = 0.25): a forecaster who knows nothing and admits it. Skill is beating it. The rating maps the shrunk Brier onto 0–1000 by how far below the coin flip you land:

rating = round( 1000 · max(0, 1 − B_shrunk / 0.25) )

B_shrunk = 0 (perfect) → 1000. B_shrunk = 0.25 (coin flip) → 0. Worse than a coin flip is clamped to 0 — the scale measures skill, not how badly you can lose.

Shrinkage: small samples don't get to be #1

A wallet with three lucky calls should not top a wallet with three hundred careful ones. Before rating, we pull the raw Brier toward the population prior B_pop (the weighted Brier of the entire pool) with an empirical-Bayes shrink:

B_shrunk = (n_eff · B_w + k · B_pop) / (n_eff + k)        k = SHRINK_K = 20

Read k = 20 as "you need about 20 effective forecasts before your own record outweighs the crowd's." A wallet with n_eff = 20 is graded 50/50 on itself vs the pool; at n_eff = 200, ~91% on itself. New wallets regress to the mean instead of spiking to the top on noise.

Eligibility & percentile

A wallet joins the leaderboard only if:

n_eff ≥ MIN_N_EFF (10)   AND   Σ w_stake ≥ MIN_STAKE ($100)

Below that, a wallet still has a rating (heavily shrunk) but is not ranked — too little signal to compare fairly. The "Top X%" chip is the percentile within the eligible cohort: the share of eligible wallets with a strictly lower rating (0–100).

Murphy decomposition (what a Brier score is made of)

Brier decomposes into three intuitive pieces (Murphy 1973):

Brier = Reliability − Resolution + Uncertainty

• Reliability (calibration) — when you say 70%, does it happen 70% of the time? Lower is better. This is the reliability curve on your profile: bucket forecasts by p, plot predicted vs realized. On the diagonal = perfectly calibrated.
• Resolution — do you say different things about different outcomes, and are you right to? Higher is better; it rewards confident calls that pan out. A forecaster who says 50% to everything has zero resolution.
• Uncertainty — the irreducible difficulty of the questions themselves (ō·(1−ō)), the same for everyone facing the same slate; not something you control.

The takeaway: you improve your rating either by being better calibrated (reliability) or by making sharper correct calls (resolution) — and the w_unc weight means the score cares most about the genuinely uncertain questions, where forecasting skill actually lives.

Anti-farming rules (and why they hold)

The adversarial audit (B8) turns each row into an executable test on synthetic fixtures.

Known limitations (stated plainly)

• Off-book pair mints are invisible. HIP-4 mints a Yes/No pair off the trade tape; only subsequent secondary trades appear. A wallet that mints a pair and sells one side looks, on the tape, like it acquired the other. Net exposure is right; the entry price of a minted leg can be off. Documented, not hidden.
• Oversell clamps to zero. Because of the above, a wallet can appear to sell more than it visibly bought; the residual is clamped to a flat position (no negative "forecast").
• Makers fall out by design. A market maker who quotes both sides and net flattens leaves no surviving lot, so Brier says nothing about them — correctly. Makers are graded on markout and uptime (adverse selection and liquidity), not forecast accuracy. Two different jobs, two different scoreboards.

Worked example

One observation. You bought a coin at p = 0.9 and still held it when it resolved No (o = 0). Its contribution to your Brier is (0.9 − 0)² = 0.81 — a confident wrong call is punished hard. (This is the side_expired_to_zero_is_graded_loss test in B1; a single such observation gives B_w = 0.81, n_eff = 1.)

Two observations, one question (the cap is a no-op for a single question):

B_w   = (14.4·0.16 + 12.6·0.09) / (14.4 + 12.6) = 3.438 / 27  = 0.1273
n_eff = 27² / (14.4² + 12.6²)   = 729 / 366.12               = 1.99

Aggregate → rating. Take an eligible wallet with B_w = 0.10, n_eff = 20, in a pool with B_pop = 0.20:

B_shrunk = (20·0.10 + 20·0.20) / (20 + 20) = 6 / 40 = 0.15
rating   = round( 1000 · (1 − 0.15/0.25) ) = round(1000 · 0.4) = 400

A coin-flip forecaster (p = 0.5 on everything) lands at B = 0.25 → rating 0. Every number here is reproducible from a wallet's own receipt (/v1/score/{addr}/receipt lists every lot, p, o, and weight).

Part 2 — Trader Rating (the market's score)

Foresight grades only what you held to resolution. Someone who bought Yes at 30¢ and sold at 60¢ before the game ended is invisible to it — and if the outcome landed No, Foresight would even call the "forecast" wrong. But that trade made real money. The forecaster is graded by the world; the trader is graded by the market. So exits get their own rating, and it is emphatically not Brier.

Round-trip return

The netting pass emits trips as a side output: each FIFO buy lot matched against a later sell — (buy p → sell q, s shares). An unsold remainder still held at resolution becomes a trip with q := o (the market's final verdict is your exit). For each trip:

profit = (q − p)·s          cost = p·s          (fees reserved, 0 today)

The wallet's window return is capital-weighted:

R_w = Σ profit / Σ cost      (in basis points)

Robustness mirrors the forecaster side: trip returns are winsorized to the pool's p1/p99, one question may not exceed 30% of Σ cost, trips below $5 are dropped, self-cross tids are removed, and n_eff is Kish-weighted by cost.

Shrink to zero (not to the pool)

The prior for a trader is "you have no edge" — flat return — not the population mean. So we shrink toward zero:

R̂ = R_w · n_eff / (n_eff + k)        k = SHRINK_K = 20 (same constant)

Thin samples collapse toward "no demonstrated edge," which is the honest default.

Percentile mapping

Absolute basis points are not comparable across market regimes (a bull run flatters everyone). So the rating is a percentile among eligible traders (n_eff ≥ 10, Σ cost ≥ $100):

Trader Rating = round( 10 · percentile(R̂) )      → 0–1000

Timing edge (markout)

A companion diagnostic, not part of the rating: the notional-weighted price move after your fill,

markout(h) = (mid_{t+h} − px) · dir / px      h ∈ {30s, 5m, 1h}, winsorized

Positive markout means the market moved your way right after you traded — good timing. The same function measures a maker's adverse selection in the rebate reporting: one metric, two readers. When no mid exists at horizon h the observation drops and coverage is reported, never silently assumed.

Reproduce it yourself

Nothing here is a black box. The scoring math is a pure library (crates/metrics/src/score/, unit-tested), the batch that runs it over the archive is scorebatch, and every rating is backed by a receipt of the exact lots that produced it. The constants:

We don't ask for trust; we hand you the arithmetic.

Краткое резюме (RU)

Два публичных рейтинга по шкале 0–1000, оба считаются из публичной ленты сделок HIP-4 (в каждом филле есть buyer/seller) — никаких самоотчётов.

• Foresight Rating — точность прогнозов: то, что вы держали до резолюции. Позиции восстанавливаются FIFO-неттингом ленты; закрытые до резолюции — не грейдятся (мейкеры и wash-трейды выпадают сами). Наблюдение = оставшийся лот на момент резолюции: p (цена входа как вероятность), o ∈ {0,1} (исход). Вес w = w_stake·w_unc, где w_stake = p·sz (нотионал), а w_unc = p(1−p) (монетка 0.5 информативнее почти-достоверности). Оценка — взвешенный Brier B_w = Σw(p−o)²/Σw; 0.25 = монетка (BASELINE), поэтому rating = round(1000·max(0, 1 − B̂/0.25)). Малые выборки сжимаются к пулу (B̂ = (n_eff·B_w + 20·B_pop)/(n_eff+20), k = 20); n_eff — Kish. В лидерборд — при n_eff ≥ 10 и стейке ≥ $100. Анти-фарм: кап вопроса 30%, пыль < $5 вон, self-cross вон, w_unc обесценивает фарм почти-достоверностей.
• Trader Rating — скилл трейдера: прогнозиста грейдит мир, трейдера — рынок. Трип = FIFO-пара buy p → sell q; недопродажа на резолюции → q := o. R_w = Σprofit/Σcost в bp, винзоризация p1/p99, шринк к нулю («эджа нет», та же k = 20), рейтинг = round(10 · перцентиль). Markout — тайминг-эдж и adverse selection мейкера.

Ограничения честно: офф-бук минты пар невидимы в ленте (нетто-экспозиция верна, цена минтованной ноги может плыть); мейкеров меряем markout/uptime, не Brier. Все константы — из crates/metrics/src/score/mod.rs, единый источник. Каждый рейтинг пересчитывается по receipt.