Prediction Performance

Part of the Paulopus case study. The product graded a call on all 104 matches of the 2026 World Cup — though four now carry a verdict with no prediction on file. This focus is the accountability layer: what the calls were actually worth once real results came in. Every number below is drawn from performance-dataset.json, the frozen record of 104 graded matches that backs the karma page.

A four-tier verdict, not a coin flip

Decision. A binary "right/wrong" would have thrown away most of what a scoreline prediction contains. Getting the winner right but the margin wrong is a different quality of call than getting the loser right, and both differ from nailing the exact score. So the grading uses four verdicts instead of two.

Approach. Each graded match carries a verdict and a signed points value. Reading the point weights straight off the dataset gives the scoring rule:

Artifact. The weighting is self-consistent against the tournament totals: 14 × 1.5 + 39 × 1 + 16 × 0 + 35 × −1 = 25, which is exactly the stored netKarma of +25 across 104 graded matches (ungraded: 0).

Result. Net karma landed positive but modest — the scale is deliberately punitive, and a +25 finish over a full tournament means the good calls carried real weight against a heavy RedCard tax.

Where the 104 verdicts fell

Decision. The headline anyone asks first is "how often was it right?" — so the distribution needed to be legible at a glance, not buried in the match log.

Approach. Counting verdicts across all 104 graded matches and expressing each as a share of the total turns the raw counts object into a hit-rate picture.

Artifact.

Result. Perfect and Inspired together — the calls that scored positive karma — account for 53 of 104, a hair over half. RedCards alone make up a third of the tournament. The product was right about the outcome more often than not, but it paid for its confidence: predicting decisive scorelines that finished level was the single largest failure mode.

The fourteen exact hits

Decision. Perfect verdicts are the marketing surface of the whole project — the moments the model called a scoreline to the goal. They earned their own list.

Approach. The notableCalls array names all 14 Perfect matches. Cross-referencing each ID against its match record recovers the scoreline it nailed.

Artifact.

Result. The exact hits skew toward the knockout rounds: eight came in the group stage, six from the round of 32 onward, including the France–Morocco quarter-final (M097, 2-0) and the England–Argentina semi-final (M102, 1-2). The product got sharper, not looser, as the field narrowed.

Where the calls went red

Decision. RedCards are the honest part of the record. Suppressing them would make the karma total meaningless, so the misses get the same treatment as the hits.

Approach. The notableMisses array lists all 35 RedCard matches. A pattern emerges when the predicted scoreline is set beside the actual: the model repeatedly staked a decisive result on a match that finished level.

Artifact. A representative slice (predicted → actual):

Result. The recurring wound is the confident home win that never arrived — Spain and England both held to goalless draws they were tipped to win 3-0. The knockout rounds bit hardest: the France–Spain semi-final flipped entirely (M101), and the third-place match (M103) detonated into a 4-6 that no 2-1 prediction could survive.

Streaks: momentum in both directions

Decision. Aggregate counts hide sequencing. A tournament that alternates hit-miss-hit reads very differently from one that runs hot then cold, so the record tracks run length in both directions.

Approach. The streaks object stores the two extremes: the longest unbroken run of positive-karma verdicts and the longest run of consecutive RedCards.

Artifact.

Result. At its best the product strung together 10 consecutive positive-karma calls; at its worst it went 4 matches red in a row. The asymmetry — a peak streak more than double the worst slump — is what kept net karma above water despite RedCards claiming a third of the tournament.

The four calls with no prediction on file

Decision. Honesty about the data itself matters as much as honesty about the results. Four graded matches carry a verdict but no recoverable prediction, and the record says so rather than papering over it.

Approach. Matches M001, M002, M007, and M019 show "predicted": null — they appear to have lost their originating prediction when the storage moved from write-once locks to regenerable drafts. Crucially, their verdicts survived, so they remain fully counted.

Artifact.

Result. These four are part of every aggregate on this page — the counts, the +25 net karma, and the streaks all include them, which is precisely why the totals here match the product's own karma page. The predicted scorelines are gone and are not reconstructed; the grade stands on the verdict that was stored at the time.