Companion to the forecasting bakeoff scoring method. Every participant forecast ~1,000 wells; this page walks from a single month's log error to the one number that ranks them, with cartoon decline curves you can move.
Forecast A is the true decline shifted by a constant factor: the level is wrong, the shape is right. Forecast B has the wrong decline rate, with its initial rate set so its average log error equals A's. Both are equally biased on this well, yet B is the worse forecast and only the standard deviation of the log error says so. The mean μ measures level; the standard deviation σ measures shape. Press play to sweep B's decline-rate error while its average error stays pinned.
Rate vs. time (semilog)actualAB
Log error e(t) = ln(f/x)— mean▒ ±1 s.d.
The σ here is the standard deviation of the same e(t) = ln(f/x) series from section 01, for the same Forecasts A and B; the left-hand plot is section 01's log-error plot with a straight line fitted to each trace. Fitting that line splits σ exactly: σ² = slope²·Var(t) + σ²resid. The slope term is the decline-rate error. The residual is everything else that makes a forecast's shape disagree with the actuals: month-to-month noise in the data, shut-ins, curvature the forecast did not capture. A has the right shape, so its σ is residual only: the noise floor every forecast pays. B carries that same floor plus a trend term from the wrong decline rate; its residual is slightly larger than A's because a hyperbolic decline-rate error is a curve, and the part of it a straight line does not capture lands in the residual. The three sliders are shared with section 01: move the bias and both traces shift together while the slopes, residuals and bars stay where they are.
Log error e(t) with fitted lineA: level shiftB: wrong Di- - fit
σ² split: trend vs. residual■ slope²·Var(t)▒ residual
Each participant forecasts the same wells (40 here, ~1,000 in the bakeoff). Each well yields one μi and one σi, so a participant is a distribution of μ and a distribution of σ. Three numbers summarize them. Level, the median of μi: are you typically high or low? Consistency, the standard deviation of μi: how much does that level vary from well to well? Shape, the median of σi: how well do your decline shapes match? Medians are used where a few badly behaved wells should not dominate ~1,000. Within-well σ is held at 0.08 for every well below so that only level varies and Level and Consistency can be seen on their own.
The three numbers are not an arbitrary choice. Pool every log error across every well and square-average it: the total splits exactly into the same three pieces (population variances, equal points per well). The score reports each piece on its own, using medians so that ~1,000 wells are summarized by their typical behaviour rather than their worst, and so that the ranking shows why a participant placed where they did.
Dividing each component by its median across participants makes it unitless and centres the field on 1.0, so a value of 1.3 means "30% worse than a typical entry" whichever component it is. Once the three are on that common scale there is no remaining reason to prefer one over another: each describes a different way a forecast can fail, so each gets one third. The table uses invented participants. The sliders are there to show how little or how much the ranking depends on the weights; the reference column is the method as adopted.