Weather prediction market accuracy

The research brief, one page. It compares the ForecastEx prediction market’s forecasts of the daily high with 20 forecast systems at 24 US locations, by lead time, by location and system by system 30 hours ahead. — **Research brief**, one page, October 2026 Download PDF

First page of the working paper Improvement of Daily Temperature Forecasts from a Prediction Market, by Patrick T. Brown, with its abstract. — **Working paper**, EarthArXiv preprint, 33 pages, not peer reviewed doi.org/10.31223/X5522D

ForecastEx runs daily prediction markets on the high and low temperature in cities across the US. This page scores the ForecastEx prediction market against twenty alternative forecast systems, among them the National Weather Service point forecast, the National Blend of Models, the Aviation Forecast (LAMP), statistical guidance from four model families, and models and ensembles from the US, Europe, Germany, Canada, the UK, France, and Japan, including ECMWF’s AI forecasting system. Each is described at the foot of the page. Every comparison holds the city, the day, and the lead time fixed, and scores each system against the observation it is built to predict.

The working paper Improvement of Daily Temperature Forecasts from a Prediction Market (Brown, 2026, EarthArXiv preprint) reports this comparison for target days from 11 February to 1 October 2026, with its robustness tests, and the brief gives its findings on one page. This page rebuilds every day, so its numbers move on from the paper’s as days are added.

DETERMINISTIC SKILL

One value per system, scored against the recorded temperature by hours remaining before the day ends, each system against the observation it is built to predict, as in the working paper. For the ForecastEx prediction market that value is the median of its price ladder.

Mean absolute error by lead

PROBABILISTIC SKILL

The whole distribution, for every system that publishes one, the ForecastEx prediction market’s price ladder and four ensembles, scored on the same contracts by hours remaining before the day ends. CRPS scores the distribution in degrees, and the reliability diagrams show how the prices behind those scores paid.

CRPS by lead

Reliability diagrams in the paper’s two windows

CITY MAP

Compares the ForecastEx prediction market’s error with one alternative forecast system’s in each of the 24 cities, at the working paper’s four leads, as a percent improvement with its uncertainty interval.

SCORECARD GRID

Summarizes each system’s mean absolute error at the six leads of the working paper’s tables, with the market’s percent improvement over it, and the CRPS of the market and the four ensembles.

FORECAST SYSTEMS

Every system on this page. A deterministic system issues one forecast per run. A probabilistic system publishes a spread of outcomes as well as a centre, which is what lets the probabilistic skill figures score it on the same contracts as the ForecastEx prediction market. An ensemble mean is the average of an ensemble’s members, scored as one value. Grid spacing and time step are the provider’s published values for the run behind each row. Update frequency is measured from this record’s own capture where it holds the run times, and otherwise is the provider’s published schedule. The record is how far back this page scores it. Eight systems, the ECMWF IFS, NOAA GFS, DWD ICON, ECCC GEM, Met Office UM, Météo-France ARPEGE and JMA GSM models and the ECMWF AIFS, are carried back to February from Open-Meteo’s archive of previous runs before their live capture began.

System	What it is	Kind	Grid	Time step	Updates	Record
Prediction marketThe system every other row is compared against.
ForecastEx prediction market	Daily contracts on each city’s high and low temperature. The ladder of Yes prices across strikes is a probability distribution for the settle, and its central value, where the ladder crosses fifty cents, is scored as one value.	Probabilistic	The settlement station	The station-local day	Continuously, as orders arrive	—
Station MOSStatistical guidance fitted at the settlement station itself, from model output corrected against what the models got wrong there in the past. These are left as issued by the bias correction. GFS MOS, NAM MOS and the NBM station guidance appear twice, on the daytime maximum each issues and on its hourly temperature rows.
LAMP	LAMP, statistical guidance refreshed with the latest observations.	Deterministic	The settlement station itself	Hourly, to about 38 h	Every hour	—
NBM station MOS, issued maximum	The station text bulletin of the National Blend of Models, read on its own daytime maximum, the highest temperature its TXN row expects from 7 AM to 7 PM local standard time, and scored against the National Weather Service climate report on the location-days whose report puts the maximum inside that window.	Deterministic	The settlement station itself	3-hourly, to 72 h	Every 6 hours	—
NBM station MOS, hourly rows	The same bulletin read on its hourly temperature rows, the extreme of the hours left in the day, and scored against the METAR settle.	Deterministic	The settlement station itself	3-hourly, to 72 h	Every 6 hours	—
GFS MOS, issued maximum	Statistical guidance built on the GFS alone, with no forecaster editing, read on its own daytime maximum, the X/N row’s highest temperature from 7 AM to 7 PM local standard time, and scored against the climate report on the location-days whose report puts the maximum inside that window.	Deterministic	The settlement station itself	3-hourly, to 72 h	Every 6 hours	—
GFS MOS, hourly rows	The same bulletin read on its hourly temperature rows, the extreme of the hours left in the day, and scored against the METAR settle.	Deterministic	The settlement station itself	3-hourly, to 72 h	Every 6 hours	—
NAM MOS, issued maximum	Statistical guidance built on NOAA’s North American Mesoscale model, read on its own daytime maximum, the X/N row’s highest temperature from 7 AM to 7 PM local standard time, and scored against the climate report on the location-days whose report puts the maximum inside that window.	Deterministic	The settlement station itself	3-hourly, to 72 h	Every 12 hours	—
NAM MOS, hourly rows	The same bulletin read on its hourly temperature rows, the extreme of the hours left in the day, and scored against the METAR settle.	Deterministic	The settlement station itself	3-hourly, to 72 h	Every 12 hours	—
DWD MOSMIX	MOSMIX, which combines and corrects the German and European models station by station.	Deterministic	The settlement airport’s station in DWD’s list, which lacks Buckley Field, Houston Hobby and Chicago Midway	Hourly, to 240 h	Every 6 hours	—
BlendA calibrated blend of many models on a fine grid.
National Blend of Models, METAR	NOAA’s calibrated blend of many models, the starting point for the Weather Service’s own forecast.	Deterministic	2.5 km	Hourly, 3-hourly after 36 h	Every hour	—
National Blend of Models, URMA	The same Blend scored a second time, as issued, against URMA, the Weather Service’s gridded analysis, at the grid box holding the station, its daily value against the largest analysis over the station-local day. Highs only.	Deterministic	2.5 km	Hourly, 3-hourly after 36 h	Every hour	—
Human-editedForecasters editing model guidance with their own judgment.
National Weather Service, climate report	The official public forecast from the National Digital Forecast Database, which forecasters edit starting from the National Blend of Models. Its high is a daytime maximum for 7 AM to 7 PM local standard time and its low an overnight minimum for 7 PM to 8 AM, assigned to the morning the night ends, while the contracts settle on the calendar day.	Deterministic	2.5 km	Hourly for the first 36 h	About twice a day, amended as needed	—
National Weather Service, URMA	The same forecast scored a second time, as issued, against URMA, the Weather Service’s gridded analysis, at the grid box holding the station, its daytime maximum against the largest analysis over 7 AM to 7 PM local standard time. Highs only.	Deterministic	2.5 km	Hourly for the first 36 h	About twice a day, amended as needed	—
Artificial intelligenceModels trained on decades of past weather rather than stepping physical equations forward.
ECMWF AIFS ens. mean	AIFS, ECMWF’s machine-learning forecasting system. Live, it is the 51-member ensemble. Its single value is the extreme of the ensemble’s hourly means over the hours left in the day, the median of its distribution, and the members’ spread gives the distribution. Before live capture began the forecast archive holds its single deterministic run.	Probabilistic	0.25°, about 25 km	6-hourly	Every 6 hours	—
Raw numerical weather predictionPhysics models that step the atmosphere forward in time. Their output applies to a grid box rather than the exact point where a station sits, which the bias correction addresses.
NOAA HRRR	NOAA’s High-Resolution Rapid Refresh, covering the contiguous United States.	Deterministic	3 km	Hourly	Every hour	—
ECMWF IFS	ECMWF’s global model, the IFS, as one deterministic run. Before live capture began the record uses ECMWF’s 0.25° open data from the forecast archive, and from then on a data provider’s high-resolution series, whose grid spacing could not be confirmed.	Deterministic	0.25°, about 25 km, in the archive, not confirmed for the live series	3-hourly in the archive	Every 6 hours	—
ECMWF IFS ens. mean	Described by the data provider as the average of ECMWF’s 51-member ensemble, which could not be independently confirmed. Only its mean is used here.	Ensemble mean	0.25°, about 25 km	3-hourly	Every 6 hours	—
NOAA GFS	NOAA’s Global Forecast System. Before live capture began the record comes from a forecast archive that returns HRRR within 48 hours of the run.	Deterministic	0.11°, about 13 km	Hourly, 3-hourly after 120 h	Every 6 hours	—
NOAA GEFS median	NOAA’s Global Ensemble Forecast System, 31 members. Its median, the extreme of the hourly ensemble means over the hours left in the day, is scored as one value beside the systems, and its spread as a distribution.	Probabilistic	0.25°, about 25 km	3-hourly	Every 6 hours	—
DWD ICON	ICON, the global model of Germany’s weather service.	Deterministic	0.1°, about 11 km	Hourly, 3-hourly after 78 h	Every 6 hours	—
DWD ICON-EPS median	ICON’s global ensemble, 40 members. Its median is scored as one value beside the systems, and its spread as a distribution.	Probabilistic	26 km	Hourly	Every 12 hours	—
ECCC GEM	GEM, the model of Environment and Climate Change Canada, as a seamless series that uses its regional and high-resolution configurations at US stations.	Deterministic	10 km or 2.5 km	Hourly	Every 6 hours	—
ECCC GEPS median	Canada’s global ensemble, 21 members. Its median is scored as one value beside the systems, and its spread as a distribution.	Probabilistic	0.25°, about 25 km	3-hourly	Every 12 hours	—
Met Office UM	The Met Office Unified Model, run globally.	Deterministic	0.09°, about 10 km	Hourly, 3-hourly after 54 h	Every 6 hours	—
Météo-France ARPEGE	ARPEGE, the global model of Météo-France.	Deterministic	0.25°, about 25 km	Hourly, 3-hourly after 2 days	Every 6 hours	—
JMA GSM	The Global Spectral Model of the Japan Meteorological Agency.	Deterministic	0.5°, about 55 km	6-hourly	Every 6 hours	—

OTHER DATA SOURCES

Data	Source	Notes
Observed temperature	aviationweather.gov METAR	Hourly reports plus special reports. The contracts settle on METAR observations, so this is the matching feed. The Iowa State Environmental Mesonet supplies the same reports for earlier dates.
NWS climate report	National Weather Service daily climate reports	The report’s daily high and low, the truth in the climate-report frame. Buckley Field has no report of its own and is left out of that frame.
URMA analysis	NOAA Unrestricted Mesoscale Analysis on NOAA Open Data	The gridded analysis the National Weather Service verifies its forecast and the National Blend of Models against. The page reads its hourly 2 m temperature at the 2.5 km grid box holding each station and scores those two systems a second time against it, on the daily high.
Contract prices	ForecastEx public market data and its published trade archive	Every order is a bid to buy Yes or a bid to buy No, and the two sides of a contract sum to one dollar. This page reads a contract’s Yes price as the midpoint of the Yes bid and one dollar less the No bid, never fee adjusted.
Model forecast archive	Open-Meteo	Past runs of the models Open-Meteo carries, used under its CC BY 4.0 licence.
Statistical guidance archive	Iowa State Environmental Mesonet	Past MOS and LAMP bulletins, each as it was issued.
Basemap	US Census Bureau boundaries	The state outlines under the city map. Public domain.