We calculate error as latitude-weighted RMSE. RMSE = root mean squared error: a measure of how far off the forecast is from the truth where we take the difference between prediction and truth for each point, square those, take the mean, and take the square root again. This rewards being close to the truth and penalizes differences more the larger they get. We weight the error by the cosine of the latitude of the point, so that points near the equator are weighted more heavily than points near the poles, as is standard in the weather forecasting community.

For our source of "truth", we use ERA5: a dataset widely regarded as the world's best guess at what weather actually occurred around the globe. We also internally validate against observations at weather stations, as well as observations collected by our own balloon constellation; these results are consistent with those from the ERA-5 comparison.

We compared to ECMWF models, both the deterministic model (HRES) and the ensemble (ENS), as they are generally-accepted as the best operational models. We also validate against other operational models, such as GFS and AIFS, which will be added to this page in the future.

Want more details on our models? Check out our blog for more details on our methodology and results. Interested in using WeatherMesh or our atmospheric data? Contact us!