The ProblemAir-quality forecasts exist to warn citizens before pollution reaches dangerous thresholds. However, the machine-learning literature overwhelmingly evaluates these models using Mean Absolute Error (MAE). Because air quality is highly autocorrelated, naïve persistence models (predicting that the next hour equals the current one) achieve excellent average error scores despite being structurally incapable of predicting the onset of pollution episodes. The MethodologyThis study evaluates this metric mismatch using three years of hourly air quality measurements from Cairo, Egypt. Evaluation focuses on 3,556 held-out validation hours containing 218 distinct regulatory exceedance onsets. We benchmark standard baselines—including persistence and hourly climatology—against a machine learning forecaster currently driving a deployed public warning system. Key Findings The 1-Hour Paradox: For one-hour horizons, persistence achieves the best MAE of all models (15.47 AQI points) but detects 0 out of 218 pollution onsets. The 24-Hour Inversion: At a 24-hour horizon, model rankings completely invert. MAE favors a learned baseline, whereas event-based performance favors persistence (detecting 40.1% vs. 11.8% of onsets). Operational Success: When retrained on this dataset, the deployed production model bridges this gap. At a six-hour horizon, it dominates both paradigms—achieving a 32.19 MAE and a 19.9% detection rate at a low 9.2% false-alarm rate. The ConclusionRelying solely on average error metrics like MAE can lead to selecting "blind" models that fail to predict critical health events. During Cairo's notorious autumn rice-straw burning season—which accounts for 845 unhealthy hours in this dataset—this metric choice is the difference between a functional public health instrument and a meaningless number. We argue that the air-quality forecasting literature must adopt event-based scoring frameworks standard in operational meteorology.