Vector-borne diseases (VBDs) such as dengue, malaria, and chikungunya are recurring public health challenges in tropical and sub-tropical regions. Their transmission dynamics are influenced by environmental conditions, vector ecology, and human behavior, resulting in strong temporal variability and periodic outbreaks.
Early warning systems based on predictive modeling can play a critical role in public health decision making by enabling proactive interventions such as vector control, healthcare preparedness, and resource allocation. Predictive models can capture temporal trends and provide estimates of future case counts, which are essential for triggering timely actions.
Multiple modelling approaches have been applied for VBD forecasting, like statistical models, machine learning approaches, and mechanistic models. However, the assessment of performance of these models is often limited to a few standard metrics, which does not capture their operational utility for a public health use case. Hence, there is a need for a comprehensive evaluation methodology to assess the operational readiness of VBD forecasting models.
The evaluation methodology at VBD-ModelBench uses multiple metrics, including both standard and operationally relevant metrics. It is based on a stratified evaluation process, where models are benchmarked across different lead times, outbreak regimes, seasons, and total disease load.
While evaluating models for implementation, it is important to understand the requirements of the health system and the context in which the model will be used. The final model choice is context dependent. A strong candidate should improve over naive persistence, degrade gracefully with lead time, avoid underprediction, track epidemic trajectory, and remain reliable across the strata that matter for public-health action.
