Testing for ontological errors in probabilistic forecasting models of natural systems

We show that testability of the ontological null hypothesis derives from an experimental concept, external to the model, that identifies collections of data, observed and not yet observed, that are judged to be exchangeable when conditioned on a set of explanatory variables

W. Marzocchi; T. H. Jordan

2014

Scholarcy highlights

  • Probabilistic forecasting models describe the aleatory variability of natural systems as well as our epistemic uncertainty about how the systems work
  • The purpose of this paper is to clarify the conceptual issues associated with the testing of probabilistic forecasting models for ontological errors in the presence of aleatory variability and epistemic uncertainty
  • Bayesian modeling checking has been criticized by purists on both sides, but one version, prior predictive checking, provides us with an appropriate framework for the testing of forecasting models for ontological errors
  • We clarify several conceptual issues regarding the testing of probabilistic forecasting models, including the ambiguity of the aleatory/epistemic dichotomy, the quantification of uncertainties as degrees of belief, the interplay between Bayesian and frequentist methods, and the scientific pathway for capturing predictability
  • Models are not scientific. We explore these issues in the probabilistic seismic hazard analysis framework developed by earthquake engineers in the 1970s and 1980s and refined in the 1990s by the Senior Seismic Hazard Analysis Committee of the US Nuclear Regulatory Commission
  • This central value measures the aleatory variability of the hazard, conditional on the model, and the dispersion of p about φ describes the epistemic uncertainty in its estimation
  • By testing the ontological hypothesis under appropriate experimental concepts, we can answer the important question of whether a model’s predictions conform to our conditional view of nature’s true variability

Need more features? Save interactive summary cards to your Scholarcy Library.