Previous studies (eg. https://doi.org/10.57874/k5kj-2y84) suggest that people understand, and take into consideration, the uncertainties around the estimation of current numbers such as the ‘R’ number of COVID-19 when they are communicated either overtly in text (e.g. ‘there is low consensus on this number’), or graphically through illustrating individual estimates (and hence the range which they span). The UK’s Scientific Advisory Group for Emergencies (SAGE) frequently used these formats to communicate the current value of ‘R’ during the COVID-19 pandemic.
They also used them to communicate numerical projections, for example of numbers of deaths per day. With projections into the future, more uncertainties come into play: there are not only limitations of our knowledge (epistemic uncertainties), but also the effects of randomness (aleatory uncertainties). Statistical models will produce a range of possible outcomes, which include calculated uncertainty ranges around the numbers (‘direct uncertainties’, as defined by van der Bles et al. [1]), but there are also uncertainties caused by limitations in the evidence on which the models are based (‘indirect uncertainties’, as defined by van der Bles et al. [1]), which will not be present in the outputs of any individual model – but can to some degree become evident by comparing the outputs of models and seeing the degree of consensus between them (although models may share assumptions and limitations which lead them to similar results).
Direct uncertainties – the statistically-calculated range of uncertainty around an estimate – can be communicated fairly simply, for example through shaded ranges or fan plots (showing bands of outcomes of different probabilities). There are then different ways to communicate the indirect uncertainties. In text, phrases directly regarding the quality of the evidence or the degree of consensus can be used. In graphical forms, ensemble plots showing the projections of different models can give a sense of the degree of consensus between them.
Alternatively, communicators can numerically combine direct and indirect uncertainties, for instance, by combining multiple model outputs to create consensus outputs (which could again be illustrated via shaded ranges or fan charts, according to the ensemble of models), or restrict the outputs to reasonable worst/best and most likely scenario outcomes.
At the end of October 2020, when decisions were being made about mitigation policies, the UK’s SPI-M-O (Scientific Pandemic Influenza group on Modelling, Operational subgroup) published projections of the number of expected hospitalisations and deaths per day from individual models (See Figure 1a and 1b) and as a combined projection (see Figure 1c and 1d) using several of these methods. These were used both in decision-making by policy-makers and some of them were used to explain the situation to the public in national broadcasts.
a)
Figure 1. Projections of hospitalisations or deaths from COVID-19 as presented by SPI-M-O in late October 2020 in different formats: a) and b) illustrating individual model outputs in which the direct uncertainty is shown and the indirect uncertainty illustrated by the degree of disagreement between models. In a) the thick part of each bar shows the model’s interquartile range, and the thin part shows the 90% credible interval. The faint dotted line the Reasonable Worst Case Scenario[2]. In b) each line represents the output from an individual model, but the lines and shaded areas were not labelled (presumably 90% CIs) [3]. In c) different models are combined and the uncertainty therefore represents a combination of both the direct and the indirect uncertainty. In c) the shaded areas represent the interquartile ranges of model combinations, and the pale grey dotted line the Reasonable Worst Case Scenario calculated by the Cabinet Office Civil Contingencies Secretariat (open grey circles incomplete data expected to increase) [2]. In d) the line shows the consensus projection, with the dark blue shading showing the interquartile range and the light blue the 90% CI to form a fan chart [4].
Given the fact that such a variety of uncertainty communication formats have been used to support important policy decisions, it would be extremely useful for future situations (epidemiological and otherwise) where forecasts, and their various uncertainties, need to be communicated to non-expert audiences, to have an evaluation of their effects on important outcomes such as the perception of the risk in the future, usefulness in decision-making and perceived trustworthiness of the experts.