Research in fields such as weather forecasting has shown that people tend to be accepting of aleatory uncertainty when projecting into the future [1–3], but there is no consensus on which formats best help communicate the uncertainties appropriately.&nbsp;Good communication of uncertainty gives the audience a sense of what experts think the chances are of a range of different outcomes, along with a sense of how confident they are in those forecasts. The audience should not be surprised by, or have their trust undermined by, outcomes manifesting that were not considered the most likely, but did not fall outside the bounds of expectation presented to them. When experts are less confident in their forecasts (for example, though less consensus), the audience’s expectations of the range of possible outcomes should broaden accordingly.&nbsp;An empirical study, on the communication of uncertainty around Covid-19 vaccine effectiveness using a text format, suggested that the communication of a range of possible effectiveness (“between 50% and 70% effective”) – alongside an explanation for the uncertainty (“The reason I can’t give a more precise estimate is because the data we have doesn’t allow that”) – led to a lower decrease in trust and confidence in the communicator when participants were then told that the vaccine’s effectiveness was “actually nearer 40%” than when participants had been given an effectiveness estimate of 60%, with no uncertainty communicated [4]. This potential effect on future trust is also an important outcome for communicators and experts.&nbsp;There have been a number of empirical studies into the effects of forecast communication, particularly in hurricane forecasting. The geospatial path of hurricanes is usually forecast using an ensemble of different models, but then combined into a summary display to be easier to understand [5], however summary displays in general can be misinterpreted both by the public and by experts [6–8]. Detailed work on the effects of hurricane track forecast communication has shown issues with both summary plots such as the ‘cone of uncertainty’ and with showing individual tracks within an ensemble [5,9].&nbsp;An evaluation of fan charts for the forecast of population statistics suggest that they are broadly well-understood [10], but they have not been compared with ensemble plots.&nbsp;The range of formats created at the end of October 2020 for the communication of uncertain forecasts provides an excellent opportunity to evaluate their effects, using outcome measures that evaluate the communication of the likelihoods of different outcomes (via the degree of surprise if they manifest), and perceived trustworthiness of the communicator (both before and after the actual outcome is known).&nbsp;

Theoretical rationale for investigating the effects of communicating uncertainty around future projections of the effects of COVID-19

Previous studies (eg. <a target="_blank" rel="noreferrer noopener" href="https://doi.org/10.57874/k5kj-2y84">https://doi.org/10.57874/k5kj-2y84)</a> suggest that people understand, and take into consideration, the uncertainties around the estimation of current numbers such as the ‘R’ number of COVID-19 when they are communicated either overtly in text (e.g. ‘there is low consensus on this number’), or graphically through illustrating individual estimates (and hence the range which they span). The UK’s Scientific Advisory Group for Emergencies (SAGE) frequently used these formats to communicate the current value of ‘R’ during the COVID-19 pandemic.They also used them to communicate numerical projections, for example of numbers of deaths per day. With projections into the future, more uncertainties come into play: there are not only limitations of our knowledge (epistemic uncertainties), but also the effects of randomness (aleatory uncertainties). Statistical models will produce a range of possible outcomes, which include calculated uncertainty ranges around the numbers (‘direct uncertainties’, as defined by van der Bles et al. [1]), but there are also uncertainties caused by limitations in the evidence on which the models are based (‘indirect uncertainties’, as defined by van der Bles et al. [1]), which will not be present in the outputs of any individual model – but can to some degree become evident by comparing the outputs of models and seeing the degree of consensus between them (although models may share assumptions and limitations which lead them to similar results).Direct uncertainties – the statistically-calculated range of uncertainty around an estimate – can be communicated fairly simply, for example through shaded ranges or fan plots (showing bands of outcomes of different probabilities). There are then different ways to communicate the indirect uncertainties. In text, phrases directly regarding the quality of the evidence or the degree of consensus can be used. In graphical forms, ensemble plots showing the projections of different models can give a sense of the degree of consensus between them.Alternatively, communicators can numerically combine direct and indirect uncertainties, for instance, by combining multiple model outputs to create consensus outputs (which could again be illustrated via shaded ranges or fan charts, according to the ensemble of models), or restrict the outputs to reasonable worst/best and most likely scenario outcomes.At the end of October 2020, when decisions were being made about mitigation policies, the UK’s SPI-M-O (Scientific Pandemic Influenza group on Modelling, Operational subgroup) published projections of the number of expected hospitalisations and deaths per day from individual models (See Figure 1a and 1b) and as a combined projection (see Figure 1c and 1d) using several of these methods. These were used both in decision-making by policy-makers and some of them were used to explain the situation to the public in national broadcasts.a)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clpv8q2ut0001kz08an2y7o7n" alt="A graph with different colored and black arrows

Description automatically generated">b)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clpv8q3ls0005kz08dnhysi5u">c)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clpv8q3f60003kz08achvlpxf" alt="A graph of different colored lines

Description automatically generated">d)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clpv8q3v80007kz089a2zmlx7" alt="A graph showing a graph showing a graph showing a graph showing a graph showing a graph showing a graph showing a graph showing a graph showing a graph showing a graph showing a graph showing a graph showing

Description automatically generated">Figure 1. Projections of hospitalisations or deaths from COVID-19 as presented by SPI-M-O in late October 2020 in different formats: a) and b) illustrating individual model outputs in which the direct uncertainty is shown and the indirect uncertainty illustrated by the degree of disagreement between models. In a) the thick part of each bar shows the model’s interquartile range, and the thin part shows the 90% credible interval. The faint dotted line the Reasonable Worst Case Scenario[2]. In b) each line represents the output from an individual model, but the lines and shaded areas were not labelled (presumably 90% CIs) [3]. In c) different models are combined and the uncertainty therefore represents a combination of both the direct and the indirect uncertainty. In c) the shaded areas represent the interquartile ranges of model combinations, and the pale grey dotted line the Reasonable Worst Case Scenario calculated by the Cabinet Office Civil Contingencies Secretariat (open grey circles incomplete data expected to increase) [2]. In d) the line shows the consensus projection, with the dark blue shading showing the interquartile range and the light blue the 90% CI to form a fan chart [4].Given the fact that such a variety of uncertainty communication formats have been used to support important policy decisions, it would be extremely useful for future situations (epidemiological and otherwise) where forecasts, and their various uncertainties, need to be communicated to non-expert audiences, to have an evaluation of their effects on important outcomes such as the perception of the risk in the future, usefulness in decision-making and perceived trustworthiness of the experts.

Communicating uncertainties around future projections

How does communicating uncertainty affect people’s reaction to future projections of the effects of COVID-19 and their accuracy?

During the first two years of the COVID-19 pandemic, the UK government – in common with most, internationally - made a series of policy-level decisions, many of which would have involved taking into account the current R value (the Reproduction number) of the disease, both nationally and in population subgroups. See Figure 1(1).<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clm82t14e0001la08lc3fg4ac" alt="Diagram, timeline

Description automatically generated">Figure 1: A timeline of UK government responses to the Covid-19 pandemic between March 2020 and December 2021 (Courtesy of the Institute for Government).The public availability of the minutes and briefing papers from the different scientific advisory groups that were providing information to the government allow us to see the different formats that were used to communicate the uncertainties around the values of R to decision-makers and the public throughout this time. These show a trend for increasing sophistication in the calculations and communication of R, presumably as data became more available and as the modelling and communication became more formalised.In March and April 2020, the communications consisted of text such as “The observed rapid increase in ICU admissions is consistent with a higher reproduction number than 2.4 previously estimated and modelled; we cannot rule out it being higher than 3” (March 20th 2020(2)) and “Current estimates of Rt in the community from different modelling groups range from 0.6 to 0.9. Admissions to hospital are falling at a different rate to deaths, resulting in some of the variation” (April 27th 2020(3)).By June 2020, tables and graphs were being used in briefing documents that showed the modelled point estimate and range of R from the six different scientific groups that made up the advisory panel (known as SPI-M: The Scientific Pandemic Infections group on Modelling). By the end of June 2020, a graphic style had apparently been settled on (see Figure 2)(4).<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clm82t1gg0003la08t66fonqz" alt="Chart, scatter chart

Description automatically generated">Figure 2: Example graph of the uncertainty around R from SPI-M consensus paper in the UK on 10th June 2020 We therefore decided to test the effects of a text format and a graphical format based on these formats.The formats are communicating two forms of uncertainty. One is the (‘direct’) modelled uncertainty around the point estimate, communicated in the form of a range around it. The other is the deeper (‘indirect’) uncertainty about the quality of the evidence that went into the consensus value and range(5), communicated either as verbal caveats about the underlying quality of the evidence and level of consensus, or by graphically representing the individual modelled ranges, which gives the audience an impression of the degree of consensus (or lack of it) amongst the experts.We decided to investigate the effects of the communication of indirect uncertainty (in combination with direct uncertainty) in a pair of experiments within a single online survey: the first experiment presenting people with a text form of the information (using terms such as ‘consensus’ or ‘likely to be’ to indicate the quality of the evidence for the range stated, as used by the scientific advisory groups to the UK government), the second experiment presenting people with a graphical format illustrating the degree of consensus between different groups’ predicted ranges (as also used by the scientific advisory groups). The measures in each experiment were planned to be the same.<h2>Dependent variables</h2>Perception of the uncertaintyHow likely do you think it is that the value of R is [UPPER LIMIT]?How surprised would you be if the value of R actually turned out to be [CENTREPOINT]?How surprised would you be if the value of R actually turned out to be [0.2 ABOVE UPPER LIMIT]?How surprised would you be if the value of R actually turned out to be [0.2 BELOW UPPER LIMIT]?How certain or uncertain do you think the R number is?Trustworthiness How trustworthy do you think the information in the message is? Gist comprehension/risk perceptionHow risky does this R number make the COVID-19 situation feel to you?How likely do you think it is that COVID-19 is currently increasing, given this information?Subjective comprehensionHow much effort did you have to put in to understand the statement? InformednessHow well informed did the statement make you feel about the COVID-19 situation?Participants will also complete a numeracy test and demographic information.<h2>Conditions</h2>Experiment 1Participants will be randomly assigned to one of five conditions to read statements about the R number range. The control condition (1) is shown with additions comprising other conditions in brackets: “The SPI-M group report that this week [2: their consensus is that] the overall reproduction number, R, of Covid-19 is [3: highly likely to be] between X and Y. [they state that there is [4: little] [5: high] consensus on this.]. Each participant will rate three statements in a randomised order, where the X-Y range is: 1.4-1.6 (Covid-19 increasing), 0.9-1.1 (Covid-19 stable), and 0.4-0.6 (Covid-19 decreasing). Experiment 2Participants will be randomly assigned to one of three conditions to see graphs depicting the R number range. The control condition will show a single bar illustrating a point estimate and a symmetrical range with error bars. The two uncertainty conditions will illustrate several point estimates with error bars (all showing the same direct uncertainty interval), representing the range outputs of different modelling groups. In one condition there will be considerable consensus between the different ranges (‘low uncertainty’) and in the other, less consensus (‘high uncertainty’)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clm82t1nb0005la08hvfo9z9a" alt="A graph of a graph

Description automatically generated with medium confidence"><img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clm82t1vq0007la08nbzbi6xw" alt="A diagram of a graph

Description automatically generated with medium confidence"><img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clm82t20e0009la08ody65v3g" alt="A graph of colored lines

Description automatically generated with medium confidence">Figure 3: Examples of the three (indirect) uncertainty conditions in Experiment 2: Top left, no uncertainty cues; top right, high uncertainty cues; bottom left, low uncertainty cuesEach participant will rate two graphs in a randomised order, where the direct uncertainty range is: 1.4-1.6 (Covid-19 increasing), and 0.4-0.6 (Covid-19 decreasing). <h2>Analysis</h2>All analyses will be exploratory. Our main outcomes of interest are the perceived uncertainty and perceived trustworthiness of the different arms within and between each experiment.We will use mixed 5(condition; between) x 3(range; within) ANOVAs for Experiment 1 and mixed 3(condition; between) x 2(range; within) ANOVAs for Experiment 2. Interactions will be decomposed to one-way ANOVAs of condition at each level of range. Pairwise tests between groups will be adjusted using the Benjamini-Hochberg procedure.We will also compare responses to the graphs with those to the text statements. Specifically the responses to graphs will be compared to the control condition text with the corresponding R interval (Welch’s t-test). We may conduct other comparisons in an exploratory capacity and any reported results will be clearly reported as exploratory. Participants who fail a simple attention check (“please select ‘quite worried’”) will be excluded from analyses.<h2>Participants</h2>A sample size of 1500 will be recruited, To calculate the sample size we conducted power calculations in GPower (6) based on a linear model with one, between subjects categorical predictor with 5 levels/arms (reflecting the design of Experiment 1) specifying 80% power and an expected difference of d = 0.3. We have used an overly-conservative Bonferroni correction to account for all 10 multiple comparisons that stem from each combination of 5 arms in the between subjects design, correcting alpha to 0.005 for the purposes of the input parameters in our power calculation (i.e. 0.05 divided by the 10 comparisons). This gives n = 1500 in total. In analyses, we will use the Benjamini-Hochberg procedure to keep the false discovery rate at 5%.

A method to test the effects of different verbal/numeric and graphical communications of uncertainty around the Covid-19 R number (pre-registration)

Method to test the effects of different formats of communication of indirect uncertainty around the COVID-19 R number on a public audience

Previous empirical work provides a basis for designing a study to understand how best to communicate the epistemic uncertainty around the current R value of COVID-19.&nbsp;Verbal and numerical formatsOne commonly used way of communicating COVID-19 related statistics and associated uncertainties during the pandemic was to convey that information in a text format (combining words and numbers).&nbsp; This kind of format can generally be used to communicate different types of uncertainty, such as statistical confidence intervals – a form of ’direct uncertainty’ or levels of consensus – a form of ‘indirect uncertainty’ as defined by [1]. These text-based communications can range from simple mentions that uncertainty exists (such as “the precise figure is not known”), through verbal expressions of the magnitude of uncertainty (such as “there is a high degree of confidence”) to more precise numerical uncertainty (such as “between 30-35%”) [1].&nbsp; Empirical work has been carried out mainly on the communication of the direct uncertainty around an estimate (such as a confidence interval or range) – whether in numerical or verbal form. For example [2–6]and [7] looking at the effects of adding numeric uncertainty to statements, and [5,6,8–10] investigating the effects of adding verbal hedge words or statements to communicate some uncertainty around an estimate or claim. The effects tend to differ depending on the exact communication format used, the context, and the measures used. On the whole, though, audiences seem to understand numerical ranges, sometimes don’t notice single ‘hedge words’ and adjust their confidence or trust in the numbers depending on their certainty, but their trust in the providers of the numbers can also be reduced by communications of uncertainty around the estimate in some cases.&nbsp;Less work has been done on the effects of communication of indirect uncertainty – uncertainty around the evidence underlying an estimate (and its range). Estimates of the underlying quality of evidence or confidence in findings is more qualitative, and can comprise elements such as the degree of consensus between experts, the amount of raw data available to draw conclusions from, or the presence of large confounders or unknowables. Assessments of the quality of evidence can be made informally, or via a ‘check-list’ (for example [11,12]), and are then communicated via a qualitative or semi-quantitative scale (such as describing confidence as ‘high’ or ’low’, or using icons, such as those used by GRADE [13] or the IPCC [14,15]). Some empirical studies have looked at the effects of overt ‘high’ and ‘low’ quality labels [16,17], finding that lay audiences are sensitive to them and adjust their perceptions of the accuracy and trustworthiness of the estimates. Additionally explaining that there was high uncertainty because “there is disagreement between experts” or “there is a lack of data” (or the converse, in the case of low uncertainty), did not affect participants’ response [17]. This is despite scientific consensus being considered a strong cue of trustworthiness in lay readers [18,19].&nbsp;When it comes to the communication of uncertainty around Covid-19 statistics, Baker et al. [20] analysed linguistic markers of uncertainty in SAGE’s COVID meeting minutes during the pandemic, and found that the use of hedge words (e.g. “probably”, “may”, “indicates”) increased significantly over the course of time analysed (22nd Jan 2020 – 13th May 2021).&nbsp; These could be used to communicate either direct or indirect uncertainty, or a combination of both. However, they also found that these minutes largely communicated using a consensus voice during the study period (e.g. speaking as a whole “SAGE advises…”), not explicitly signposting disagreement, but emphasising agreement where present (e.g. “unanimous”, “completely”), communicating indirect uncertainty, but only when it was at a low level, not when there was high indirect uncertainty (low consensus).&nbsp; Baker et al. [20] further found inconsistent use of explicit verbal levels of confidence (e.g. high, medium, low confidence) communicating indirect uncertainty and explicit confidence intervals in the minutes (direct uncertainty), although found that both types of uncertainty communication were used more frequently to emphasise higher levels of confidence and certainty than lower.&nbsp;&nbsp;Graphical formatsVisualisations provide another way of communicating COVID-19 statistics, and provide a useful way of presenting data that facilitates understanding and more rapid intuition of meaning [21–23].&nbsp; They can also help communicate both direct uncertainty (such as illustrating a range or confidence interval via bars) and direct uncertainty (such as plotting individual datapoints in a scatter plot to illustrate amount and variability of data, or illustrating individual model outputs within an ensemble, such as a spaghetti plot).&nbsp;Various empirical work has shown how distributional visualisations of probability, such as violin or density plots, generally reduce mental effort and bias and improve accuracy over visualisations of summary statistics such as means and confidence intervals [24–26].&nbsp; The relative efficacy of different types of visualisations are not always clear cut, and can depend on the context within which they are received and what the communicator is measuring as their outcome of success (e.g. comprehension versus behaviour change).&nbsp;&nbsp;An ensemble plot is a particular example that is useful for visualising the indirect uncertainty (degree of consensus) in modelled data.&nbsp; For example, ensemble plots have been developed as an alternative to the ’cone of uncertainty’ that was commonly used to communicate the possible path of a hurricane. Instead of a single, summarised distribution (the ‘cone’), the ensemble plot illustrates a selection of representative ensemble members derived from runs of individual forecasting models [24,27–29].&nbsp;&nbsp;Such ensembles of possible scenarios or model runs has sometimes been used for COVID-19-related statistics [30], including for the modelled R-value [31] – they are not solely used for forecasts, but any time where a value has to be estimated.&nbsp; Some empirical work on lay peoples’ understanding of ensemble plots (in the context of hurricane forecasts) has been done, raising some concerns over how well they convey the true range of possibilities and their likelihoods [32,33].Padilla et al. [34] also studied the presentations of ensemble line graphs with respect to COVID-19 statistics. They found that visual representations of six or more model forecasts (i.e. combining both epistemic and aleatory uncertainty) raised people’s perception of the risk of COVID-19 more than other formats tested, including those that used a gradient depiction of uncertainty or those presenting 60% or 95% confidence interval summary statistics.&nbsp;&nbsp;Taken together, then, the empirical and theoretical literature suggests that different formats of communicating epistemic uncertainty around the COVID-19 R value might affect audience’s comprehension of the likelihood of different outcomes, and therefore their perception of the risk from COVID-19 and likelihood of adopting various behaviours as well as their perceptions of the accuracy and trustworthiness of the R-value, and of the trustworthiness of the scientists and communicators behind the presentation of the R-value.&nbsp;There are a number of different formats which could be (and have been) used to communicate either the direct or the indirect uncertainty around the R-value, including graphical or numerical indications of the statistical range around a point estimate (direct uncertainty), verbal descriptions of the uncertainty (both direct and indirect) – such as hedge words and indications of consensus or evidence quality, or graphical ensemble plots (indirect uncertainty, although the direct uncertainty could be simultaneously illustrated).&nbsp;Empirical work could be carried out, based on previous study designs, to investigate the effects of these formats.

A review of what is known about the effects of communicating uncertainty around the R value of an epidemic, or similar numbers.

How might the format used to communicate uncertainty around the R value of COVID-19 affect audiences?

The SARS-CoV-2 reproduction number or ‘R’ number is a key figure that has been communicated throughout the pandemic in many countries, as it is an important way for policy-makers and the public to understand the current trajectory of infections. In the UK, for example, the consensus on the current value for R, based on modelling by several independent scientific groups, was issued in papers to the UK’s Scientific Advisory Group for Emergencies (SAGE) on a frequent basis during the pandemic, and subsequently to the Government Office for Science for public release. As the pandemic has progressed, there have been changes in the extent to which SAGE themselves communicated uncertainty linguistically (Baker et al., 2022) as well as in the format of communication of uncertainties around the current R consensus value (epistemic uncertainties) in various SAGE related minutes. For example, both verbal and graphical formats (including graphs and tables) have been used to illustrate the degree of uncertainty, including consensus, on the R number, see Figure 1.In line with advice in SAGE’s terms of reference (Cabinet Office, 2012; Baker et al., 2022), these formats generally communicate both ‘direct’ and ‘indirect’ uncertainties, as defined by van der Bles et al. (van der Bles et al., 2019). The direct uncertainty is the uncertainty about the estimate itself, often communicated in the form of a range or confidence interval. The indirect uncertainty is the deeper uncertainty about that estimate and range, in the form of some expressions or illustrations of the underlying quality of the evidence (such as disagreement between different models). Little is known about how these different formats are perceived and integrated into decision making by the audiences receiving them, and yet individuals and policy-makers rely on these official estimates of R to make important and potentially life-changing decisions.Empirical research has the potential to help guide future communicators of COVID-19 statistics, as well as other important and epistemically uncertain numbers.a)“ … SPI-M-O’s consensus view is that Rt in the community is well below 1 and likely significantly lower than the overall Rt in the population. The likely range is 0.5-0.9. We have independent data sources that informs our estimates of Rt in the community.”b)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clm507hgc0001l40867tenbr1">c)<h5><img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clm507hno0003l408e9ofoh9h" alt="Table

Description automatically generated">Figure 1: Three formats of communication of the uncertainty around R, the reproduction number, of COVID-19 from official government sources in the UK: a) Extract from verbal communication of the uncertainties around R in different subpopulations in the UK (community, hospitals and care homes)(SPI-M-O, 2020b) b) Graph showing the individual modelling estimates and consensus range for the UK as a whole (SPI-M-O, 2020a), c) Table showing the individual modelling estimates for different regions of the UK (SPI-M-O, 2020b).</h5>

‘R’ is a key figure communicated throughout the pandemic for decision-making, but is always uncertain. How is that uncertainty perceived when communicated?

How to communicate uncertainty around the R number of COVID-19?

This Interpretation is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>&nbsp;The interviews and the survey responses combined help give us an overall view of the public’s information needs regarding their personal COVID-19 risk, as of July 2020.The majority of participants clearly showed a desire for quantitative information about COVID-19 (much of which was not, at the time of this survey, available). The comments from the primary care physicians suggested that they viewed patients as perhaps becoming used to being given simplified instructions rather than information on which to base their own decisions. However, the results from the surveys implied that the majority of the public would actually appreciate being given more detailed information on which to base their own risk decisions.The quantitative survey emphasized that there was a clear public concern about the virus and its potential consequences, and people’s perception of chances of themselves dying of the virus if they caught it were not highly correlated with the actual chances as calculated from a personalized risk calculator given the health information participants gave us, with only a weak relationship between the two.Some of this discrepancy seemed very likely to be due to the way that most people think about ‘risks’—as a subjective impression, not easily translated into a numerical concept [1]. Even when pushed to quantify their feeling of risk, most members of the public in our interviews described the risk in terms of a ‘persona’ that they had in their mind to represent that level of risk: a person who embodied the major risk factors that would exemplify that level of risk (such as age and relevant health attributes).Given that only a minority of people in the surveys wanted simply to be told that their risk fell into a simple category such as ‘low’ (as well as the practical and ethical difficulties of doing that given that an absolute risk and relative risk would need to be combined in some way), it would be useful to consider further ways in which to help people understand the likelihood figure of them dying from COVID-19 should they catch it.The other themes emerging from the interviews were useful in determining important endpoints to measure in future work developing a personalised communication tool: levels of perceived uncertainty, perceptions of the quality of the underlying evidence, trust (in the data itself and in the source of the data) and degree of worry associated with the results. Trustworthiness is a crucial dimension to consider during the design of any communication—something previously emphasized in the risk communications literature (e.g. [2]). In line with previous research (e.g. [3]), the interviews suggested that trustworthiness and relevance were enhanced by appropriate branding and making it clear that the results presented were based on research (ideally from a trustworthy source) and on relevant data. Previous experience (e.g. [3]) suggests that people are sensitive to cues of quality of evidence (such as sizes of datasets and relevance of the population on which the evidence is based), as well as assessing the information’s source.

Interpretation: what the public already know, want to know and need to know, about their personal risks from COVID-19

This Method is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>&nbsp;A combination of in depth qualitative interviews and large-scale quantitative surveys are ideal for gaining insights into people’s understanding of a topic such as their risk from COVID-19, and their information needs.Quantitative surveys can give broad insights into opinions across a population. Table 1 gives an outline of questions designed to help gain these insights.In our prior research [1], we found that a sample size of 700 per country was more than adequate to characterize predictors of COVID-19 risk perception in that country. Further analysis of the earliest UK dataset in [1] indicated that the mean value of 700 participants versus just the first 500 differed by less than a quarter of a point on every 7-point Likert item, suggesting that 500 participants would be adequate for a descriptive study to characterize attitudes on similar topics.<table><tbody><tr><td colspan="1" rowspan="1">Theme</td><td colspan="1" rowspan="1">Questions asked</td><td colspan="1" rowspan="1">Answer options</td></tr><tr><td colspan="1" rowspan="2">Personal estimated risk</td><td colspan="1" rowspan="1">If you caught coronavirus, how likely do you think it is that you would die from it?&nbsp;&nbsp;Why did you rate your chance at this level?&nbsp;</td><td colspan="1" rowspan="1">9-point Likert scale:Not at all likely-Almost certain&nbsp;Free text</td></tr><tr><td colspan="1" rowspan="1">What is your age?Would you say your weight/BMI is:&nbsp;Have you had an organ transplant?Have you had any form of cancer in the last 12 months?If so, have you undergone treatment for your cancer in the last 12 months?&nbsp;Do you suffer from COPD (Chronic obstructive pulmonary disorder)?&nbsp;Do you suffer from asthma?&nbsp;&nbsp;Are you currently taking steroids or steroid inhalers for any medical condition?&nbsp;Are you currently taking any medicine that suppresses your immune system?&nbsp;Are you aware of suffering any medical condition that suppresses your immune system?&nbsp;Do you suffer from diabetes?&nbsp;</td><td colspan="1" rowspan="1">&nbsp;5-point Likert scale:Very Low, Low, Average, High, Very High&nbsp;Yes/No&nbsp;Yes/NoYes/No&nbsp;&nbsp;Yes/No&nbsp;No/Yes – low level/Yes – moderate level/Yes – severe&nbsp;Yes/No&nbsp;&nbsp;Yes/No&nbsp;&nbsp;Yes/No&nbsp;No/Yes – Type 1/Yes – Type 2</td></tr><tr><td colspan="1" rowspan="1">What information the audience want</td><td colspan="1" rowspan="1">I would like to know now what my personal risk of dying from COVID-19 would be if I were to catch the virus&nbsp;I think that people are entitled to know now what their personal risk of dying from COVID-19 would be if they were to catch it&nbsp;If my doctor was going to tell me now my personal risk of dying from COVID-19 if I caught the virus, I would like to know that risk as a precise number, rather than as a category (e.g. low, medium, high)&nbsp;I would not like my employer to know my personal risk of dying from COVID-19&nbsp;I feel that I have enough information already about my personal risks from COVID-19&nbsp;I would like to know by how much each personal behavioural change (e.g. wearing face masks, washing hands) reduces my personal risk of catching the virus&nbsp;I would not trust any information about my personal risk of dying from COVID-19 as I don’t believe enough is known about it&nbsp;</td><td colspan="1" rowspan="1">7-point Likert scales: Completely disagree - Completely agree</td></tr><tr><td colspan="1" rowspan="2">&nbsp;</td><td colspan="1" rowspan="1">Which of the following pieces of information would you most want to know about your risk, if it were available? Please drag and drop into the order you feel puts the most important at the top:&nbsp;The risk of you dying of COVID-19 if you caught it;The risk of you catching COVID-19 given where you live and how many other cases have been reported nearby (will vary day-to-day a bit like a pollen count);The risk of you being hospitalised by COVID-19 if you caught it;Whether you might suffer long-term complications from COVID-19 if you caught it</td><td colspan="1" rowspan="1">Rank order </td></tr><tr><td colspan="1" rowspan="1">&nbsp;Is there any other information about your risk from COVID-19 that you would like to know that we haven't mentioned?</td><td colspan="1" rowspan="1">Free text</td></tr><tr><td colspan="1" rowspan="1">&nbsp;</td><td colspan="1" rowspan="1">Please drag and drop the following into the order you feel puts the most important at the top:&nbsp;The person's risk score: the chances of that person dying if they caught COVID-19;Information about what data was used to calculate that risk score (e.g. whether it was data from the UK, how much data the researchers had);&nbsp;Information about who developed the maths behind the calculation;&nbsp;Information about the certainty and precision of the risk score: how much lower or higher the actual risk for that person could be;Context about that risk score: how it rates compared to a similar person at different ages (e.g. 'someone like you but aged 20'; 'someone like you but aged 60');&nbsp;Context about that risk score: where that person's risk lies in comparison to everyone else in the UK (e.g. a graph of the risks of everyone in the UK with an arrow showing 'you are here' to make it clear how many people have a higher risk than that person, and how many a lower risk);Link to where people can get more information about things they can do to reduce their chances of catching COVID-19 (e.g. hand washing, social distancing, wearing face masks etc);Information reminding people that as well as the risk to themselves, they also pose a risk to others (in case they catch COVID-19 and pass it on);Details about how the risk score was calculated (e.g. what factors makes someone's risk higher or lower, and how important each of those factors are);Information about thing they can do to reduce their chances of dying from COVID-19 if they caught it (such as losing weight)&nbsp;</td><td colspan="1" rowspan="1">Rank order</td></tr><tr><td colspan="1" rowspan="1">&nbsp;</td><td colspan="1" rowspan="1">“A tool like this can be designed in many different ways, which give people a different impression of the numbers. The tool can be made as neutral as possible, to try to just inform people and let them make up their own mind about the risk to them and how they should behave. Or the differences between numbers can be made very obvious, making the risk look larger or smaller. This would be more persuasive and make people more likely to change their behaviour. How do you think a national tool should be designed?”&nbsp;(For those who marked 4-7 on question above):“In which direction do you think the tool should try to persuade people?”</td><td colspan="1" rowspan="1">7-point Likert scale: As neutral as possible: just inform people - As persuasive as possible: change people's behaviour 3 options:It should try to reassure people by showing the risk is generally low; It should try to make people be more cautious by showing that even if the risk is low to them, they can spread it to others; It should try to be persuasive to different people in different ways (reassuring some and making others more cautious)&nbsp;</td></tr></tbody></table>&nbsp;1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Dryhurst S, Schneider CR, Kerr J, Freeman ALJ, Recchia G, van der Bles AM, Spiegelhalter D, van der Linden S. 2020 Risk perceptions of COVID-19 around the world. J. Risk Res. 23, 1466–4461. (doi:10.1080/13669877.2020. 1758193)&nbsp;

Method for online surveys to find out what the public already know, want to know and need to know, about their personal risks from COVID-19

This Hypothesis/Theoretical rationale is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>One method of approaching risk communication is the ‘mental models’ approach, whereby researchers use both qualitative and quantitative methods to build up an understanding of the audience’s intuitive model of causality in a situation, as well as the probabilities they ascribe to the likelihoods of different events and the strengths of causal relationships. The idea is then to provide the audience with information to help bring their understanding of the situation closer to the expert understanding of it, correcting any misunderstandings or providing missing pieces of information [1,2].In order to apply this method to COVID-19, it would be useful to know firstly how accurate people’s estimate of the likelihood of them dying from COVID-19 is. This includes two likelihoods: the likelihood of catching it (which depends very much on local prevalence as well as a person’s behaviour, and so is an ever-changing number), combined with the likelihood of dying from it if they catch it (which depends on their health risk factors, as well as the availability and effectiveness of treatments and so is more stable). It is probably, therefore, only possible to ask people to estimate the second: the risk of themselves dying from COVID-19 if they catch it – and then compare that with estimates from risk calculators, based on information about their risk factors.It would then be important to build up a picture of what people understand the major risk factors to be, how important they feel they are, and what information they would actually want to know about their risks. This can be done through a combination of in depth qualitative interviews and quantitative surveys covering a broad diversity of people.1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Granger Morgan M, Fischoff B, Bostrom A, Atman CJ. 2002 Risk communication: a mental models approach. Cambridge, UK: Cambridge University Press.2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;Fischhoff B. 1999 Why (cancer) risk communication can be hard. Monogr. Natl Cancer Inst. 25, 7–13. (doi:10.1093/ oxfordjournals.jncimonographs.a024213)&nbsp;

Finding out what the public already know, and want to know, about their personal risks from COVID-19

These Results are based on work published in: https://doi.org/10.1098/rsos.201721The data from this experiment is available in the Open Science Framework repository: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.17605/OSF.IO/AUF8H%20Survey%204">https://doi.org/10.17605/OSF.IO/AUF8H Survey 4</a>.This study was approved by the Psychology Research Ethics Committee at the University of Cambridge (PRE.2020.070).The survey was carried out online, with 2,500 participants recruited through the ISO- accredited polling company Respondi (the power calculation was done for another experiment within the same survey that had more arms than this one, and so required more participants). Quotas were set to ensure recruitment of UK participants representative of the national population on age and sex. Participants were paid £2.50 and the survey lasted 20–25 min, and included several different sections. This experiment came after sections asking some personal risk perception and COVID-19 experience questions, and an experiment in which participants were asked to estimate the likelihood of 5 different individuals (whose risk factors were described) dying from COVID-19 if they caught it. Characteristics of the participants are shown in Table 1Table 1: Characteristics of the participants in the survey<table><tbody><tr><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1">Total n=2500</td></tr><tr><td colspan="1" rowspan="1">Sex (%)</td><td colspan="1" rowspan="1">MaleFemale</td><td colspan="1" rowspan="1">1199 (48)1301 (52)</td></tr><tr><td colspan="1" rowspan="1">Ages(%)</td><td colspan="1" rowspan="1">18-2425-3435-4445-5455-6465+</td><td colspan="1" rowspan="1">303 (12.1)454 (18.1)480 (19.2)480 (19.2)429 (17.2)354 (14.2)</td></tr><tr><td colspan="1" rowspan="1">Ethnicity(%)</td><td colspan="1" rowspan="1">WhiteBlackAsianMixedOther</td><td colspan="1" rowspan="1">2174 (87)64 (2.6)160 (6.4)55 (2.2)15 (0.6)</td></tr><tr><td colspan="1" rowspan="1">Education(%)</td><td colspan="1" rowspan="1">No qualGCSE-levelA-levelBachelorsMastersDoctoralOther</td><td colspan="1" rowspan="1">116 (4.6)578 (23.1)692 (27.7)717 (28.7)250 (10)49 (2.0)71 (2.9)</td></tr><tr><td colspan="1" rowspan="1">Experience of COVID-19</td><td colspan="1" rowspan="1">Confirmed infectionUnconfirmed infectionNo infection</td><td colspan="1" rowspan="1">14 (0.6)373 (15.0)2113 (84.5)</td></tr><tr><td colspan="1" rowspan="1">Numeracy*(%)</td><td colspan="1" rowspan="1">12345678</td><td colspan="1" rowspan="1">166 (6.7)250 (10.1)363 (14.7)445 (18)467 (18.9)428 (17.3)137 (5.5)217 (8.8)</td></tr></tbody></table>*Score 1-8, based on a sum of participants’ score on the adaptive Berlin numeracy test [1], 3 items from [2] and a single item from [3]As per the pre-registered method, to investigate how participants would perceive a range of realistic absolute risks of dying from COVID-19 (if infected), and to quantify the effects of numerical format on these perceptions, participants were asked to rate five different numerical risks (presented to them in a random order) on a sliding scale marked only ‘very low risk’ and ‘very high risk’ at the extremes. Their choice of position on the slider was coded as 0–100 by the survey software. Each participant saw the same risks, but presented in one of five different formats:- Estimates presented as frequency, ‘ X in 100’- Estimates presented as frequency, ‘ X in 1000’- Estimates presented as frequency, ‘ 1 in X’- Estimates presented as percentage, X%- Estimates presented as percentage, X%, responses captured on 11pt Likert scale rather than a slider (as an exploratory check)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5qjz5l5000109mhplt7y7jl" alt="Chart, scatter chart, box and whisker chart

Description automatically generated">The mean responses from the 4 different formats that used a sliding scale are shown in Figure 1, with 95% confidence intervals as bars, and the full distribution of responses shown as a violin plot.Figure 1. Mean (95% CI) ratings (from ‘very low risk’ (0) to ‘very high risk’ (100)) of five different COVID-19 infection fatality risk figures, presented as a percentage or as one of three frequency formats. Violin plots indicate underlying distribution.Mean risk ratings and SDs across risk levels and formats are also shown in Table 2 below.Table 2: The mean risk ratings and SDs that participants gave to each of the four formats, across risk levels and formats<table><tbody><tr><td colspan="1" rowspan="1">Risk level</td><td colspan="4" rowspan="1">Format</td></tr><tr><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1">Percentage</td><td colspan="1" rowspan="1">X in 100</td><td colspan="1" rowspan="1">X in 1000</td><td colspan="1" rowspan="1">1 in X</td></tr><tr><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1">M (SD)</td><td colspan="1" rowspan="1">M (SD)</td><td colspan="1" rowspan="1">M (SD)</td><td colspan="1" rowspan="1">M (SD)</td></tr><tr><td colspan="1" rowspan="1">0.1%</td><td colspan="1" rowspan="1">12.26 (24.09)</td><td colspan="1" rowspan="1">17.91 (25.94)</td><td colspan="1" rowspan="1">23.02 (27.22)</td><td colspan="1" rowspan="1">29.13 (26.20)</td></tr><tr><td colspan="1" rowspan="1">1%</td><td colspan="1" rowspan="1">15.69 (23.27)</td><td colspan="1" rowspan="1">27.05 (29.00)</td><td colspan="1" rowspan="1">31.25 (27.34)</td><td colspan="1" rowspan="1">44.46 (29.11)</td></tr><tr><td colspan="1" rowspan="1">5%</td><td colspan="1" rowspan="1">23.83 (25.18)</td><td colspan="1" rowspan="1">33.37 (27.36)</td><td colspan="1" rowspan="1">40.28 (27.84)</td><td colspan="1" rowspan="1">57.81 (29.34)</td></tr><tr><td colspan="1" rowspan="1">12%</td><td colspan="1" rowspan="1">35.42 (27.09)</td><td colspan="1" rowspan="1">42.77 (26.99)</td><td colspan="1" rowspan="1">49.02 (27.04)</td><td colspan="1" rowspan="1">65.93 (29.66)</td></tr><tr><td colspan="1" rowspan="1">20%</td><td colspan="1" rowspan="1">45.36 (26.94)</td><td colspan="1" rowspan="1">54.23 (27.56)</td><td colspan="1" rowspan="1">58.34 (26.49)</td><td colspan="1" rowspan="1">73.50 (29.35)</td></tr></tbody></table><ol><li>Cokely ET, Galesic M, Schulz E, Ghazal S, Garcia- Retamero R. 2012 Measuring risk literacy: the Berlin numeracy test. Judgm. Decis. Mak. 7, 25–47. (doi:10.1037/t45862-000)</li><li>Schwartz LML, Woloshin SS, Black WCW, Welch HGH. 1997 The role of numeracy in understanding the benefit of screening mammography. Ann. Intern. Med. 127, 966–972. (doi:10.7326/0003-4819-127-11- 199712010-00003)</li><li>Lipkus IM, Samsa G, Rimer BK. 2001 General performance on a numeracy scale among highly educated samples. Med. Decis. Mak. 21, 37–44. (doi:10.1177/0272989X0102100105)</li></ol>

This work was funded by the Winton Centre for Risk & Evidence Communication at the University of Cambridge, which is financed by a donation from the David & Claudia Harding Foundation.

Results of testing the effects of different numerical formats on people’s perception of their risk of dying from COVID-19

This Problem is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>Different communicators might have different aims when approaching the communication of personalized risk from COVID-19. Some may feel it important to encourage people to take mitigating actions, and hence to ensure that risks are perceived as relatively high to increase motivation [1,2]. Others might want to reduce worry and hence ensure that risks are perceived as low. There is no ‘correct’ perception of a risk, as risk perception is inherently subjective and incorporates both likelihood and personal impact (which depends on personal vulnerabilities and priorities) [3].In order to be able to provide any useful information, however, it is important to understand what the public’s current state of knowledge on the subject is, and what information they particularly want to have (or specifically not want to have).1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Bish A, Michie S. 2010 Demographic and attitudinal determinants of protective behaviours during a pandemic: a review. Br. J. Health Psychol. 15, 797–824. (doi:10. 1348/135910710X485826)2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Dryhurst S, Schneider CR, Kerr J, Freeman ALJ, Recchia G, van der Bles AM, Spiegelhalter D, van der Linden S. 2020 Risk perceptions of COVID-19 around the world. J. Risk Res. 23, 1466–4461. (doi:10.1080/13669877.2020. 1758193)3.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Slovic, P. and Peters, E., 2006. Risk perception and affect.&nbsp;Current directions in psychological science,&nbsp;15(6), pp.322-325.

Organisation	Role	Department	Start date	End date
University College London	Lecturer	Not specified	2023-07-01
University of Cambridge	Research Associate	Not specified	2018-10-01	2023-01-01

Title	DOI	Published date
The effects of communicating uncertainty around statistics, on public trust	10.1098/rsos.230604	2023-11
How People Understand Risk Matrices, and How Matrix Design Can Improve their Use: Findings from Randomized Controlled Studies	10.1111/risa.13822	2022-05
The effects of communicating uncertainty around statistics on public trust: an international study	10.1101/2021.09.27.21264202	2021-09-29
Correlates of intended COVID-19 vaccine acceptance across time and countries: results from a series of cross-sectional surveys	10.1136/bmjopen-2020-048025	2021-08
COVID-19 risk perception: a longitudinal analysis of its predictors and associations with health protective behaviours in the United Kingdom	10.1080/13669877.2021.1890637	2021-04-03
Communicating personalized risks from COVID-19: guidelines from an empirical study	10.1098/rsos.201721	2021-04
Predictors of COVID-19 vaccine acceptance across time and countries	10.1101/2020.12.09.20246439	2020-12-11
Communicating personalised risks from COVID-19: guidelines from an empirical study	10.1101/2020.10.05.20206961	2020-10-06
Susceptibility to misinformation about COVID-19 around the world	10.1098/rsos.201199	2020-10
Risk perceptions of COVID-19 around the world	10.1080/13669877.2020.1758193	2020-08-02

Sarah Dryhurst

Employment

Education

Works

Octopus publications