This Interpretation is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>&nbsp;The interviews and the survey responses combined help give us an overall view of the public’s information needs regarding their personal COVID-19 risk, as of July 2020.The majority of participants clearly showed a desire for quantitative information about COVID-19 (much of which was not, at the time of this survey, available). The comments from the primary care physicians suggested that they viewed patients as perhaps becoming used to being given simplified instructions rather than information on which to base their own decisions. However, the results from the surveys implied that the majority of the public would actually appreciate being given more detailed information on which to base their own risk decisions.The quantitative survey emphasized that there was a clear public concern about the virus and its potential consequences, and people’s perception of chances of themselves dying of the virus if they caught it were not highly correlated with the actual chances as calculated from a personalized risk calculator given the health information participants gave us, with only a weak relationship between the two.Some of this discrepancy seemed very likely to be due to the way that most people think about ‘risks’—as a subjective impression, not easily translated into a numerical concept [1]. Even when pushed to quantify their feeling of risk, most members of the public in our interviews described the risk in terms of a ‘persona’ that they had in their mind to represent that level of risk: a person who embodied the major risk factors that would exemplify that level of risk (such as age and relevant health attributes).Given that only a minority of people in the surveys wanted simply to be told that their risk fell into a simple category such as ‘low’ (as well as the practical and ethical difficulties of doing that given that an absolute risk and relative risk would need to be combined in some way), it would be useful to consider further ways in which to help people understand the likelihood figure of them dying from COVID-19 should they catch it.The other themes emerging from the interviews were useful in determining important endpoints to measure in future work developing a personalised communication tool: levels of perceived uncertainty, perceptions of the quality of the underlying evidence, trust (in the data itself and in the source of the data) and degree of worry associated with the results. Trustworthiness is a crucial dimension to consider during the design of any communication—something previously emphasized in the risk communications literature (e.g. [2]). In line with previous research (e.g. [3]), the interviews suggested that trustworthiness and relevance were enhanced by appropriate branding and making it clear that the results presented were based on research (ideally from a trustworthy source) and on relevant data. Previous experience (e.g. [3]) suggests that people are sensitive to cues of quality of evidence (such as sizes of datasets and relevance of the population on which the evidence is based), as well as assessing the information’s source.

Interpretation: what the public already know, want to know and need to know, about their personal risks from COVID-19

Based on work published in: https://doi.org/10.1098/rsos.201721The following does not report the results of a statistical analysis; these conclusions are drawn from the descriptive data alone. We found widespread demand for personalized information on people’s risk of dying from COVID-19 if they caught it, and their risk of catching it. A minority thought that they would not want such information. The majority also stated that they would want that information in numerical form, rather than as simple ‘low, medium, high’ categories. The reasons given in interviews in favour of a simple categorization were to help people with low numeracy, and to be more persuasive in terms of getting people to change their behaviour. Reasons given against included that it might be more frightening for those who were told they were at high risk. To get a sense of how individuals’ perceptions of dying from COVID-19 if they caught it compared with their actual risk, we estimated the latter by applying an approximation of the online COVID-AGE algorithm by Coggon et al. [1] to the demographic and health data provided by participants. This approximation was necessarily imprecise as we did not collect the full breadth of health information that the full COVID-AGE uses, but nevertheless provided a rough estimate of actual risk levels. <img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5wb8153000109mm4t354h9n" alt="Graphical user interface, application, table

Description automatically generated">Figure 1. The ratings of personal risk of participants (on a 1–9 Likert scale) against their approximate actual risk (presented as ‘COVID-AGE’) as calculated approximately by the algorithm by Coggon et al. [1], as a means of assessing the degree to which people’s assessments of their risk from COVID-19 correlate with their actual risk (survey 1, n = 500). <ol><li>Coggon D, Croft P, Cullinan P, Williams A. 2020 Assessment of workers’ personal vulnerability to covid-19 using ‘covid-age.’ Occup. Med. (Chic Ill) 70, 461–464. (doi:10.1093/occmed/kqaa150) </li></ol>

Description of survey responses of members of the UK public to opinion questions about risks posed by COVID-19, and desire for information about this risk

Analysis of survey findings: what the public already know, want to know and need to know, about their personal risks from COVID-19

These Results are based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a> This study was approved by the Psychological Research Ethics Committee at the University of Cambridge (PRE.2020.070).All stages of the study were carried out between the 3 June 2020 and 23 July 2020.The questions were split over four different surveys carried out with members of the public. All quantitative experiments were carried out online, with participants recruited through the ISO-accredited polling company Respondi. Participants were paid £2.50 and each survey lasted 20–25 min. Participants in the quantitative surveys were quota sampled so as to be representative of the UK population on age and sex. In survey 1, they were also quota sampled by ethnicity to be representative of the UK (we prioritized ethnicity representation in this survey over representation by sex, as we believed it more important when collecting information to be representative on ethnicity in this context, hence there was a slight difference in sex ratio in survey 1). See Table 1.Participants were excluded from participating in more than one survey. In addition, demographic information was collected: age, sex registered at birth, COVID-19 risk perception (measured by six items from [1]), numeracy (using a sum of their score on the adaptive Berlin numeracy test [2], three items from [3] and a single item from [4], household income, employment status, whether they had, or suspected they had, coronavirus, perceived social status [5], health literacy (using subscale 9, ‘Understanding health information well enough to know what to do’, of the Health Literacy Questionnaire [6], ethnicity and education level.Table 1: Characteristics of participants in the surveys<table><tbody><tr><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1">Survey 1 (n=500)</td><td colspan="1" rowspan="1">Survey 2 (n=700)</td><td colspan="1" rowspan="1">Survey 3 (n=1820)</td><td colspan="1" rowspan="1">Survey 4 (n=2500)</td></tr><tr><td colspan="1" rowspan="1">Sex (%)</td><td colspan="1" rowspan="1">MaleFemale</td><td colspan="1" rowspan="1">234 (46.8)266 (53.2)</td><td colspan="1" rowspan="1">336 (48)364 (52)</td><td colspan="1" rowspan="1">873 (48)947 (52)</td><td colspan="1" rowspan="1">1199 (48)1301 (52)</td></tr><tr><td colspan="1" rowspan="1">Ages(%)</td><td colspan="1" rowspan="1">18-2425-3435-4445-5455-6465+</td><td colspan="1" rowspan="1">56 (11.2)91 (18.2)98 (19.6)99 (19.8)86 (17.2)70 (14)</td><td colspan="1" rowspan="1">78 (11.2)136 (19.4)134 (19.1)133 (19)118 (16.9)101 (14.4)</td><td colspan="1" rowspan="1">229 (12.6)337 (18.5)344 (18.9)346 (19)310 (17)254 (14)</td><td colspan="1" rowspan="1">303 (12.1)454 (18.1)480 (19.2)480 (19.2)429 (17.2)354 (14.2)</td></tr><tr><td colspan="1" rowspan="1">Ethnicity(%)</td><td colspan="1" rowspan="1">WhiteBlackAsianMixedOther</td><td colspan="1" rowspan="1">422 (84.4)26 (5.2)17 (3.4)11 (2.2)7 (1.4)</td><td colspan="1" rowspan="1">636 (90.9)6 (0.9)37 (5.3)9 (1.3)5 (0.7)</td><td colspan="1" rowspan="1">1597 (87.7)38 (2.1)123 (6.8)30 (1.6)6 (0.3)</td><td colspan="1" rowspan="1">2174 (87)64 (2.6)160 (6.4)55 (2.2)15 (0.6)</td></tr><tr><td colspan="1" rowspan="1">Education(%)</td><td colspan="1" rowspan="1">No qualGCSE-levelA-levelBachelorsMastersDoctoralOther</td><td colspan="1" rowspan="1">21 (4.2)116 (23.2)140 (28)133 (26.6)49 (9.8)13 (2.6)19 (3.8)</td><td colspan="1" rowspan="1">25 (3.6)163 (23.2)178 (25.4)207 (29.6)72 (10.3)17 (2.4)29 (4.1)</td><td colspan="1" rowspan="1">78 (4.3)407 (22.4)495 (27.2)525 (28.8)189 (10.4)44 (2.4)55 (3.0)</td><td colspan="1" rowspan="1">116 (4.6)578 (23.1)692 (27.7)717 (28.7)250 (10)49 (2.0)71 (2.9)</td></tr><tr><td colspan="1" rowspan="1">Experience of COVID-19</td><td colspan="1" rowspan="1">Confirmed infectionUnconfirmed infectionNo infection</td><td colspan="1" rowspan="1">1 (0.2)77 (15.4)422 (84.4)</td><td colspan="1" rowspan="1">4 (0.6)99 (14.1)597 (85.3)</td><td colspan="1" rowspan="1">18 (1.0)275 (15.1)1526 (83.8)</td><td colspan="1" rowspan="1">14 (0.6)373 (15.0)2113 (84.5)</td></tr><tr><td colspan="1" rowspan="1">Numeracy*(%)</td><td colspan="1" rowspan="1">12345678</td><td colspan="1" rowspan="1">19 (3.9)59 (12.1)80 (16.4)75 (15.4)93 (19.1)93 (19.1)24 (4.9)44 (9)</td><td colspan="1" rowspan="1">47 (7)67 (9.9)100 (14.8)120 (17.8)119 (17.7)121 (18)40 (5.9)60 (8.9)</td><td colspan="1" rowspan="1">171 (9.7)177 (10)280 (15.8)355 (20.1)287 (16.2)279 (15.8)83 (4.7)138 (7.8)</td><td colspan="1" rowspan="1">166 (6.7)250 (10.1)363 (14.7)445 (18)467 (18.9)428 (17.3)137 (5.5)217 (8.8)</td></tr></tbody></table>*Score 1-8 on using a sum of their score on the adaptive Berlin numeracy test [2], 3 items from [3] and a single item from [4]The full data from the surveys is available in the Open Science Framework repository: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.17605/OSF.IO/AUF8H%20Survey%204">https://doi.org/10.17605/OSF.IO/AUF8H Survey 4</a>.Example quotes from the free text of participants about why they selected their estimate for their survival should they catch COVID-19 (sex M/F, age):“The recovery rate is good” M58“Nobody knows until there’s a cure it’s 50/50.” M53“I'm young with no health conditions and I don't smoke” F18“NHS saves white life first” M30“I think the risk is there. Just how our body cope with the infection and recovering and from the news it seems like more illness are related to the virus which seems to be worrying..” M43“My age is probably against me.” F69“I am old and fat.” F55“50/50 chance.” M67“Diabetic, blood pressure, likely candidate.” F66“I think the treatment is getting better the more doctors understand the illness” F62“Just don’t think I will.” F64<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5v196kh000109i4lzp5blkw" alt="Chart, histogram

Description automatically generated">Figure 1: The proportion of people answering 1–7 on a Likert scale ‘I would like to know now my personal risk of dying from COVID-19 if I were to catch it.’ (surveys 2&amp;3 combined, n = 2520).<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5v197c9001409i4agpytjcu" alt="Chart, bar chart

Description automatically generated">Figure 2: The proportion of people answering 1–7 on a Likert scale ‘I think that people are entitled to know now what their personal risk of dying from COVID-19 would be if they were to catch it.’ (survey 1, n = 500).<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5v19724000709i4v7fel49r" alt="Chart, bar chart

Description automatically generated">Figure 3: Participants’ rating on a 7-point Likert scale of how much they agreed or disagreed with the statement ‘If my doctor was going to tell me now my personal risk of dying from COVID-19 if I caught the virus, I would like to know that risk as a precise number<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5v1989t002709i4moy1d1j7" alt="Chart, histogram

Description automatically generated">Figure 4 Participants’ rating on a 7-point Likert scale of how much they agreed or disagreed with the statement ‘I would not like my employer to know a result like this about my personal risk from COVID-19’ (surveys 2&amp;3 combined n=2520)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5v197u7002109i4yjzjfvwp" alt="Chart, bar chart

Description automatically generated">Figure 5: Participants’ ranking of ‘Which of the following pieces of information would you most want to know about your risk, if it were available?’ (surveys 2&amp;3 combined n=2520)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5v198s7003309i46nbk4uid" alt="Table

Description automatically generated">Figure 6: Participants’ ranking of the importance of different pieces of information in a hypothetical personal COVID-19 risk communication tool. Triangle indicates mean. (surveys 2&amp;3 combined n=2520)a)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5v19958003909i4ohhl99qe" alt="Chart, histogram

Description automatically generated">b)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5v199gw004509i4c988s1zz" alt="Chart, funnel chart

Description automatically generated">Figure 7: a) Participants’ answers on a 7-point Likert scale of answer to “A tool like this can be designed in many different ways, which give people a different impression of the numbers. The tool can be made as neutral as possible, to try to just inform people and let them make up their own mind about the risk to them and how they should behave. Or the differences between numbers can be made very obvious, making the risk look larger or smaller. This would be more persuasive and make people more likely to change their behaviour. How do you think a national tool should be designed?”b) (For those who marked 4-7 on question above): “In which direction do you think the tool should try to persuade people?” (Survey 4, n=2500)<ol><li>Dryhurst S, Schneider CR, Kerr J, Freeman ALJ, Recchia G, van der Bles AM, Spiegelhalter D, van der Linden S. 2020 Risk perceptions of COVID-19 around the world. J. Risk Res. 23, 1466–4461. (doi:10.1080/13669877.2020. 1758193)</li><li>Cokely ET, Galesic M, Schulz E, Ghazal S, Garcia- Retamero R. 2012 Measuring risk literacy: the Berlin numeracy test. Judgm. Decis. Mak. 7, 25–47. (doi:10.1037/t45862-000)</li><li>Schwartz LML, Woloshin SS, Black WCW, Welch HGH. 1997 The role of numeracy in understanding the benefit of screening mammography. Ann. Intern. Med. 127, 966–972. (doi:10.7326/0003-4819-127-11- 199712010-00003)</li><li>Lipkus IM, Samsa G, Rimer BK. 2001 General performance on a numeracy scale among highly educated samples. Med. Decis. Mak. 21, 37–44. (doi:10.1177/0272989X0102100105)</li><li>Adler NE, Epel ES, Castellazzo G, Ickovics JR. 2000 Relationship of subjective and objective social status with psychological and physiological functioning: preliminary data in healthy white women. Health Psychol. 19, 586–592. (doi:10.1037/0278-6133.19.6.586)</li><li>Osborne RH, Batterham RW, Elsworth GR, Hawkins M, Buchbinder R. 2013 The grounded psychometric development and initial validation of the Health Literacy Questionnaire (HLQ). BMC Public Health 13, 1–7. (doi:10.1023/ A:1026291218634)</li></ol>

This study was approved by the Psychological Research Ethics Committee at the University of Cambridge (PRE.2020.070). 

Results of surveys: finding out what the public already know, want to know and need to know, about their personal risks from COVID-19

This Interpretation is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>&nbsp;The analysis of the data - from 2,500 members of the UK general public who were asked to rate how risky they felt five different numerical risk levels were (0.1%, 1%, 5%, 12%, 20%) when presented to them in a random order - confirms the collected findings of previous studies.&nbsp;The numbers were all presented in one of four formats: as percentages, as frequencies out of 100, as frequencies out of 1000, or in the ‘1 in X’ format. Percentages were rated as conveying the lowest risk, ‘x in 100’ as about 10 percentage points higher in risk, ‘x in 1000’ as higher again (about 13 percentage points higher than the percentage figure) and ‘1 in x’ as the highest risk.&nbsp;Peters et al [1] had also found that frequencies (out of 100) were perceived as more risky than percentages, and this new study extends their findings over two orders of magnitude. In fact, our difference between the perception of percentages and frequencies out of 100 of about 10 percentage points seems to be very similar in a study by Windschitl [2]. The data also confirms the findings of ‘ratio bias’ that are well known from previous literature (see Denes-Raj et al [3] and Reyna &amp; Brainard [4]), where 10 in 1000 is perceived as higher than 1 in 100. However, it is shown here that this effect appears to extend broadly across a range of ‘out of 1000’ figures.&nbsp;Several psychological bases for these differences in perception have previously been suggested, as reviewed in Reyna &amp; Brainard [4]. One theory is that the more ‘concrete’ a scenario – something that can easily be brought to the imagination – the more likely it seems. Under this theory the ‘1 in x’ numbers, where the ‘x’ is often small, are the most easily imagined as they involve only a small number of imaginary persons. A percentage is a more abstract concept, and so not so easily imagined - seeming less concrete and less likely [3-4]. For the frequencies expressed as ‘x out of 100’ or ‘x out of 1000’, people concentrate on the numerator, as this is a smaller number and so more easily imagined, and so they seem more concrete and riskier than the percentages, and the ‘x out of 1000’ has larger numerators so feels larger (the denominator tends to be neglected) [4]. Hopefully this new data comparing 4 formats directly will allow theorists to draw more conclusions about the psychological basis for these different perceptions (particularly, perhaps, comparing the responses of participants on the first number they saw versus the later numbers, where the analysis discussed here already shows that the response to later numbers was influenced by the context of the previous numbers).&nbsp;The fact that the variance around the answers of those rating the risks displayed as a percentage was lower than those rating them as a frequency could indicate that these participants were perhaps using the answer slider as a 0–100 number line (even though it was not explicitly labelled), and hence were judging the percentage as a distance along this line. However, the fact that these results are in line with those found in other experiments – including those where participants were asked questions such as whether they would release a prisoner based on their chance of re-offending, with that chance being given as a percentage or a frequency [3] - &nbsp;suggests that they are likely to be robust.&nbsp;1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Peters E, Hart PS, Fraenkel L. 2011 Informing patients: the influence of numeracy, framing, and format of side effect information on risk perceptions. Med. Decis. Mak. 31, 432–436. (doi:10.1177/0272989X10391672)2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Windschitl, P. (2002). Judging the accuracy of a likelihood judgment: The case of smoking risk. Journal of Behavioral Decision Making, 15, 19−35.3.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Slovic, P., Monahan, J., &amp; MacGregor, D. G. (2000). Violence risk assessment and risk communication: The effects of using actual cases, providing instructions, and employing probability vs. frequency formats. Law and Human Behavior, 24, 271−296.4.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Reyna, V.F. and Brainerd, C.J., 2008. Numeracy, ratio bias, and denominator neglect in judgments of risk and probability.&nbsp;Learning and individual differences,&nbsp;18(1), pp.89-107.&nbsp;

Interpretation of an analysis of the effects of different numerical formats on people’s perception of their risk of dying from COVID-19

This Method is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>&nbsp;A combination of in depth qualitative interviews and large-scale quantitative surveys are ideal for gaining insights into people’s understanding of a topic such as their risk from COVID-19, and their information needs.Quantitative surveys can give broad insights into opinions across a population. Table 1 gives an outline of questions designed to help gain these insights.In our prior research [1], we found that a sample size of 700 per country was more than adequate to characterize predictors of COVID-19 risk perception in that country. Further analysis of the earliest UK dataset in [1] indicated that the mean value of 700 participants versus just the first 500 differed by less than a quarter of a point on every 7-point Likert item, suggesting that 500 participants would be adequate for a descriptive study to characterize attitudes on similar topics.<table><tbody><tr><td colspan="1" rowspan="1">Theme</td><td colspan="1" rowspan="1">Questions asked</td><td colspan="1" rowspan="1">Answer options</td></tr><tr><td colspan="1" rowspan="2">Personal estimated risk</td><td colspan="1" rowspan="1">If you caught coronavirus, how likely do you think it is that you would die from it?&nbsp;&nbsp;Why did you rate your chance at this level?&nbsp;</td><td colspan="1" rowspan="1">9-point Likert scale:Not at all likely-Almost certain&nbsp;Free text</td></tr><tr><td colspan="1" rowspan="1">What is your age?Would you say your weight/BMI is:&nbsp;Have you had an organ transplant?Have you had any form of cancer in the last 12 months?If so, have you undergone treatment for your cancer in the last 12 months?&nbsp;Do you suffer from COPD (Chronic obstructive pulmonary disorder)?&nbsp;Do you suffer from asthma?&nbsp;&nbsp;Are you currently taking steroids or steroid inhalers for any medical condition?&nbsp;Are you currently taking any medicine that suppresses your immune system?&nbsp;Are you aware of suffering any medical condition that suppresses your immune system?&nbsp;Do you suffer from diabetes?&nbsp;</td><td colspan="1" rowspan="1">&nbsp;5-point Likert scale:Very Low, Low, Average, High, Very High&nbsp;Yes/No&nbsp;Yes/NoYes/No&nbsp;&nbsp;Yes/No&nbsp;No/Yes – low level/Yes – moderate level/Yes – severe&nbsp;Yes/No&nbsp;&nbsp;Yes/No&nbsp;&nbsp;Yes/No&nbsp;No/Yes – Type 1/Yes – Type 2</td></tr><tr><td colspan="1" rowspan="1">What information the audience want</td><td colspan="1" rowspan="1">I would like to know now what my personal risk of dying from COVID-19 would be if I were to catch the virus&nbsp;I think that people are entitled to know now what their personal risk of dying from COVID-19 would be if they were to catch it&nbsp;If my doctor was going to tell me now my personal risk of dying from COVID-19 if I caught the virus, I would like to know that risk as a precise number, rather than as a category (e.g. low, medium, high)&nbsp;I would not like my employer to know my personal risk of dying from COVID-19&nbsp;I feel that I have enough information already about my personal risks from COVID-19&nbsp;I would like to know by how much each personal behavioural change (e.g. wearing face masks, washing hands) reduces my personal risk of catching the virus&nbsp;I would not trust any information about my personal risk of dying from COVID-19 as I don’t believe enough is known about it&nbsp;</td><td colspan="1" rowspan="1">7-point Likert scales: Completely disagree - Completely agree</td></tr><tr><td colspan="1" rowspan="2">&nbsp;</td><td colspan="1" rowspan="1">Which of the following pieces of information would you most want to know about your risk, if it were available? Please drag and drop into the order you feel puts the most important at the top:&nbsp;The risk of you dying of COVID-19 if you caught it;The risk of you catching COVID-19 given where you live and how many other cases have been reported nearby (will vary day-to-day a bit like a pollen count);The risk of you being hospitalised by COVID-19 if you caught it;Whether you might suffer long-term complications from COVID-19 if you caught it</td><td colspan="1" rowspan="1">Rank order </td></tr><tr><td colspan="1" rowspan="1">&nbsp;Is there any other information about your risk from COVID-19 that you would like to know that we haven't mentioned?</td><td colspan="1" rowspan="1">Free text</td></tr><tr><td colspan="1" rowspan="1">&nbsp;</td><td colspan="1" rowspan="1">Please drag and drop the following into the order you feel puts the most important at the top:&nbsp;The person's risk score: the chances of that person dying if they caught COVID-19;Information about what data was used to calculate that risk score (e.g. whether it was data from the UK, how much data the researchers had);&nbsp;Information about who developed the maths behind the calculation;&nbsp;Information about the certainty and precision of the risk score: how much lower or higher the actual risk for that person could be;Context about that risk score: how it rates compared to a similar person at different ages (e.g. 'someone like you but aged 20'; 'someone like you but aged 60');&nbsp;Context about that risk score: where that person's risk lies in comparison to everyone else in the UK (e.g. a graph of the risks of everyone in the UK with an arrow showing 'you are here' to make it clear how many people have a higher risk than that person, and how many a lower risk);Link to where people can get more information about things they can do to reduce their chances of catching COVID-19 (e.g. hand washing, social distancing, wearing face masks etc);Information reminding people that as well as the risk to themselves, they also pose a risk to others (in case they catch COVID-19 and pass it on);Details about how the risk score was calculated (e.g. what factors makes someone's risk higher or lower, and how important each of those factors are);Information about thing they can do to reduce their chances of dying from COVID-19 if they caught it (such as losing weight)&nbsp;</td><td colspan="1" rowspan="1">Rank order</td></tr><tr><td colspan="1" rowspan="1">&nbsp;</td><td colspan="1" rowspan="1">“A tool like this can be designed in many different ways, which give people a different impression of the numbers. The tool can be made as neutral as possible, to try to just inform people and let them make up their own mind about the risk to them and how they should behave. Or the differences between numbers can be made very obvious, making the risk look larger or smaller. This would be more persuasive and make people more likely to change their behaviour. How do you think a national tool should be designed?”&nbsp;(For those who marked 4-7 on question above):“In which direction do you think the tool should try to persuade people?”</td><td colspan="1" rowspan="1">7-point Likert scale: As neutral as possible: just inform people - As persuasive as possible: change people's behaviour 3 options:It should try to reassure people by showing the risk is generally low; It should try to make people be more cautious by showing that even if the risk is low to them, they can spread it to others; It should try to be persuasive to different people in different ways (reassuring some and making others more cautious)&nbsp;</td></tr></tbody></table>&nbsp;1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Dryhurst S, Schneider CR, Kerr J, Freeman ALJ, Recchia G, van der Bles AM, Spiegelhalter D, van der Linden S. 2020 Risk perceptions of COVID-19 around the world. J. Risk Res. 23, 1466–4461. (doi:10.1080/13669877.2020. 1758193)&nbsp;

Method for online surveys to find out what the public already know, want to know and need to know, about their personal risks from COVID-19

This Hypothesis/Theoretical rationale is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>One method of approaching risk communication is the ‘mental models’ approach, whereby researchers use both qualitative and quantitative methods to build up an understanding of the audience’s intuitive model of causality in a situation, as well as the probabilities they ascribe to the likelihoods of different events and the strengths of causal relationships. The idea is then to provide the audience with information to help bring their understanding of the situation closer to the expert understanding of it, correcting any misunderstandings or providing missing pieces of information [1,2].In order to apply this method to COVID-19, it would be useful to know firstly how accurate people’s estimate of the likelihood of them dying from COVID-19 is. This includes two likelihoods: the likelihood of catching it (which depends very much on local prevalence as well as a person’s behaviour, and so is an ever-changing number), combined with the likelihood of dying from it if they catch it (which depends on their health risk factors, as well as the availability and effectiveness of treatments and so is more stable). It is probably, therefore, only possible to ask people to estimate the second: the risk of themselves dying from COVID-19 if they catch it – and then compare that with estimates from risk calculators, based on information about their risk factors.It would then be important to build up a picture of what people understand the major risk factors to be, how important they feel they are, and what information they would actually want to know about their risks. This can be done through a combination of in depth qualitative interviews and quantitative surveys covering a broad diversity of people.1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Granger Morgan M, Fischoff B, Bostrom A, Atman CJ. 2002 Risk communication: a mental models approach. Cambridge, UK: Cambridge University Press.2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;Fischhoff B. 1999 Why (cancer) risk communication can be hard. Monogr. Natl Cancer Inst. 25, 7–13. (doi:10.1093/ oxfordjournals.jncimonographs.a024213)&nbsp;

Finding out what the public already know, and want to know, about their personal risks from COVID-19

These Results are based on work published in: https://doi.org/10.1098/rsos.201721The data from this experiment is available in the Open Science Framework repository: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.17605/OSF.IO/AUF8H%20Survey%204">https://doi.org/10.17605/OSF.IO/AUF8H Survey 4</a>.This study was approved by the Psychology Research Ethics Committee at the University of Cambridge (PRE.2020.070).The survey was carried out online, with 2,500 participants recruited through the ISO- accredited polling company Respondi (the power calculation was done for another experiment within the same survey that had more arms than this one, and so required more participants). Quotas were set to ensure recruitment of UK participants representative of the national population on age and sex. Participants were paid £2.50 and the survey lasted 20–25 min, and included several different sections. This experiment came after sections asking some personal risk perception and COVID-19 experience questions, and an experiment in which participants were asked to estimate the likelihood of 5 different individuals (whose risk factors were described) dying from COVID-19 if they caught it. Characteristics of the participants are shown in Table 1Table 1: Characteristics of the participants in the survey<table><tbody><tr><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1">Total n=2500</td></tr><tr><td colspan="1" rowspan="1">Sex (%)</td><td colspan="1" rowspan="1">MaleFemale</td><td colspan="1" rowspan="1">1199 (48)1301 (52)</td></tr><tr><td colspan="1" rowspan="1">Ages(%)</td><td colspan="1" rowspan="1">18-2425-3435-4445-5455-6465+</td><td colspan="1" rowspan="1">303 (12.1)454 (18.1)480 (19.2)480 (19.2)429 (17.2)354 (14.2)</td></tr><tr><td colspan="1" rowspan="1">Ethnicity(%)</td><td colspan="1" rowspan="1">WhiteBlackAsianMixedOther</td><td colspan="1" rowspan="1">2174 (87)64 (2.6)160 (6.4)55 (2.2)15 (0.6)</td></tr><tr><td colspan="1" rowspan="1">Education(%)</td><td colspan="1" rowspan="1">No qualGCSE-levelA-levelBachelorsMastersDoctoralOther</td><td colspan="1" rowspan="1">116 (4.6)578 (23.1)692 (27.7)717 (28.7)250 (10)49 (2.0)71 (2.9)</td></tr><tr><td colspan="1" rowspan="1">Experience of COVID-19</td><td colspan="1" rowspan="1">Confirmed infectionUnconfirmed infectionNo infection</td><td colspan="1" rowspan="1">14 (0.6)373 (15.0)2113 (84.5)</td></tr><tr><td colspan="1" rowspan="1">Numeracy*(%)</td><td colspan="1" rowspan="1">12345678</td><td colspan="1" rowspan="1">166 (6.7)250 (10.1)363 (14.7)445 (18)467 (18.9)428 (17.3)137 (5.5)217 (8.8)</td></tr></tbody></table>*Score 1-8, based on a sum of participants’ score on the adaptive Berlin numeracy test [1], 3 items from [2] and a single item from [3]As per the pre-registered method, to investigate how participants would perceive a range of realistic absolute risks of dying from COVID-19 (if infected), and to quantify the effects of numerical format on these perceptions, participants were asked to rate five different numerical risks (presented to them in a random order) on a sliding scale marked only ‘very low risk’ and ‘very high risk’ at the extremes. Their choice of position on the slider was coded as 0–100 by the survey software. Each participant saw the same risks, but presented in one of five different formats:- Estimates presented as frequency, ‘ X in 100’- Estimates presented as frequency, ‘ X in 1000’- Estimates presented as frequency, ‘ 1 in X’- Estimates presented as percentage, X%- Estimates presented as percentage, X%, responses captured on 11pt Likert scale rather than a slider (as an exploratory check)<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/cl5qjz5l5000109mhplt7y7jl" alt="Chart, scatter chart, box and whisker chart

Description automatically generated">The mean responses from the 4 different formats that used a sliding scale are shown in Figure 1, with 95% confidence intervals as bars, and the full distribution of responses shown as a violin plot.Figure 1. Mean (95% CI) ratings (from ‘very low risk’ (0) to ‘very high risk’ (100)) of five different COVID-19 infection fatality risk figures, presented as a percentage or as one of three frequency formats. Violin plots indicate underlying distribution.Mean risk ratings and SDs across risk levels and formats are also shown in Table 2 below.Table 2: The mean risk ratings and SDs that participants gave to each of the four formats, across risk levels and formats<table><tbody><tr><td colspan="1" rowspan="1">Risk level</td><td colspan="4" rowspan="1">Format</td></tr><tr><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1">Percentage</td><td colspan="1" rowspan="1">X in 100</td><td colspan="1" rowspan="1">X in 1000</td><td colspan="1" rowspan="1">1 in X</td></tr><tr><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1">M (SD)</td><td colspan="1" rowspan="1">M (SD)</td><td colspan="1" rowspan="1">M (SD)</td><td colspan="1" rowspan="1">M (SD)</td></tr><tr><td colspan="1" rowspan="1">0.1%</td><td colspan="1" rowspan="1">12.26 (24.09)</td><td colspan="1" rowspan="1">17.91 (25.94)</td><td colspan="1" rowspan="1">23.02 (27.22)</td><td colspan="1" rowspan="1">29.13 (26.20)</td></tr><tr><td colspan="1" rowspan="1">1%</td><td colspan="1" rowspan="1">15.69 (23.27)</td><td colspan="1" rowspan="1">27.05 (29.00)</td><td colspan="1" rowspan="1">31.25 (27.34)</td><td colspan="1" rowspan="1">44.46 (29.11)</td></tr><tr><td colspan="1" rowspan="1">5%</td><td colspan="1" rowspan="1">23.83 (25.18)</td><td colspan="1" rowspan="1">33.37 (27.36)</td><td colspan="1" rowspan="1">40.28 (27.84)</td><td colspan="1" rowspan="1">57.81 (29.34)</td></tr><tr><td colspan="1" rowspan="1">12%</td><td colspan="1" rowspan="1">35.42 (27.09)</td><td colspan="1" rowspan="1">42.77 (26.99)</td><td colspan="1" rowspan="1">49.02 (27.04)</td><td colspan="1" rowspan="1">65.93 (29.66)</td></tr><tr><td colspan="1" rowspan="1">20%</td><td colspan="1" rowspan="1">45.36 (26.94)</td><td colspan="1" rowspan="1">54.23 (27.56)</td><td colspan="1" rowspan="1">58.34 (26.49)</td><td colspan="1" rowspan="1">73.50 (29.35)</td></tr></tbody></table><ol><li>Cokely ET, Galesic M, Schulz E, Ghazal S, Garcia- Retamero R. 2012 Measuring risk literacy: the Berlin numeracy test. Judgm. Decis. Mak. 7, 25–47. (doi:10.1037/t45862-000)</li><li>Schwartz LML, Woloshin SS, Black WCW, Welch HGH. 1997 The role of numeracy in understanding the benefit of screening mammography. Ann. Intern. Med. 127, 966–972. (doi:10.7326/0003-4819-127-11- 199712010-00003)</li><li>Lipkus IM, Samsa G, Rimer BK. 2001 General performance on a numeracy scale among highly educated samples. Med. Decis. Mak. 21, 37–44. (doi:10.1177/0272989X0102100105)</li></ol>

This work was funded by the Winton Centre for Risk & Evidence Communication at the University of Cambridge, which is financed by a donation from the David & Claudia Harding Foundation.

Results of testing the effects of different numerical formats on people’s perception of their risk of dying from COVID-19

This Method is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>This method was pre-registered before data was collected, at: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.17605/OSF.IO/RDTC4">https://doi.org/10.17605/OSF.IO/RDTC4</a>&nbsp;Participants will be asked to imagine they have received a personalised estimate of their risk of dying if infected with COVID-19. They will be asked ‘How would you classify that risk in your mind?’ and be given a sliding scale from ‘Very low risk’ to ‘Very high risk’. In one exploratory condition participants will instead respond on an 11-point Likert scale in order to test whether there is a difference between Likert scales (which have previously been used in the literature on this topic) and sliders. This is because we have some concerns that those being given numbers as a percentage might merely use the slider scale to measure out the approximate percentage of the entire slider scale distance, rather than to indicate the degree of risk that they perceived. The sliders will all have the marker shown starting in the middle of the scale.&nbsp;Unseen to participants, the position of the marker along the slider will be coded 0-100, with the starting position of the marker anchored at 50.&nbsp;Participants will be presented with five different COVID-19 risk estimates (in random order) to rate [0.1%, 1%, 5%, 12%, 20%; or equivalent, depending on condition]. We will refer to these as risk levels.&nbsp;There are five experimental conditions, and participants will be randomly assigned to one condition:&nbsp;- Estimates presented as frequency, ‘ X in 100’- Estimates presented as frequency, ‘ X in 1000’- Estimates presented as frequency, ‘ 1 in X’- Estimates presented as percentage, X%- Estimates presented as percentage, X%, responses captured on 11pt Likert (exploratory)&nbsp;Where X represents appropriate the percentage, numerator, or denominator for the risk level.&nbsp;Mean perceived risk scores will be compared between conditions (excluding the exploratory condition with a Likert scale response item) and risk levels using a 4x5 two-way ANOVA.&nbsp;The condition with the Likert scale will have the scores rescaled to align with the slider scale. Then mean risk ratings between this condition and the condition presenting the risks as percentages but with a slider response will be compared a 2(condition)x5(risk level) ANOVA.&nbsp;We will also explore the possibility of treating risk levels as a single continuous variable and examining, pair-wise, the effect of format as well as interactions with risk level using a regression based approach, i.e. comparing linear or quadratic slopes.&nbsp;Participants will also complete a measure of numeracy: the Berlin Numeracy Test [1] and the Schwartz Numeracy Scale [2], scores combined as recommended by Cokely et al [1]. This will allow the potential to investigate the effects of numeracy on participants’ responses.&nbsp;1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Cokely ET, Galesic M, Schulz E, Ghazal S, Garcia- Retamero R. 2012 Measuring risk literacy: the Berlin numeracy test. Judgm. Decis. Mak. 7, 25–47. (doi:10.1037/t45862-000)2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Schwartz LML, Woloshin SS, Black WCW, Welch HGH. 1997 The role of numeracy in understanding the benefit of screening mammography. Ann. Intern. Med. 127, 966–972. (doi:10.7326/0003-4819-127-11- 199712010-00003)

Testing the effects of different numerical formats on people’s perception of their risk of dying from COVID-19

This Problem is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>Different communicators might have different aims when approaching the communication of personalized risk from COVID-19. Some may feel it important to encourage people to take mitigating actions, and hence to ensure that risks are perceived as relatively high to increase motivation [1,2]. Others might want to reduce worry and hence ensure that risks are perceived as low. There is no ‘correct’ perception of a risk, as risk perception is inherently subjective and incorporates both likelihood and personal impact (which depends on personal vulnerabilities and priorities) [3].In order to be able to provide any useful information, however, it is important to understand what the public’s current state of knowledge on the subject is, and what information they particularly want to have (or specifically not want to have).1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Bish A, Michie S. 2010 Demographic and attitudinal determinants of protective behaviours during a pandemic: a review. Br. J. Health Psychol. 15, 797–824. (doi:10. 1348/135910710X485826)2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Dryhurst S, Schneider CR, Kerr J, Freeman ALJ, Recchia G, van der Bles AM, Spiegelhalter D, van der Linden S. 2020 Risk perceptions of COVID-19 around the world. J. Risk Res. 23, 1466–4461. (doi:10.1080/13669877.2020. 1758193)3.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Slovic, P. and Peters, E., 2006. Risk perception and affect.&nbsp;Current directions in psychological science,&nbsp;15(6), pp.322-325.

What information do the public need and want to know about their personal risk from COVID-19?

This Problem is based on work published in: <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1098/rsos.201721">https://doi.org/10.1098/rsos.201721</a>Probabilistic information can be represented numerically in different ways, and research has shown that these different formats can affect the perception of likelihoods as well as affecting the ease with which people can make mental comparisons and manipulations of the information [1-5. In the case of people’s risk of dying should they catch COVID-19, the probabilities being represented are often going to be very small (down to around 1 in 10,000 chance of dying), but also fall along a very wide range (up to 25% chance), depending on risk factors, which presents a particular challenge. Common formats include percentages (e.g. 0.5%), natural frequencies (e.g. 5 in 1000), and ‘1 in X’ (e.g. 1 in 200), each of which may have different effects on people’s comprehension and perception of the risk. It is clear that communicators would want to aim for high comprehension (which could be measured as both gist and verbatim), but there is no such thing as a ‘correct’ absolute perception of the risk – although communicators should be aware of the differences that the different formats make to perception.1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Pighin S, Savadori L, Barilli E, Cremonesi L, Ferrari M, Bonnefon JF. 2011 The 1-in-X effect on the subjective assessment of medical probabilities. Med. Decis. Mak. 31, 721–729. (doi:10.1177/0272989X11403490)2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Denes-Raj V, Epstein S, Cole J. 1995 The generality of the ratio-bias phenomenon. Pers. Soc. Psychol. Bull. 21, 1083–1092. (doi:10.1177/ 01461672952110009)3.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Siegrist M, Orlow P, Keller C. 2008 The effect of graphical and numerical presentation of hypothetical prenatal diagnosis results on risk perception. Med. Decis. Mak. 28, 567–574. (doi:10.1177/0272989X08315237)4.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Woloshin S, Schwartz LM. 2011 Communicating data about the benefits and harms of treatment: a randomized trial. Ann. Intern. Med. 155, 87–96. (doi:10.7326/0003-4819-155-2- 201107190-00004)5.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Peters E, Hart PS, Fraenkel L. 2011 Informing patients: the influence of numeracy, framing, and format of side effect information on risk perceptions. Med. Decis. Mak. 31, 432–436. (doi:10.1177/0272989X10391672)

When presenting people with their personal risk from COVID-19, what format should they be shown the numbers in?

Making BRCA1 genetic test reports easier to understand through user-centered design: A randomized trial

How People Understand Risk Matrices, and How Matrix Design Can Improve their Use: Findings from Randomized Controlled Studies

Correlates of intended COVID-19 vaccine acceptance across time and countries: results from a series of cross-sectional surveys

How well did experts and laypeople forecast the size of the COVID-19 pandemic?

How do the UK public interpret COVID-19 test results? Comparing the impact of official information about results and reliability used in the UK, USA and New Zealand: a randomised controlled trial

COVID-19 risk perception: a longitudinal analysis of its predictors and associations with health protective behaviours in the United Kingdom

Communicating personalized risks from COVID-19: guidelines from an empirical study

Communicating risks and benefits to cardiology patients

Predictors of COVID-19 vaccine acceptance across time and countries

How do the public interpret COVID-19 swab test results? Comparing the impact of official information about results and reliability used in the UK, US and New Zealand: a randomised, controlled trial

Title	DOI	Published date
Making BRCA1 genetic test reports easier to understand through user-centered design: A randomized trial	10.1016/j.gim.2022.04.016	2022-08
How People Understand Risk Matrices, and How Matrix Design Can Improve their Use: Findings from Randomized Controlled Studies	10.1111/risa.13822	2022-05
Correlates of intended COVID-19 vaccine acceptance across time and countries: results from a series of cross-sectional surveys	10.1136/bmjopen-2020-048025	2021-08
How well did experts and laypeople forecast the size of the COVID-19 pandemic?	10.1371/journal.pone.0250935	2021-05-05
How do the UK public interpret COVID-19 test results? Comparing the impact of official information about results and reliability used in the UK, USA and New Zealand: a randomised controlled trial	10.1136/bmjopen-2020-047731	2021-05
COVID-19 risk perception: a longitudinal analysis of its predictors and associations with health protective behaviours in the United Kingdom	10.1080/13669877.2021.1890637	2021-04-03
Communicating personalized risks from COVID-19: guidelines from an empirical study	10.1098/rsos.201721	2021-04
Communicating risks and benefits to cardiology patients	10.1136/heartjnl-2020-317593	2020-12-11
Predictors of COVID-19 vaccine acceptance across time and countries	10.1101/2020.12.09.20246439	2020-12-11
How do the public interpret COVID-19 swab test results? Comparing the impact of official information about results and reliability used in the UK, US and New Zealand: a randomised, controlled trial	10.1101/2020.12.04.20243840	2020-12-04
Communicating personalised risks from COVID-19: guidelines from an empirical study	10.1101/2020.10.05.20206961	2020-10-06
Susceptibility to misinformation about COVID-19 around the world	10.1098/rsos.201199	2020-10
Risk perceptions of COVID-19 around the world	10.1080/13669877.2020.1758193	2020-08-02
Creating genetic reports that are understood by nonspecialists: a case study	10.1038/s41436-019-0649-0	2020-02
Correction: Creating genetic reports that are understood by nonspecialists: a case study.	10.1038/s41436-019-0663-2	2020-01
Projected Impact of Dengue Vaccination in Yucatán, Mexico.	10.1371/journal.pntd.0004661	2016-05
Archaeology Through Computational Linguistics: Inscription Statistics Predict Excavation Sites of Indus Valley Artifacts.	10.1111/cogs.12311	2015-10
Graph-Theoretic Properties of Networks Based on Word Association Norms: Implications for Models of Lexical Semantic Memory.	10.1111/cogs.12299	2015-10
Encoding sequential information in semantic space models: comparing holographic reduced representation and random permutation.	10.1155/2015/986574	2015-04
Reproducing affective norms with lexical co-occurrence statistics: Predicting valence, arousal, and dominance.	10.1080/17470218.2014.941296	2014-09
The semantic richness of abstract concepts.	10.3389/fnhum.2012.00315	2012-11
The role of semantic diversity in lexical organization.	10.1037/a0026727	2012-06
The shape of action.	10.1037/a0024310	2011-11
Toward a scalable holographic word-form representation.	10.3758/s13428-011-0125-5	2011-09
More data trumps smarter algorithms: comparing pointwise mutual information with latent semantic analysis.	10.3758/brm.41.3.647	2009-08

Gabriel Recchia

Employment

Education

Works

Octopus publications