Data Ethics Club: Transparent communication of evidence does not undermine public trust in evidence#
Whatâs this?
This is summary of Wednesday 23th Octoberâs Data Ethics Club discussion, where we spoke and wrote about the article Transparent communication of evidence does not undermine public trust in evidence by John R. Kerr, Claudia R. Schneider, Alexandra L. J. Freeman, Theresa Marteau, and Sander van der Linden. The summary was written by Jessica Woodgate, who tried to synthesise everyoneâs contributions to this document and the discussion. âWeâ = âsomeone at Data Ethics Clubâ. Huw Day helped with the final edit.
Article Summary#
The paper examines whether public trust in scientific information is undermined by communicating risks, benefits, and uncertainties according to a guideline set of rules. In conducted experiments, participants read either a persuasive method, or a balanced and informed method adhering to evidence communication recommendations. The studies found that balanced messages are perceived as just as trustworthy or more trustworthy than persuasive messages, noting that prior beliefs moderated how the balanced messages were perceived. In one of the studies, participants who had read the persuasive message voiced significantly stronger support for the issue, despite rating the message as less trustworthy.
Discussion Summary#
Were you surprised by the findings of the study? Do they mirror experiences in your own domain?#
We found the paperâs discussion around the findings thought provoking and interesting to read, although were not surprised by the results. Findings were a bit self-evident and the covid results were especially intuitive; we felt the balanced approach to be more trustworthy in both contexts. We questioned whether results were that strong from a psychological standpoint.
Whilst we agree that there are benefits to having balanced communication, we wondered if the paper diminishes the case for persuasive communication. Sometimes, it is important to tailor communication to the audience. It can be difficult to maintain neutrality, as engaging people in statistics and numbers without any emotion or story is challenging. From our experience with upskilling people in data and communicating directly with industry partners, we found that science in itself should but may not be convincing enough.
Consistently providing balanced information is not always a viable goal as people will interpret messages in different ways. People have a tendency to adopt insights in favour of various conclusions and widen confidence intervals in their minds. Data needs to be interpreted to be understood, and this interpretation is coloured by the experiences of the person analysing the data. When working with childrenâs data, for example, there are recommendations at the ends of reports which the reader will interpret; points like âneglectâ may be experienced in different ways. The way data will be interpreted is also influenced by the way the data is presented. The best propaganda is truth, but it is easy to present enough and not all information. Being selective with which parts of the truth are portrayed can foster false narratives, highlighting the impact information control.
Sometimes, it is logistically challenging to convey balanced information. For example, in vaccination centres not all volunteers are able to pass on information; there is more focus on collecting consent for vaccination than giving statistics. We wondered where to locate expertise if people have more questions. People are not always capable of being informed. For example, when giving epidurals to people in labour the urgency of the situation overrides attempts to fully discuss risks.
Identifying where exactly the balance lies is not easy and sometimes attempting to provide balance can cause more harm than good. For issues which are much more weighted, including both sides might do more harm than good. For example, presenting a climate researcher and a fossil fuels business owner as 50/50 in climate change issues can detract from accurately presenting facts as one side of the argument is much more incentivised than the other.
However, making some attempt at providing a balanced approach is important as sometimes only attempting to persuade and omitting balanced information can backfire. Even if the information provided in the persuasive case is generally true, disregarding generally irrelevant but more complete information can foster misinformation and conspiracy theories. We have found that people in industry can have a different idea of what constitutes sufficient evidence, and we have had to explain to them the difference between scientific and anecdotal evidence. We would like to upskill everyone in data skills so that they get a better understanding of how to interpret data, but some of us thought this might not work. In medicine, for example, it might be difficult to put interpretive skills into the hands of patients without guiding their hands.
Discussing how to communicate data interpretation and teach interpretive skills led us to ask whether the roles of medical professionals are to persuade or inform. The informed consent model is very important in the context of medicine. Consent and informing are closely monitored in clinical trials; providing information to those who need or want it is a necessity. Psychologists among us found it surprising that midwives are informing over persuading. Systems tend to focus on educating a public who doesnât âknow enoughâ; we have found the Spanish healthcare system to be very paternalistic with restricted access to information for patients. Paternalistic attitudes could be helpful or harmful to the patients; there is a spectrum from informing to persuading to coercing. Examples of coercing are policies that stop welfare payments for people that arenât vaccinated.
What do you think is behind the difference between the results of the nuclear power study and the vaccine study?#
The paper found that there were different trends of opinions in the vaccination and nuclear scenarios; differences between the studies may occur because the mechanism for participation in each situation is quite different. Whilst there is active and individual participation in the vaccination scenario, there is passive and public participation in the nuclear scenario. It is much easier for people to take personal action in the case of vaccines than nuclear power. The proximity to each scenario is different; whilst it is fairly common for people to interact with vaccines or be jabbed involuntarily as a child, directly confronting the concept of nuclear power (e.g. by visiting Hiroshima) is much rarer. Whilst the existential risk is on a personal level for vaccination, it is more community oriented for nuclear. Saying this, vaccines do also influence herd immunity and the choice to vaccinate can be made for community benefit (co-incidental with individual benefit).
The type and level of life experience that people have also affects the way they engage with issues. This study was conducted in the UK, but we would be curious as to the attitudes of different countries towards evaluating the news and informing themselves. We tend to do studies in our home countries and conclude that they are universally applicable. However, there are cultural differences; France, Switzerland, and Germany are not necessarily taught that the news is neutral and are largely aware that there is implicit bias. In Russia, on the other hand, the news is tightly controlled and questioning official messaging is discouraged.
Timing of the study may have an influence over results, as experiments were conducted post-covid in 2021. The outcome of the study might have been different if it was done pre-covid, as during covid a lot of people were living under very specific restrictions. Now, and before covid, nobody was prevented from entering bars and restaurants because they didnât have a flu jab. Anti-vaccine sentiment was likely amplified for many during covid, but pro-vaccine sentiment could also have been increased among those who wanted more freedom. In the wake of fake news and misinformation exploding into the public domain, people might be less likely to trust arguments they encounter.
Results for each study may be suffering from selection bias as the study was conducted online which restricts the data set. In the vaccine study, there was a sampling problem as there werenât a lot of people with strong anti-vaccination prior beliefs and the population was fairly homogeneous in this regard. Statistics are thus missing due to low numbers of people with particular life experiences and beliefs. Having a homogeneous population could explain why it is difficult to see a moderating effect on the trustworthiness of information.
People have a variety of prior opinions which influence how they interact with different issues and how likely they are to trust certain arguments. For example, it seems likely that people are more likely to have a pre-formed opinion on vaccines than nuclear science. Having pre-formed opinions can affect how people perceive an argument. If a âbalancedâ view includes both sides of an argument, people may see their side of the argument presented and think âoh good, you agreed with meâ, not considering that the other view is also presented.
Different issues also incite different emotional responses and social pressure, as nothing is ever fully neutral. Sometimes, people manipulate emotions to scare people into taking action by using specific examples like âI had a friend who took this vaccine and diedâ. Social pressure to take a particular view changes for different issues; itâs more acceptable to be sceptical of nuclear power than vaccines, possibly because there is more evidence of the drawbacks of nuclear power. Getting vaccinated generally is viewed as quite a pro-social thing to do; it would be interesting to conduct the study with another more personal issue with less community focus (e.g. diet or fitness). We also wondered to what extent the results were specific to the covid vaccine versus other vaccines. Covid particularly is surrounded by somewhat novel interventions such as lockdowns which could influence how people respond.
People will react differently according to the person that is trying to convince them. There are conflicting results regarding the effects that bots have on convincing people about conspiracy theories. Some studies have found that bots can get conspiracy theorists to change their mind; other research finds that facts are ineffective in changing conspiracy theoristsâ minds. We wondered at what point does a conspiracy theory cross the line to become enlightened.
How people interpret messages can be affected by the type of education they have. Data literacy is key for people to understand what the data is telling us, and stories are often required to help us understand data. In many schools and even universities, critical thinking isnât taught very well, which may influence how people make decisions regarding trust. The study doesnât match groups for reading age or education level, so it is unclear how education may have affected the results.
Considering that people interpret information in different ways, we wondered if ânudgingâ is a good idea. Nudging revolves around subtly trying to get people to behave in certain ways through low information provision. As this paper demonstrates, persuasion techiques may not be a one size fits all and can backfire when people need more information than they are provided with. When receiving a nudge, tools should be able to provide extra information when the receiver requests more information, but this does not always happen.
Should research articles have a place in persuading the public, or should their intention always be to focus on robust, trustworthy information?#
Attempting to persuade is not always sufficient as persuasive techniques might miss important information, such as recognising the doubts that people have or acknowledging the negatives. Most of the time people know when youâre trying to manipulate them. Information should always be available, and being open about both sides can strengthen arguments by highlighting things which might not have otherwise been talked about. In saying this, we should be practical with respect to the fact that we live in a society and have to be strategic about where information is kept and who provides it.
On the other hand, sometimes being persuasive is needed to make change happen and for things to improve we donât always want to be neutral. If you donât make the case for your viewpoint, sometimes it can be ignored. Sometimes the facts are available, but they arenât very impactful unless there is a story that prompts enough public shock for change to happen. Except for political opinions which have been built over decades, a lot of opinions can be easily swayed. Although, even political opinions can be changed; historically a lot of sexual misconduct was seen as the womanâs fault but today this is changing.
Getting the right balance between explaining things in enough detail to communicate robust information and not overwhelming people is tricky. People often say they want nuance, but a lot of the time nuance isnât effective and explaining things clearly is more likely to garner support. When politicians explain complex and nuanced things they are criticised from both sides but receive a more welcoming response when they use more lucid language. On a personal level, some people are more statistically driven than others. Researchers can be very involved in their specialities and all the background which underpins it, but it is important to remember that the general public donât need to know every detail.
When trying to communicate something scientific to the public, it is important to think about how it will be heard by people with less scientific understanding. From our experience in the veterinary field, we have found that people are quite good at removing jargon but can forget that even some non-jargon or non-scientific words can be difficult for a wider audience to interpret. Communicating maths is also challenging, and a lot of people are thrown off when academic terms are used. It is important to be able to explain things better and in less abstract terms. To be appropriate for a wider audience, it is beneficial to aim for a young reading age of 5-9; in 2012, 1 in 6 adults in England had a literacy level equivalent to the ages of 5-7. Using a young reading age as a starting point and then gradually building up the complexity helps to create a more accessible narrative. Some of us have a series of graphs we use to communicate good or bad science. In an example of a graph we use to explain âto a 6 year oldâ, we essentially just make it obvious that the further to the right a point is, the worse it is. People seem to intuitively understand this even though there are other complicated things happening, including confidence intervals.
Tailoring research articles so that they can be understood by the public may not be necessary as many people wonât read the articles directly. Most people will get their information through people in the middle that interpret research and influence public views. Weâve seen academics on Twitter using strong language and interpreting the same data completely differently. More and more journals are also doing plain language summaries. Good summaries donât influence the number of people who read the full article, but they do influence the number of people who talk about the article in blogs, on social media, and in the news. Whether the article itself should be easy to read may depend on the topic; 9 year olds donât need to understand high energy physics, but making sure public health research is interpretable could be beneficial.
It is difficult to say how important it is that researchers should intend to make articles trustworthy, as the extent to which we can conflate trust with choosing an appropriate action is challenging to discern. We were uncertain about the extent to which trust influences peopleâs decisions, and wondered about the significance of people trusting scientific research and not taking a recommended action. There seems to be low trust in institutions at the moment, which could explain why people arenât taking recommended actions like getting vaccinated.
What change would you like to see on the basis of this piece? Who has the power to make that change?#
We see relevance of the paper to our own work; the goal of the paper was to help people who need to communicate information to do so in a trustworthy fashion, and communication is an important part of many of our roles. The studies in the paper were the culmination of a lot of work to identify guidelines for trustworthy informative (not persuasive) communication. We liked how there was a detailed reflection on how the authors would have done the studies differently if they did it again. The guidelines are pertinent for some of our current work with a team who want to push an intervention based on a result that may not be sound due to missing data. The guidelines are also relevant to regulatory compliance, where people often have to make accountable decisions risking audit. Being precise with making accountable decisions may be more relevant in the public sector than the private sector, as the public sector may involve more compliance informed and recorded decision making.
The paper recommends pre-empting misinterpretations, but we wondered how achievable this is in reality. Often attempts to pre-empt misinterpretations may not pierce peopleâs wilful ignorance as many are much more comfortable resting in the illusion that everything is fine. However, if people are emphatic that thereâs no issue whatsoever when there are issues, we can run into problems. It is key to find the balance between pre-empting misunderstandings, whilst being open about risk associated with all scientific interventions. As a community, we need to get better at communicating the contingency of our scientific findings.
Attendees#
Huw Day, Data Scientist, Jean Golding Institute, https://www.linkedin.com/in/huw-day/
Amy Joint, Programme Manager, ISRCTN Clinical Study Registry, BioMed Central https://www.linkedin.com/in/amyjoint/
Vanessa Hanschke, PhD Student, University of Bristol -Noshin Mohamed, Service Manager for QA in childrenâs and young peopleâs service
Euan Bennet, Lecturer, University of Glasgow
Alex Freeman, ex-Winton Centre, Cambridge University (co-author on paper)
Paul Smith, Statistician, NHS Blood and Transplant (Bristol)
Sarah Jones, vet working in clinical pathology lab at Glasgow uni
Kamilla Wells, Citizen Developer, Australian Public Service, Brisbane
Zoë Turner Senior Data Scientist, NHS
Jessica Bowden, Neuroscience Research Associate, University of Bristol
Robin Dasler, product manager for academic data curation software, California
Dan Levy, Data Analyst, BNSSG ICB (NHS, Bristol), https://www.linkedin.com/in/danrsl/
Veronica Blanco Gutierrez, Midwife and PhD candidate in Digital Health, University of Bristol https://www.linkedin.com/in/veronica-blanco-gutierrez/