Chapter c: presenting probabilities

2012 UPDATED CHAPTER C:
PRESENTING PROBABILITIES
SECTION 1:
AUTHORS/AFFILIATIONS
Lyndal Trevena
Vrije Universiteit (Free University) Medical Center Children’s Hospital of Eastern Ontario Max Planck Institute for Human Development Dartmouth Institute for Health Policy and Clinical The University of Texas MD Anderson Cancer Center Max Planck Institute for Human Development Suggested Citation: Trevena L, Zikmund-Fisher B, Edwards A, Gaissmaier W, Galesic M, Han P, King J, Lawson M, Linder S, Lipkus I, Ozanne E, Peters E, Timmermans D, Woloshin S. (2012). Presenting probabilities. In Volk R & Llewellyn-Thomas H (editors). 2012 Update of the International Patient Decision Aids Standards (IPDAS) Collaboration's Background Document. Chapter C. http://ipdas.ohri.ca/resources.html. SECTION 2:
CHAPTER SUMMARY

What is this quality dimension?
Medical decisions have outcomes that may have been quantified through research. To assist
patients and health professionals in balancing the benefits and harms of these options, decision
aids aim to communicate estimates of the likelihoods of these outcomes based on the best
available evidence.
What is the theoretical rationale for including this quality dimension?
When considering decision options and the likelihoods of their outcomes, estimates of both the
changes and the outcome frequencies associated with each option need to be conveyed in a way
that maximizes patients’ understanding and thereby facilitates informed decision making.
What is the evidence to support including or excluding this quality dimension?
Sixteen out of the 86 RCTs in the updated Cochrane review of decision aids for people facing
treatment and screening decisions measured the effects of including probabilities on the accuracy
of patients’ risk understanding. Presenting probabilities within a DA significantly improved the
accuracy of risk comprehension (RR 1.7, 95% CI 1.5 to 2.1), compared with not receiving
probability estimates. In updating this section, we have tried to identify the key issues in
communicating quantitative information for decision aid development and to draw on the
combined expertise of the authors to summarise current evidence in this field, but not to provide
a systematic review. We summarise the evidence for ten key issues in presenting the likelihood
of decision outcomes. The key points are summarized below.
Suitable formats for presenting numeric chances depend on the nature of the task. To present the
chance of a single event, simple frequency formats such as “Every year, 1 in 100 deaths in the
EU is due to stomach cancer”, or percents such as “Every year, 1% of deaths in the EU are due to
stomach cancer” are best understood; the denominator should always be defined. Formats should
aim to be consistent throughout a decision aid. 1 in x formats should be avoided. When the task
is to compare the chance of occurrence of two or more independent events (e.g., the chance of
symptom relief with drug A compared with placebo), formats that express the chance of an event
using one number, such as percentages, work better than simple frequencies involving more than
one number, such as 1 in 100. When presenting changes in rates, preferably absolute risks
should be given either in percentages or simple frequencies, and if possible along with the
absolute risk increase (or decrease). With frequencies, the same denominator should always be
used. If a decision aid requires people to calculate the probabilities associated with jointly
occurring events, then a natural frequency format would be preferable to conditional
probabilities.
Providing context for risk statistics in the form of comparative data and providing evaluative
labels can have a substantial influence on how patients use that information. However, the choice
of comparative data has a strong impact on how people respond to such data, and labels should
be applied carefully. The effect of tailoring health risk information on improving health decision-
making appears mixed.
Evidence on the effects of conveying uncertainty is limited but growing. Novel representational
methods have been developed to communicate both randomness (aleatory uncertainty), and
ambiguity (epistemic uncertainty), and may be useful to incorporate in decision aids. However,
the communication of uncertainty can be psychologically aversive, and more research is needed.
On a similar note, communicating the likelihood of outcomes over time is particularly difficult;
although significant research has addressed this issue, no clear consensus regarding optimal
methods has emerged.
When designing a decision aid, measuring objective numeracy, subjective numeracy, graph
literacy, and possibly other aspects of the health literacy of the prospective users can help in
designing presentation formats that are suitable for individuals who vary in skills.
Visual aids such as icon array diagrams, bar charts, human figure representations, or flow
diagrams appear to aid accurate understanding of probabilities in many contexts. They can help
reduce several biases, such as denominator neglect, framing effects, and the undue influence of
anecdotes, and they can aid the comprehension of more complicated concepts such as
incremental risk. They can be especially helpful for people with higher graphical literacy and
among those who have problems with understanding and applying numbers. Emerging formats
such as animated or interactive visual displays are intuitively appealing, but evidence is lacking
to determine whether these techniques provide a net positive experience or degrade knowledge
versus evidence-based static formats.
Using positive and negative framed narratives to present benefit and risk information may
increase perceptions of risk severity, decrease the ability to accurately recall risk probabilities,
and influence treatment choice. The relative number and type of narratives used influences
decision making; they should be used with caution and be accompanied by quantitative or visual
displays such as pictographs.
SECTION 3:
DEFINITION (CONCEPTUAL/OPERATIONAL) OF THIS QUALITY DIMENSION

a)
Updated Definition

Medical decisions have outcomes which may have been quantified through research. To assist
patients and health professionals in weighing up the benefits and harms of these options, decision
aids aim to communicate estimates of the likelihoods of these outcomes based on the best
available evidence.
b)
Changes from Original Definition

See Appendix.

c)
Emerging Issues/Research Areas in Definition

See Section 6.

SECTION 4:
THEORETICAL RATIONALE FOR INCLUDING THIS QUALITY DIMENSION
a)
Updated Theoretical Rationale

When considering decision options and the likelihoods of their outcomes, estimates of both the
changes and the outcome frequencies associated with each option need to be conveyed in a way
that maximizes patients’ understanding and thereby facilitates informed decision making.
b)
Changes from Original Theoretical Rationale

See Appendix.

c)
Emerging Issues/Research Areas in Theoretical Rationale

See Section 6.
SECTION 5:
EVIDENCE BASE UNDERLYING THIS QUALITY DIMENSION

a)
Updated Evidence Base & Changes from Original Evidence Base
In 2011, the Cochrane Collaboration’s review of decision aids for people facing treatment and
screening decisions was updated; it now includes 86 RCTs and reports on outcomes according to
the IPDAS criteria [Stacey et al. (1)]. Sixteen out of the 86 RCTs measured the effects of
including probabilities on the accuracy of patients’ understanding of these. Presenting
probabilities within a DA significantly improved the accuracy of risk comprehension (RR 1.7,
95% CI 1.5 to 2.1), compared with not receiving probability estimates. This effect was greater if
the probabilities were presented as numbers, but was also significant if they were described in
words although the effect size was smaller.
However, the literature on risk communication—particularly beyond decision aid-specific
research—is vast. In updating this section, we have tried to identify the key issues in
communicating quantitative information for decision aid development and to draw on the
combined expertise of the authors to summarise current evidence in this field, but not to provide
a systematic review. This is a rapidly emerging area of research and there has been substantive
new evidence on presentation of rates, visual formats, uncertainty, and interactive web-based
formats.

This chapter summarises the evidence for ten key issues in presenting the likelihood of decision
outcomes.
 To reflect new knowledge in this field, we have divided the original chapter sub-section on
‘Presenting Numbers’ into two sub-sections: ‘Presenting the Chance an Event Will Occur’ and ‘Presenting Changes in Numeric Outcomes. This latter category now includes ‘Framing’, which had previously been a separate sub-section. - ‘Probabilities for Tests and Screening Decisions’ to ‘Outcome Estimates for Tests and - ‘Probabilities in Context’ to ‘Numerical Estimates in Context and Evaluative Labels’; - ‘Tailoring Probabilities’ to ‘Tailoring Estimates’; and - ‘Visual Aids’ to ‘Visual Formats’.  We have added new sub-sections on ‘Formats for Understanding Outcomes over Time’, ‘Narrative Methods for Conveying Numerical Estimates’, and ‘Important Skills for Understanding Numerical Estimates’.  The previous sub-section called ‘Evidence for Probabilities Used’ remains unchanged. Throughout these ten sub-sections, the cited evidence has been derived from the decision-aid specific, other health, and non-health literature. Reference: 1. Stacey D, Bennett CL, Barry MJ, Col NF, Eden KB, Holmes-Rovner M, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database of Systematic Reviews 2011(10) CD001431
1. Presenting the Chance an Event Will Occur

For both written and verbal information, patients have a more accurate understanding of risk if
probabilistic information is presented as numbers rather than words, even though some may
prefer receiving words [Trevena et al. (1)].

Suitable formats for presenting numeric chances depend on the nature of the task. The
terminology in this field can be very confusing so to simplify in this section, we will refer to
percents (e.g., 10%) and simple frequencies (e.g., 1 in 100), regardless of whether these are
normalized or not.
When the task is to present the chance of a single event, simple frequency formats such as
“Every year, 1 in 100 deaths in the EU is due to stomach cancer”, or percents such as “Every
year, 1% of deaths in the EU are due to stomach cancer”, are more transparent than formats such
as “The chance of dying from stomach cancer is 1%”. The last statement is problematic because
it does not specify the ‘denominator’ (i.e. reference class) -“all deaths in the EU per year”.
Without a clear description of who this estimate refers to, people might think this statement
means that “Every year, 1 of 100 citizens of the EU dies of stomach cancer” [Gigerenzer et al.
(2)]. Similarly, when patients who take fluoxetine for mild depression hear from their doctor that
there is a “30-50% chance of developing a sexual problem such as impotence or loss of sexual
interest” some may think this means they will have problems in 30% of their own sexual
encounters. The ‘denominator’ or reference class used by the doctor is ‘patients on fluoxetine’
but the denominator used by the patient is ‘their own sexual encounters’ [Gigerenzer & Galesic
(3)].

There is also some evidence that risks presented in simple frequencies are perceived as higher
than when they are presented in their equivalent percentage value, especially in patients with
lower numeracy [Peters et al. (4)] and (possibly) with smaller percents [Woloshin & Schwartz
(5)]. Given this potential format bias, one should be careful when comparing results of studies
that have used different formats (percents or simple frequencies). Formats should aim to be consistent throughout a decision aid (see below). Providing simple frequency AND percent appears to add no advantage [Woloshin & Schwartz (5)] and there is strong evidence that ‘1 in x’ formats with variable denominators are more difficult to understand. They should be avoided for all tasks. In summary, it is most important when presenting the chance of a single event to clearly define the denominator or reference class. Percent or simple frequency formats can be used for presenting the chance of a single event. However, in deciding which one to use, consider what other information needs to be presented in the same document and what the purpose of the decision aid is overall so that format consistency can be achieved. Visual formats may also help to reduce bias (see section 6). We will expand on this throughout the chapter. When the task is to compare the chance of occurrence of two or more independent events (e.g., the chance of symptom relief with drug A compared with placebo), formats that express the chance of an event using one number, such as percentages, work better than simple frequencies involving more than one number, such as 1 in 100 [Woloshin & Schwartz (5)]. If using simple frequencies such as 1 in 100, use the same denominator (e.g. 1 in 100 vs. 2 in 100) as they are easier to compare than frequencies using different denominators (e.g. 1 in 100 vs. 1 in 50) [Peters et al. (4); Cuite et al. (6)]. Thus, consistent denominators should always be used. When choosing the size of the denominator, smaller numbers (e.g. 100) are easier to understand and remember than larger numbers (e.g. 10,000) [Garcia-Retamero & Galesic (7)]. There has been discussion about whether people find percents less than one (e.g., 0.1%) more difficult to understand than the equivalent simple frequency (e.g., 1 in 1000) [Woloshin & Schwartz (5); Cuite et al. (6)]. However, this problem may reflect difficulty manipulating decimal points (e.g.,, asking someone to represent 1 in 1000 as a percentage) rather than a comprehension problem. [Woloshin & Schwartz (5)]. In summary, percents (e.g., 1 %) may have an advantage over a simple frequency format (e.g., 1 in 100) for comparing the chance of occurrence of two or more independent events. As mentioned before, it remains important to clearly define the denominator or ‘reference class’ and to aim for a consistent format throughout the decision aid taking into account the information and tasks required. Other formats are more suitable for tasks that involve presenting changes in numeric outcomes (section 2) and conveying the frequency of joint occurrences of two or more events, such as the probability that a person with a positive test result has the disease (section 3). NB: This chapter contains evidence for a number of other strategies to improve the presentation of numeric information and these sections should all be read and interpreted together as a whole. References 1. Trevena L, Davey H, Barratt A, Butow P, Caldwell P. A systematic review on communicating with patients about evidence. Journal of Evaluation in Clinical Practice. 2006;12(1):13-23. 2. Gigerenzer G, Gaissmaier W, Kurz-Milcke E, Schwartz LM, Woloshin S. Helping doctors and patients make sense of health statistics. Psychological Science in the Public Interest. 2007:8(2):53–96. 3. Gigerenzer G, Galesic, M. Why do single event probabilities confuse patients? Statements of frequency are better for communicating risk. British Medical Journal. 2012; 344:e245. Peters E, Sol Hart P, Fraenkel L. Informing Patients: the Influence of Numeracy, Framing and Format of Side Effect information on Risk Perceptions. Medical Decision Making 2011; 31(3): 432-6 5. Woloshin S, Schwartz L. Communicating data about the benefits and harms of treatment. Annals of Internal Medicine. 2011;155(2):87-96. 6. Cuite C, Weinstein N, Emmons K, Colditz G. A test of numeric formats for communicating risk probabilities. Risk Communication. 2008; 28(3):377-84. 7. Garcia-Retamero R, Galesic M. Using plausible group sizes to communicate information about medical risks. Patient Education and Counseling. 2011; 84(2):245-250.
2. Presenting Changes in Numeric Outcomes

Most efforts to communicate changes due to interventions (i.e. treatment effects) or across time
(e.g., improvements of health) use either side-by-side total risk presentations or difference
presentations.
Difference presentations depict the change in risk and can influence risk perceptions through
framing effects. Research has shown that relative risk presentations (e.g. “30% lower risk”) tend
to magnify risk perceptions and decrease understanding, compared to absolute risk presentations
(e.g., “the risk is lower by 5 percentage points”) [for a review see Akl et al. (1), and Covey (2)].
Number needed to treat (NNT) is sometimes used, but several studies suggest that this format is
poorly understood by patients and may increase the perceived effect of treatment [Halvorsen et
al. (3)].
A variant is the presentation of incremental risk (absolute risk increase), but only after a
“baseline” total risk level has been shown. This approach emphasizes the size of the change
relative to the size of the total risk, and was shown to lower risk perceptions [Zikmund-Fisher et
al. 4)]. Such language (e.g., “5 more women get…”) was incorporated into the Schwartz et al.
drug facts box [Schwartz et al. (5)], and used in decision tools such as Adjuvant! A decision aid
trial suggested that incremental risk language works best when accompanied by visual displays
[Zikmund-Fisher et al. (6)]. In particular, when the baseline risks are small, relative risk
reductions are perceived to be larger and more persuasive than absolute risk reductions [Akl et
al. (1); Malenka et al. (7); and Naylor et al. (8)].
The framing of outcomes in terms of losses or gains has been shown to affect people’s choices
[Tversky & Kahneman (9)]. Framing outcomes in terms of potential gains (e.g., the chances of
survival) often generates risk-averse choices, whereas framing outcomes in terms of potential
losses (e.g., the chances of death) often generates risk-seeking choices. In clinical situations, the
effects of the framing of outcomes as losses or gains tend to vary across situations [Moxey et al.
(10)]; the variable effect of different frames of risks or rates is due to emphasizing different
aspects of the information. For instance, framing information in terms of relative risk reductions
affects people’s choices by overemphasizing the benefits of therapies.
In summary, when presenting changes in rates, preferably absolute risks should be given either in
percentages or simple frequencies, and if possible along with the absolute risk increase (or
decrease). If frequencies are used, the denominators should be equal.
References 1. Akl EA, Oxman AD, Herrin J, Vist GE, Terrenato I, Sperati F, et al. Using alternative statistical formats for presenting risks and risk reductions. Cochrane Database Systematic Reviews. 2011;3:CD006776. 2. Covey J. A meta-analysis of the effects of presenting treatment benefits in different formats. Medical Decision Making. 2007; 27(5):638-54 3. Halvorsen PA, Selmer R, Kristiansen IS. Different ways to describe the benefits of risk-reducing treatments.
Annals of Internal Medicine. 2007; 146(12):848–56.
4. Zikmund-Fisher BJ, Fagerlin A, Roberts TR, Derry HA, Ubel PA. Alternate methods of framing information
about medication side effects: Incremental risk versus total risk occurrence. Journal of Health Communication.
2008; 13(2):107-24
5. Schwartz LM, Woloshin S, Welch HG. Using a drug facts box to communicate drug benefits and harms. Annals
of Internal Medicine. 2009;150(8):516-27
6. Zikmund-Fisher BJ, Ubel PA, Smith DM, Derry HA, McClure JB, Stark A, et al. Communicating side effect risks
in a tamoxifen prophylaxis decision aid: The debiasing influence of pictographs. Patient Education and Counselling.
2008; 73(2):209-14
7. Malenka DJ, Baron JA, Johansen S, Wahrenberger JW, Ross JM. The framing effect of relative and absolute risk.
Journal of General Internal Medicine. 1993; 8(10):543-8
8. Naylor CD, Chen E, Strauss B. Measured enthusiasm: does the method of reporting trial results alter perceptions
of therapeutic effectiveness? Annals of Internal Medicine 1992;117(11):916–21.
9. Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981; 211(4481):453-
8.
10. Moxey A, O'Connell D, McGettigan P, Henry D Describing treatment effects to patients. Journal of General
Internal Medicine. 2003; 18(11):948-59.

3. Outcome Estimates for Tests and Screening Decisions

As mentioned in section 1 of this chapter, the ideal format for presenting numeric information
depends on the task. Once again, the terminology can be confusing when considering screening
and test outcomes. [Gigerenzer & Hoffrage (1)] and [Hoffrage et al. (2)] proposed the term
‘natural frequencies’ for one specific task only – the probability of joint occurrence of events
(e.g., the probability of having breast cancer given an abnormal mammography result). This is
quite different to the task of considering the chance of independent events (see section 1).
A number of studies and a recent Cochrane review [Akl et al. (3); Hoffrage et al. (5)] have
shown that natural frequencies are better than conditional probabilities where events are
connected. Natural frequencies preserve the base rate of the outcome (e.g., breast cancer) and
report the ‘actual’ or ‘natural’ number of people having a particular outcome (e.g., having a
positive test result). It’s unclear whether people use Bayesian reasoning when making screening
decisions but natural frequency formats continue to be proposed as the best way to help people
understand these kinds of estimates [Gigerenzer et al. (1); Hoffrage et al. (2)]. An example of
Bayesian reasoning with a natural frequency format would be “Out of every 10,000 people, 30
have colorectal cancer. Of these, 15 will have a positive hemoccult test. Out of the remaining
9970 people without colorectal cancer, 300 will still test positive. How many of those who test
positive actually have colorectal cancer? Answer: 15 out of 315” [Hoffrage et al. (3)]. An
alternative representation of this information is conditional probability format such as
“probability of having colorectal cancer is .003%. Of people who have the cancer, 50% get a
positive test result. Of people who do not have cancer, 3% will nevertheless test positive. What
is the probability that a person who tests positive has colorectal cancer? Answer: 4.8%”. So, if a
decision aid requires people to calculate the probabilities associated with jointly occurring
events, then a natural frequency format would be preferable to conditional probabilities.
However, screening can also be viewed as an ‘intervention’ that has an effect (e.g., reducing death from colorectal cancer) and so the effect of screening on cancer mortality with and without screening are actually the chances of independent events (see section 1). As we noted earlier there may be some advantage to presenting such information as a percent format but cancer incidence and mortality rates are usually low in a general population and possible format biases need to be considered. Similarly, the chance of having a disease if your test result is positive can be thought of as the ‘post-test probability’ and some would suggest this could be calculated on behalf of the patient and presented as a percent (1%) or simple frequency format (e.g., 1 in 100). Thus, as before, we recommend that decision aid developers consider both the nature of the task required and the other information that needs to be conveyed in the same document. It is important to clarify what the reference class is (e.g., women aged 50 who are having biennial mammography over 10 years) and to keep the denominator constant. Once again, 1 in x formats should be avoided as they consistently perform worse. The current IPDAS criteria recommend that screening decision aids include estimates of: 1) disease with and without screening; 2) false positives; and 3) false negatives. The updated Cochrane review of decision aids includes 34 trials about screening and test decisions [Stacey et al. (4)]. Five of these trials measured the accuracy of risk perception [Gattelari & Ward (6); Kupperman et al. (7); Schapira & Van Ruiswyk (8) Wolf & Schorling (9); Lerman et al. (10)]. Four of these reported significantly improved risk perception [Gattelari &Ward (6); Kupperman et al. (7); Schapira & Van Ruiswyk (8) Wolf & Schorling (9)] and this was the case whether accurate risk comprehension was measured as numbers [Gattelari & Ward (6); Kupperman et al. (7)] or as a gist-based risk comprehension in words [Schapira & Van Ruiswyk (8); Wolf & Schorling (9)]. All four of the trials also included quantitative estimates in accordance with the IPDAS criteria recommendations (see above). Three of the decision aids were available and all used different formats for numerical outcomes. None provided a head-to-head comparison of formats. Natural frequency and percentage formats were used in [Gattelari e& Ward (6)] (RR 5.28 [95% CI 2.93. 9.50]), variable frequency format was used in [Wolf & Schorling (9)] (RR 1.31 [95%CI 1.10, 1.56]) and simple frequency format was used in [Schapira & Van Ruiswyk (8)] [RR 1.46 [95%CI 1.17, 1.83]. Given the lack of head-to-head format comparison in these trials we recommend applying the principles outlined in this chapter which are based (where possible) on comparative research. Our update confirms that, in screening decision aids, the application of IPDAS criteria about the presentation of quantitative estimates of screening outcomes improves the accuracy of risk perceptions. References 1. Gigerenzer G, Hoffrage U. "How to improve Bayesian reasoning without instruction: Frequency formats." Psychological Review 1995; 102(4): 684-704.
2.
Hoffrage, U, Gigerenzer G, Krauss S, Martignon L. "Representation facilitates reasoning: what natural frequencies are and what they are not." Cognition 2002; 84(3): 343-52.
3.
Akl EA, Oxman AD, Herrin J, Vist GE, Terrenato I, Sperati F, et al. Using alternative statistical formats for presenting risks and risk reductions. Cochrane Database Systematic Reviews. 2011;3:CD006776. 4. Hoffrage U, Lindsey S, Hertwig R, Gigerenzer G. Communicating statistical information. Science. Stacey D, Bennett CL, Barry MJ, Col NF, Eden KB, Holmes-Rovner M, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database of Systematic Reviews 2011; (10) CD001431 Gattellari M, Ward J. Does evidence-based information about screening for prostate cancer enhance consumer decision-making? A randomised controlled trial. Journal of Medical Screening. 2003;10(1):27-39. 7. Kuppermann M, Norton M, Gates E, Gregorich S, Learman L, Nakagawa S, et al. Computerised prenatal genetic testing decision-assisting tool. Obstetrics and Gynaecology. 2009;113(1):53-63. 8. Schapira M, VanRuiswyk J. The effect of an illustrated pamphlet decision-aid on the use of prostate cancer screening tests. The Journal of Family Practice. 2000;49(5):418-24. 9. Wolf A, Schorling J. Does informed consent alter elderly patients' preferences for colorectal cancer screening? Results of a randomized trial. Journal of General Internal Medicine. 2000;15(1):24-30. 10. Lerman C, Biesecker B, Benkendorf JL, Kerner J, Gomez-Caminero A, Hughes C, et al. "Controlled trial of pretest education approaches to enhance informed decision-making for BRCA1 gene testing." Journal of the
National Cancer Institute 1997; 89(2): 148-57.
4. Numerical Estimates in Context and Evaluative Labels

To help users get perspective on the risk of disease, decision aid developers should consider
including contextual information when feasible. Context is particularly important for decision
aids about disease prevention or cancer screening, in which the benefit is a reduction in disease
specific mortality. One way to provide context is to provide the chance of death over the next 10
years from the disease under consideration (where possible according to age, smoking status, and
other reliable risk factor information) as well as the chance of dying from other major causes and
from all causes combined [Woloshin et al. (1)].
Directly interpreting the meaning of numeric information (e.g., telling patients how good or bad
a 9% risk is) also can have a substantial influence on how patients use that information. In one
series of studies, providing evaluative labels for numeric quality-of-care information (e.g., telling
decision makers that the numbers represented “poor” or “excellent” quality of care) resulted in
greater use of this information in judgments and less reliance on an irrelevant emotional state
among the less numerate [Peters et al. (2)]. In another study, evaluative labels for test results
(that a patient’s test was “positive” or “abnormal”) induced larger changes to risk perceptions
and behavioural intentions than did numeric results alone [Zikmund-Fisher et al. (3)]. The
appropriateness of these changes, however, can be unclear in health contexts and evaluative
labels should be applied carefully.
Woloshin S, Schwartz LM, Welch HG. The risk of death by age, sex, and smoking status in the United States: Putting health risks in context. Journal of the National Cancer Institute. 2008; 100: 845 - 853] 2. Peters E, Dieckmann, N.F., Vastfjall, D., Mertz, C.K., Slovic, P., Hibbard, J.H. Bringing meaning to numbers: The impact of evaluative categories on decisions. Journal of Experimental Psychology: Applied. 2009;15(3):213-227. 3. Zikmund-Fisher BJ, Fagerlin A, Keeton K, Ubel PA. Does labeling prenatal screening test results as negative or positive affect a woman's responses? American Journal of Obstetrics and Gynecology.
2007;197(5):528.e1-528.e6
5. Conveying Uncertainty

This section considers two main types of uncertainty. Aleatory uncertainty is concerned with the
randomness of future events. Epistemic uncertainty, on the other hand, is the lack of knowledge
needed to predict future outcomes, also known as “ambiguity” and is concerned with the
imprecision in estimates which is typically expressed by confidence intervals [Ellsberg (1)]. An
understanding of these uncertainties is arguably an essential element of informed decision
making [Han et al. (2)]. However, the optimal methods and outcomes of conveying these uncertainties to patients have only begun to be explored [Politi et al. 2007 (3); Politi et al. 2011 (4)]. The communication of aleatory uncertainty has been examined in a small number of decision aid studies of both textual and novel visual methods of representing randomness (e.g., icon arrays displaying affected individuals in a scattered rather than clustered manner) [Lenert & Cher (5); Baty et al. (6); Schapira et al. (7); Han et al. (8); Ancker et al. (9)]. Available evidence suggests that these methods have no significant effect on risk perceptions, although evidence is lacking regarding their effects on patients’ understanding of uncertainty. In one study, however, the communication of randomness was associated with greater subjective uncertainty about estimated risk [Han et al. (10)]. The communication of ‘ambiguity’ has been examined in a small number of studies using confidence intervals to communicate probability estimates. These studies have shown that communicating ambiguity has little effect on risk perceptions, although it increases patient worry [Han et al. (10); Lipkus et al. (11)], and these effects appear to be moderated by representational method (visual vs. textual) and individual differences (e.g., dispositional optimism) [Han et al. (8); Han et al. (10)]. Evidence is limited and mixed regarding the extent to which confidence intervals are understood by patients [Mazor et al. (12); Muscatello et al. (13)] and influence perceptions of the credibility of probability estimates [Schapira et al. (5); Han et al. (14)]. Furthermore, the effects of communicating both epistemic and aleatory uncertainty on real medical decisions have not been evaluated. The communication of ambiguity has been evaluated more fully outside of health care. Numerous studies in behavioral decision research have shown that ambiguity leads to avoidance of decision making and pessimistic risk perceptions and affective responses (worry, distress) related to choice outcomes—a phenomenon known as “ambiguity aversion” [Ellsberg (1); Camerer et al. (15); Kuhn et al. (16); Viscusi et al. (17)]. However, most studies have examined hypothetical rather than real decisions. In summary, evidence on the effects of conveying uncertainty is limited but growing. Novel representational methods have been developed to communicate both randomness (aleatory uncertainty), and ambiguity (epistemic uncertainty), and may be useful to incorporate in decision aids. However, the communication of uncertainty can be psychologically aversive, and more research is needed to determine both the optimal representational methods and effects of communicating uncertainty on patient perceptions, understanding, and decision making. References 1. Ellsberg D. Risk, ambiguity, and the savage axioms. Quarterly Journal of Economics. 1961;75:643–69. Han PKJ, Klein WMP, Arora, NK. Varieties of uncertainty in health care: A conceptual taxonomy. Medical Decision Making 2011;31(6):828-838. 3. Politi MC, Han PKJ, Col NF. Communicating the uncertainty of harms and benefits of medical interventions. Medical Decision Making. 2007; 27(5):681-95. 4. Politi MC, Clark MA, Ombao H, Dizon D, Elwyn, G. Communicating uncertainty and its impact on patients’ decision satisfaction: A necessary cost of involving patients in shared decision making? Health Expectations. 2011;14(1):84–91. 5. Lenert LA, Cher DJ. Use of meta-analytic results to facilitate shared decision making. Journal of American Medical Informatics Association. 1999;6(5):412–9. Baty BJ, Venne VL, McDonald J, Croyle RT, Halls C, Nash JE, et al. BRCA1 testing: Genetic counseling protocol development and counseling issues. Journal of Genetic Counseling 1997; 6(2):223– 44. 7. Schapira MM, Nattinger AB, McHorney CA. Frequency or probability? A qualitative study of risk communication formats used in health care. Medical Decision Making. 2001;21(6):459–67. 8. Han PKJ, Klein WMP, Killam B, Lehman TC, Massett H, Freedman AN. 2011. Representing randomness in the communication of individualized cancer risk estimates: effects on cancer risk perceptions, worry, and subjective uncertainty about risk. Patient Education Counseling 2012; 86(1):106-113. 9. Ancker JS, Weber EU, Kukafka R. Effects of game-like interactive graphics on risk perceptions and decisions. Medical Decision Making. 2011;31(1):130–42. 10. Han PKJ, Klein WMP, Lehman TC, Killam B, Massett H, Freedman AN. Communication of uncertainty regarding individualized cancer risk estimates: Effects and influential factors. Medical Decision Making 2011; 31(2):354-366. 11. Lipkus IM, Klein WM, Rimer BK. Communicating breast cancer risks to women using different formats. Cancer Epidemiology, Biomarkers & Prevention. 2001; 10(8):895–98. 12. Mazor KM, Dodd KS, Kunches L. Communicating hospital infection data to the public: A study of consumer responses and preferences. American Journal of Medical Quality 2009; 24(2): 108-115. 13. Muscatello DJ, Searles A, Macdonald R, Jorm L. Communicating population health statistics through graphs: A randomised controlled trial of graph design interventions. BMC Medicine 2006; 4(1):33. 14. Han PKJ, Klein WMP, Lehman TC, Massett H, Lee SC, Freedman AN. Laypersons’ responses to the communication of uncertainty regarding cancer risk estimates. Medical Decision Making 2009; 29(3): 391-403. 15. Camerer C, Weber M. Recent developments in modeling preferences: Uncertainty and ambiguity. Journal of Risk and Uncertainty. 1992; 5(4): 325–70. 16. Kuhn KM. Communicating uncertainty: Framing effects on responses to vague probabilities. Organization Behavior and Human Decision Processes. 1997; 71:55–83. 17. Viscusi WK, Magat WA, Huber J. Communication of ambiguous risk information. Theory and Decision. 1991; 31(2):159–73.

6. Visual Formats

Presenting event rates with visual aids such as 100 faces diagrams, bar charts, human figure
representations, or flow diagrams may aid accurate understanding of probabilities. Visual
displays can help reduce several biases, such as denominator neglect [Garcia-Retamero, et al.
(1)], framing effects [Garcia-Retamero & Cokely (2); Garcia-Retamero & Galesic (3)], and the
undue influence of anecdotes [Fagerlin et al. (4)], and they can aid the comprehension of more
complicated concepts such as incremental risk [Zikmund-Fisher et al. (5)]. Graphs that clarify
sub-set relationships (e.g., Venn diagrams, Euler circles) can lead to better judgements, for
instance in Bayesian reasoning tasks [Barbey & Sloman (6); Sloman et al. (7)]. Others believe
graphs help, but for different reasons [Brase (8)]. However, there has been some evidence that
graphs can effect peoples’ perceptions to overestimate low probabilities and underestimate high
probabilities – the magnifier effect - [Gurmankin et al. (9)]. Others have shown the opposite
effect (i.e., less overestimation) on low probabilities and no effect on high [Woloshin et al. (10)].
While the use of visual displays is often recommended as an aid to interpretation for numerical
data [Paling (11); Spiegelhalter et al. (12)], one important caveat is that people vary in their
ability to extract data and meaning from visual displays. [Galesic & Garcia-Retamero (13)]
developed a graph literacy scale that predicts who actually profits from visual displays [Garcia-
Retamero & Galesic (13); Garcia-Retamero & Galesic (14)]. For example, visual displays are
helpful for understanding statistical information about health for people with low numeracy
[Galesic et al. (15); Garcia-Retamero & Galesic (16)]; yet people who lack graph literacy could
actually be better off with mere numbers [Gaissmaier et al. (17)].
Graphs have sometimes been shown to be suited best to convey the essential aspects of the information (i.e., “gross-level information”) [Feldman-Stewart et al. (18)], bottom line meaning, or gist [Reyna (19)], whereas numerical representations can be better suited to convey more precise aspects of the information (i.e., detailed-level information or verbatim) [Feldman-Stewart et al. (18); Hawley et al. (20)]. Thus, a potential weakness of visual displays is that people may focus more on the pattern of data than the precise values, if that is the main objective. Furthermore, some graphs are better suited for certain tasks (e.g. line graphs for trends over time, bar graphs for comparison across groups [Lipkus (21); Lipkus & Holland (22)]. But there may also be some general principles. For instance, it has been shown that the formats which are perceived most accurately and easily by patients are vertical bars, horizontal bars and systematic ovals. However, people’s preferences for a certain graph do not necessarily lead to better performance than non-preferred graphs. Furthermore, pie charts and random ovals lead to slower and less accurate estimates [Feldman-Stewart et al. (18)]. Enhancing accuracy in estimates can be aided by displaying the most crucial elements, hence omitting redundant information [Zikmund-Fisher et al. (23); Zikmund-Fisher et al. (24)], as well as by using icon arrays or stick figures that are arranged as groups in a block then random scattering - the latter of which is useful to convey the concept that events (e.g., who is afflicted by disease) occur at random [Ancker et al. (25)]. Finally, it has been shown that visual aids are most effective in accurate comprehension when the entire population at risk is shown rather than only depicting sick people, for instance [Garcia-Retamero & Galesic (14)]. In addition, for conveying small probability events (e.g. less than 1%), graphical displays (e.g., pie graphs, bar charts, etc.) that show only the number of people affected (i.e., foreground information) leads to greater risk aversion (e.g., greater willingness to pay for an improved product) than graphic displays that show part-whole relationship by including the total population or those not affected) (i.e. background) [Stone et al. (26); Stone et al. (27)]. In conclusion, visual displays can be a powerful tool to convey health related statistical information, especially for people with higher graphical literacy and among those who have problems with understanding and applying numbers. However, some caution is warranted as visual displays may not be intuitively understood by everyone, and they can be used to represent statistical information transparently, but they can also be misused to represent statistical information in a misleading way [Kurz-Milcke et al. (28)]. Overall, all visual aids should be pilot tested for understanding, and developers should take care to avoid using misleading images (such as graphs with misleading scales) or using different scales within the same patient decision aid. Finally, the field is still in dire need of a more systematic theoretical understanding of why, when, and for whom visual displays are effective [Lipkus (21)]. References 1 Garcia-Retamero R, Galesic M, Gigerenzer G. Do icon arrays help reduce denominator neglect? Medical Decision Making. 2010; 30(6): 672-84. 2. Garcia-Retamero R, Cokely ET. Effective communication of risks to young adults: Using message framing and visual aids to increase condom use and STD screening. Journal of Experimental Psychology-Applied. 2011; 17(3): 270. 3. Garcia-Retamero, R., Galesic, M. How to reduce the effect of framing on messages about health. Journal of General Internal Medicine. 2010; 25(12):1323-9. 4. Fagerlin A, Wang C, Ubel PA. Reducing the influence of anecdotal reasoning on people’s health care decisions: Is a picture worth a thousand statistics? Medical Decision Making. 2005;25(4):398–405. Zikmund-Fisher BJ, Ubel PA, Smith DM, Derry HA, McClure JB, Stark AT, Pitsch R, Fagerlin A. Communicating side effect risks in a tamoxifen prophylaxis decision aid: The debiasing influence of pictographs. Patient Education & Counseling. 2008; 73(2): 209-214. 6. Barbey AK, Sloman SA. Base-rate respect: From ecological rationality to dual processes. Behavioral and Brain Sciences. 2007; 30: 241-97. 7. Sloman S A, Over DE, Slovak L & Stibel JM. Frequency illusions and other fallacies. Organizational Behavior and Human Decision Processes. 2003; 91(2):296–309. 8. Brase GL. Pictorial representations in statistical reasoning. Applied Cognitive Psychology. 2009; Helweg-Larsen M, Armstrong K, Kimmel SE, Volpp KGM. Comparing the standard rating scale and the magnifier scale for assessing risk perceptions. Medical Decision Making. 2005; 25(5): 560-70. 10. Woloshin S, Schwartz LM, Byram S, Fischhoff B, Welch HG. A new scale for assessing perceptions of chance: a validation study. Medical Decision Making 2000; 20(3):298–307. 11. Paling J. Strategies to help patients understand risks. British Medical Journal. 2003; 327(7417): 745–748. Spiegelhalter D, Pearson M, & Short I. Visualizing uncertainty about the future. Science. 2011; 333(6048): Galesic, M. & Garcia-Retamero, R. (2011). Graph literacy: a cross-cultural comparison. Medical Decision Garcia-Retamero R & Galesic M. Who profits from visual aids: Overcoming challenges in people’s understanding of risks. Social Science and Medicine. 2010; 70(7): 1019–25. 15. Galesic M, Garcia-Retamero R, & Gigerenzer G. Using icon arrays to communicate medical risks to low- numeracy people. Health Psychology. 2009; 28(2): 210-16. 16. Garcia-Retamero R, & Galesic M. Communicating treatment risk reduction to people with low numeracy skills: A cross-cultural comparison. American Journal of Public Health. 2009; 99(12): 2196-2202. 17. Gaissmaier W, Wegwarth O, Skopec D, Müller A-S, Broschinski S, & Politi MC. Numbers can be worth a thousand pictures: Individual differences in understanding graphical and numerical representations of health-related information. Health Psychology. 2012; 31(3): 286-96. 18. Feldman-Stewart D, Kocovski N, McConnell BA, Brundage MD, Mackillop WJ. Perception of quantitative information for treatment decisions. Medical Decision Making. 2000; 20(2):228–38. 19. Reyna VF. A theory of medical decision making and health: Fuzzy-trace theory. Medical Decision Making.2008; 28(6): 829–33. 20. Hawley ST, Zikmund-Fisher BJ, Ubel PA, Jankovic A, Lucas T, Fagerlin A. The impact of the format of graphical presentation on health-related knowledge and treatment choices. Patient Education and Counseling. 2008; 73(3): 448-55. 21. Lipkus IM. Numeric, verbal, and visual formats of conveying health risks: Suggested best practices and future recommendations. Medical Decision Making. 2007; 27(5): 696–713. 22. Lipkus IM, Hollands JG. The visual communication of risk. Journal of the National Cancer Institute Monographs 1999; (25):149–63. 23. Zikmund-Fisher BJ, Fagerlin A, Ubel PA. Improving understanding of adjuvant therapy options by using simpler risk graphics. Cancer. 2008; 113(12): 3382-90. 24. Zikmund-Fisher BJ, Fagerlin A, Ubel PA. A demonstration of ''less can be more'' in risk graphics. Medical Decision Making. 2010; 30(6): 661-71. 25. Ancker JS, Weber EU, Kukafka R. Effect of arrangement of stick figures on estimates of proportion in risk graphics. Medical Decision Making. 2011; 31(1):143-50. 26. Stone ER, Yates JF, Parker AM. Effects of numerical and graphical displays on professed risk-taking behavior. Journal of Experimental Psychology: Applied. 1997; 3(4): 243-56 27. Stone ER, Sieck WR, Bull BE, Yates JF, Parks SC, Rush CJ. Foreground:background salience: Explaining the effects of graphical displays on risk avoidance. Organizational Behavior & Human Decision Processes. 2003; 90(1):19-36 28. Kurz-Milcke E, Gigerenzer G, Martignon L. Transparency in risk communication: Graphical and analog tools. In W. T. Tucker, S. Ferson, A. Finkel, T. F. Long, D. Slavin & P. Wright (Eds.), Strategies for risk communication: Evolution, evidence, experience 1128. Annals of the New York Academy of Sciences 2008 (pp. 18–28). New York: Blackwell. 7. Tailoring Estimates

Tailored health communication refers to providing information to a person based on
characteristics that are unique to that person. It is assumed that tailored messages are perceived
as more relevant to an individual and are therefore better processed and understood. Tailoring
information using an individual’s specific risk factors might likewise increase people’s
involvement with the information and lead to a better understanding.
To date, the effect of tailoring health risk information on improving health decision-making
appears mixed. Limitations in research quality and heterogeneity in outcome measures make
drawing firm conclusions about effective strategies difficult. A meta-analytic review showed that
tailored print messages about health have been effective in stimulating health behavior change,
but the size of effect is small and depends on the variable that is used for tailoring [Noar et al.
(1)]. The effect of tailoring was modified by the type, visual layout and length of the printed
material, type of behavior (more effective for preventive behaviours) and by demographic factors
[Noar et al. (1); Manne et al. (2)]. Tailored print messages have been shown to increase uptake of
mammography screening [Manne et al. (2)] and pap testing [Noar et al. (1); Manne et al. (2)]. A
review by Albada et al. showed that information tailored to an individual’s risk factors increased
realistic risk perception and resulted in better knowledge compared to generic information
[Albada et al. (3)].
Results are mixed, however, with respect to the effect of tailored health messages on
behaviour—for example, on cancer screening. Tailoring by behavioural constructs seems to be
effective, while there was limited evidence of the effectiveness of information tailored by risk
factors only, in particular for cancer screening. Bodurtha et al. also found that a ‘brief
intervention’ regarding mammography adherence did not change behavior [Bodurtha et al. (4)].
No significant differences existed in mammography intentions, actual uptake, clinical breast
examination, or self-examination between intervention and control study arms. However, among
those who were most worried, mammography rates in the intervention group were higher. Thus
individual characteristics, such as worry about breast cancer and educational status, may modify
the effects of tailored health messages.
Because most studies on tailoring health risk information were done for cancer screening, not
much is known about tailoring risk information for other decisions. Since studies did show an
effect of tailoring risk information on risk perception and knowledge, it seems likely that this
will also apply to other decisions. However, more insight is needed into why personalized risk
messages might be better understood and if personalized risk messages are relevant for all kinds
of decisions.
References
1.
Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behaviour change interventions. Psychological Bulletin 2007; 133: 673-93 2. Manne S, Coups E, Markowitz A, Meropol N, Haller D, Jacobsen P, et al. A randomized trial of generic versus tailored interventions to increase colorectal cancer screening among intermediate risk siblings. Annals of Behavioral Medicine. 2009;37:207–17. 3. Albada A, Ausems MGEM, Bensing JM and Van Dulmen S, Tailored information about cancer risk and screening: A systematic review. Patient Education and Counseling 2009; 77; 155-171 Bodurtha J, Quillin J, Tracy K, Borzelleca J, McClish D, Wilson D, et al. Mammography screening after risk-tailored messages: The Women Improving Screening through Education and Risk Assessment (WISER) randomized, controlled trial. Journal of Women's Health. 2009;18:41-7. 8. Formats for Understanding Outcomes Over Time

Choices of how to display long term outcomes to improve understanding of risk are challenged
by the difficulty in obtaining accurate relevant long-term outcome estimates of benefit and risk
[Lu et al. (1)]. Randomized controlled trials and systematic reviews usually represent a few years
of follow up at most, yet to make an informed decision patients and physicians are often
interested in longer term outcomes. Observational studies can provide longer-term data but are
prone to selection bias and confounding. An additional bias is the tendency for trials to aggregate
short and long term mortality which leads to inaccurate estimates if hazard ratios are not constant
over time [National Institute for Health and Clinical Excellence (2)]. These methodological
problems are beginning to be addressed by newer risk modeling approaches [Goldhaber-Fiebert
et al. (3); Stout et al. (4); Tunis et al. (5); Levin et al. (6)].

When data are available, formats used to improve patient understanding of outcomes over time
include: (a) the chance of a specific outcome at a single point in the future; (b) chance of an
outcome at multiple points in the future; (c) mortality or survival graphs showing risks over time;
(d) cumulative future or lifetime chance of an outcome; and (e) rate of occurrence of an outcome
that is likely constant over time.
Showing the chance of a specific outcome at a single point in the future has the advantage of
simplicity of presentation and calculation from available randomized trials or cohort studies.
Examples of this approach are the 10 year risk of cardiovascular disease used in estimates of risk
and benefit of cholesterol medications [Montori (7)] and the risk in 3-5 years of precancerous
changes on pap smear or genital warts related to HPV vaccine [Bennett et al. (8)]. This method
has also been used with multiple points in the future. Examples include presenting the risk of
having to have repeat by-pass surgery at 5 years and 10 years after the initial procedure
[Healthwise (9)], and expected deaths after lung transplantation for cystic fibroses shown at 1
month, 1 year, 3 years, 5 years, and 10 years [Vandemheen et al. (10)].

Survival and mortality graphs are commonly used in presenting research studies and have been
used to relay information to patients. However, patients’ interpretation of these graphs may be
susceptible to various biases. When people on the internet were shown survival graphs for a
hypothetical disease and treatment, they based their perceptions of treatment effectiveness on
visual differences in these graphs [Zikmund-Fisher et al. (11)]. When a longer duration of data
was shown, people perceived larger differences in risk even when the magnitude of risk was
identical. Mortality graphs may be more temporally consistent [Zikmund-Fisher et al. (12)], but
less well understood by patients [Armstrong et al. (13)]. Given these findings and current
limitations in evidence, a balanced approach using both survival and mortality may be prudent
until more information is available [Redelmeier et al. (14)]. A study presenting treatment options
for esophageal cancer showed most patients understood graphical representations of even
complex multidimensional patient-reported outcomes [McNair et al. (15)].
Another common format for understanding outcomes over time is estimating the cumulative chance of an event during a whole lifetime, although this can be difficult for people to understand [Rollison et al. (16)]. This is a commonly used method in describing cancer risk as in genetic counseling for the BRCA gene mutations [National Cancer Institute (17)]. More commonly, people are shown the cumulative chance of an event over a certain period of time into the future (e.g., 10 years); for example osteoporosis treatment [Cochrane Collaboration (18)] and hormone replacement therapy in menopause [Australian Government (19)]. Cumulative risk over time is also used in decision aids without an explicit endpoint when describing probabilities of outcomes after a specific event or intervention. Examples are comparing outcomes of Achilles tendon rupture with and without surgery [Healthwise (20)] and of cardiac resynchronization therapy in heart failure [Healthwise (21)]. Rates are also used in conditions likely to have a relatively constant risk over time. An example is birth control and the annual risk of pregnancy with a specific method [Mayo Clinic (22)]. Although decision aids providing quantitative risk information have been shown to increase accuracy of risk perceptions [O’Connor et al. (23)] and to promote knowledge and agreement between values and choices [O’Connor et al. (23)], there are no trials examining different formats for representing the risk of outcomes over time. Although common in the medical literature with measurements of statistical precision, methods for displaying uncertainty of outcomes over time to patient have not been studied. Descriptions of the nature or utility of different outcomes are also typically lacking, may be important to understanding, and require further investigation. References 1. Lu CY, Karon J, Sorich MJ. The importance of high-quality evidence of the long-term impact of nonfatal events used in randomized controlled trials: a case study of Parasugrel. Clinical Pharmacology & Therapeutics. 2011; 90(1): 27-29. 2. National Institute for Health and Clinical Excellence. Prasugrel for the treatment of acute coronary syndromes with percutaneous coronary intervention. NICE technology appraisal guidance 182. London: National Institute for Health and Clinical Excellence, 2009. http://www.guidance.nice.org.uk/TA182. 3. Goldhaber-Fiebert JD, Stout NK, Goldie SJ. Empirically evaluating decision-analytic models. Value Stout NK, Knudsen AB, Kong CYJ, McMahon PM, Gazelle GS. Calibration methods used in cancer simulation models and suggested reporting guidelines. Pharmacoeconomics. 2009; 27: 533-45. 5. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. The Journal of the American Medical Association. 2003; 290(12):1624-32. 6. Levin L, Goeree R, Levine M, Krahn M, Easty T, Brown A, et al. Coverage with evidence development: The Ontario experience. International Journal of Technology Assessment in Health Care 2001; 27(2): 159-168. 7. Montori V. Statin Choice. Mayo Knowledge and Encounter Research Unit, Mayo Clinic College of Medicine. 2011 Mayo Foundation for Education and Research. http://mayoresearch.mayo.edu/mayo/research/ker_unit/upload/StatinDecAid_AVG_Mayo.pdf. 8. Bennett C, Drake E, Hopkins L, O’Connor A. What can you do to prevent HPV and cervical cancer?: A decision aid for parents/guardians of girls in grade 8 in Ontario. Canadian Institute of Health. 2009. http://decisionaid.ohri.ca/docs/das/HPV_vaccine.pdf 9. Healthwise. Reviewer Thompson EG. Heart disease: Should I have bypass surgery? 2010. http://www.healthwise.net/cochranedecisionaid/Content/StdDocument.aspx?DOCHWID=av2037 10. Vandemheen K, Aaron S, Tullis E, Poirier C. When your lung function is getting worse: Should you be referred for a lung transplant? A decision aid for adults with cystic fibrosis. 2009. http://decisionaid.ohri.ca/docs/das/CF_group_a.pdf Zikmund-Fisher BJ, Fagerlin A, Ubel PA. What’s time got to do with it? Inattention to duration in interpretation of survival graphs. Risk Analysis. 2005; 25(3):589-95. 12. Zikmund-Fisher BJ, Fagerlin A, Ubel PA. Mortality versus survival graphs: Improving temporal consistency in perception of treatment effectiveness. Patient Education and Counselling. 2007; 66:100-107. 13. Armstrong K, Schwartz JS, Fitzergald G, Putt M, Ubel PA. Effect of framing as gain versus loss on understanding and hypothetical treatment choices: Survival and mortality curves. Medical Decision Making. 2002; 22:76-83. 14. Redelmeier DA, Rozin P, Kahneman D. Understanding patients’ decisions: Cognitive and emotional perspectives. The Journal of the American Medical Association. 1993; 270(1):72–6. 15. McNair AGK, Brookes ST, Davis CR, Argyropoulos M, Blazeby JM. Communicating the results of randomized clinical trials: Do patients understand multidimensional patient-reported outcomes? Journal of Clinical Oncology. 2010; 28:738-43. Rolison JJ, Hanoch Y, Miron-Shatz T. What do men understand about lifetime risk following genetic
testing? The effect of context and numeracy. Health Psychology. 2011; Advance online publication. doi: 10.1037/a0026562 17. National Cancer Institute. BRCA1 and BRCA2: Cancer Risk and Genetic Testing. 2009. http://www.cancer.gov/cancertopics/factsheet/Risk/BRCA 18. Cochrane Musculoskeletal Group. Updated June 2011. http://musculoskeletal.cochrane.org/decision-aids Making Decisions: Should I use hormone replacement therapy? (HRT). Australian Government, National Health and Medical Research Counsel. 2005, March. http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/wh37.pdf 20. Healthwise. Achilles tendon rupture: Should I have surgery? 2011. http://www.healthwise.net/cochranedecisionaid/Content/StdDocument.aspx?DOCHWID=ug2998 21. Healthwise. Reviewer Thompson EG. Heart Failure: Should I get a Pacemaker (Cardiac Resychronization Therapy)? 2011. http://www.healthwise.net/cochranedecisionaid/Content/StdDocument.aspx?DOCHWID=uf9843 22. Mayo Clinic Staff. Mayo Clinic. Birth control. http://www.mayoclinic.com/health/birth-control/MY01182 O’Connor AM. Decision aids for people facing health treatment or screening decisions. Cochrane Database of Systematic Reviews. 2009; 3. CD001431.

9. Narrative Methods for Conveying the Chance of an Event

Narratives and statistical information have been shown to affect perceived vaccination risk and
intentions [Betsch et al. (1)]. Narratives decreased the perceived chance of adverse events but
increased the perceived severity of adverse events. Narratives also influenced vaccination
intention even after controlling for the perception of vaccine riskiness. In the same study, the
nature of the presented information (emotionality, richness) was also varied to assess the impact
on risk perception and showed that the highly emotional narratives had a greater impact on
perceived risk although the richness of the narratives did not.
Other studies have shown that patient testimonials influence treatment choices. In one study,
participants receiving a disproportionate number of negative testimonials for surgery were less
likely to choose surgery compared to participants receiving equally positive and negative
examples for surgery [Ubel et al. (2)]. In addition, participants who received an equal number of
testimonials for each treatment option, or disproportionate number of testimonials, or a control
condition of no testimonials showed that those given no testimonials were most likely to choose
bypass surgery (58%), when compared to those receiving the proportionate number of
testimonials (37%) and those receiving the disproportionate number of testimonials (34%). In
this case the testimonials significantly reduced the choice of the presumably most effective,
invasive and risky intervention.
Another study tested whether the use of a quiz or pictograph lessened an individual’s reliance on anecdotal evidence for angina treatment (bypass surgery or balloon angioplasty) [Fagerlin et al. (3)]. It found that when statistical information was reinforced with pictographs and quizzes, anecdotes had no significant effect on treatment decisions. The same authors also found pictographs were the active ingredient which lessened the effect of anecdotes. This finding would argue for avoiding narratives without statistical information. Another study tested the effects of viewing one of three versions of a physician-patient encounter video or usual care (no video) [Mazor et al. (4)]. The videos differed in the type of evidence used (patient anecdotes, statistical evidence, or both). The results of the study suggested that all three approaches had a positive effect on knowledge, but had no effect on behaviour. There was some evidence that anecdotes may have had a greater impact than statistical information on beliefs and on knowledge. Use of a video decision support tool compared to verbal narratives played a role in encouraging less aggressive advance care planning choices amongst elderly demented patients families and made these decisions less likely to change over a six week period [Volandes et al. (5)]. In summary, using narratives to present benefit and risk information may increase perceptions of risk severity, decrease the ability to accurately recall risk probabilities, and influence treatment choice. The relative number of narratives used also influences decision making. Narratives should be used with caution when attempting to present unbiased information for informed decision making and to improve the patient-centeredness of decisions. Exceptions might be decisions where the information presented is meant to be persuasive and promote behavior change instead of informed decision making. If narratives are used to present benefit and risk information, they should be accompanied by statistical information such a pictographs. Graphical representations of risk may reduce the effect of narratives. However, it seems likely that the information included in narratives is sufficient to bias the ways individuals either search for and/or process information, limiting their usefulness in interventions designed to facilitate good decision making. We suggest that those designing interventions to facilitate informed decision-making avoid the use of patient testimonials until there is evidence to explain what type of narrative encourages bias in information processing and decision making and which mechanisms are mediating the effect. Betsch C, et al., The Influence of Narrative v. Statistical Information on Perceiving Vaccination Risks. Ubel, PA, C Jepson, and J Baron, The inclusion of patient testimonials in decision aids: effects on treatment choices. Medical Decision Making. 2001; 21: p. 60-8. 3. Fagerlin A, C Wang, and PA Ubel, Reducing the influence of anecdotal reasoning on people's health care decisions: is a picture worth a thousand statistics? Medical Decision Making. 2005; 25: p. 398-405. 4. Mazor KM, et al., Patient education about anticoagulant medication: is narrative evidence or statistical evidence more effective? Patient Educ Couns 2007; 69: p. 145-57. 5. Volandes AE, et al, Video decision support tool for advanced care planning in dementia: randomized controlled trial. BMJ 2009;338:b1964 doi:10.1136/bmj.b2159 10. Important Skills for Understanding Numerical Estimates
Numeracy is the ability to understand and apply mathematical concepts. It can affect the use and
interpretation of numerical estimates considerably. Higher numeracy can facilitate computations,
the interpretation of numbers, information seeking, depth of processing, and trust in numerical
formats, leading to improved risk comparisons, risk estimates, and value elicitations [Lipkus &
Peters, 2009 (1); Reyna, Nelson, Han, & Dieckmann, 2009 (2)]. On the other hand, lower
numeracy is associated with overestimation of risk probabilities (see section 1) [Weinstein et al.,
2004 (3); Woloshin et al., 1999 (4)], higher susceptibility to factors other than numerical data
(e.g. framing, mood states, labels used to interpret quantitative results and feedback from others)
[Peters et al., 2006 (5); Peters et al., 2009 (6)], and higher denominator neglect [Garcia-Retamero
& Galesic, 2009 (7); Reyna & Brainerd, 2008 (8)]. Higher numeracy is associated with higher
education, younger age, being Caucasian, and differs across countries [Reyna et al., 2009 (2);
Galesic & Garcia-Retamero, 2010 (9); Lipkus & Peters, 2009 (1)].

When designing a decision aid, measuring objective numeracy [Lipkus, Samsa, Rimer, 2001
(10); Schwartz, Woloshin, Black, & Welch, 1997 (11)], subjective numeracy: Fagerlin et al.,
2007 (12)], graph literacy (Galesic & Garcia-Retamero, 2011 (13)], and possibly other aspects of
the health literacy [Baker, 2006 (14); Schwartz et al., 2005 (15)] of the prospective users can
help in designing presentation formats that are suitable for their combination of skills. For
example, visual displays can improve understanding of those with lower numeracy who also
possess higher graphical literacy (see section 6) [Lipkus 2007 (16); Zikmund-Fisher et al., 2007
(17); Gaissmaier et al., in press (18)]. With few exceptions [Zikmund-Fisher et al., 2008 (19);
Lipkus et al. 2010 (20)], tests of how numeracy and graph literacy influence use and
interpretation of numerical data in decision aids is lacking, and hence sorely needed.

References
1.
Lipkus, I.M., & Peters, E. (2009). Understanding the role of numeracy in health: proposed theoretical framework and practical insights. Health Education & Behavior, 36, 1065-1081. Reyna, V.F., Nelson, W.L., Han, P.K., & Dieckmann, N.F. (2009). How numeracy influences risk comprehension and medical decision making. Psychological Bulletin, 135, 943-973. Weinstein, N.D., Atwood, K., Puleo, E., Fletcher, R., Colditz, G., Emmons, K. M. (2004). Colon cancer: risk perceptions and risk communication. Journal of Health Communication. 9, 53-65. Woloshin, S., Schwartz, L. M., Black, W. C., & Welch, H. G. (1999). Women’s perceptions of breast cancer risk: How you ask matters. Medical Decision Making, 19, 221–229. Peters, E., Västfjäll, D., Slovic, P., Mertz, C., Mazzocco, K., & Dickert, S. (2006). Numeracy and decision making. Psychological Science, 17, 407-413. Peters, E., Dieckmann, N. F., Västfjäll, D., Mertz, C. K., Slovic, P., & Hibbard, J. H. (2009). Bringing meaning to numbers: The impact of evaluative categories on decisions. Journal of Experimental Psychology:Applied, 15, 213-227. Garcia-Retamero, R., & Galesic, M. (2009). Communicating treatment risk reduction to people with low numeracy skills: A cross-cultural comparison. American Journal of Public Health, 99, 2196-2202. Reyna, V. F., & Brainerd, C. J. (2008). Numeracy, ratio bias, and denominator neglect in judgments of risk and probability. Learning and Individual Differences, 18, 89–107. Galesic, M. & Garcia-Retamero, R. (2010). Statistical numeracy for health: A cross-cultural comparison with probabilistic national samples. Archives of Internal Medicine, 170, 462-468. Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making, 21, 37-44. Schwartz, L.M., Woloshin. S., Black. W.C., Welch. G.H. (1997). The role of numeracy in understanding the benefit of screening mammography. Annals of Internal Medicine, 127, 966–971. Fagerlin, A., Zikmund-Fisher, B. J., Ubel, P. A., Jankovic, A., Derry, H. A., & Smith, D. M. (2007). Measuring numeracy without a math test: development of the subjective numeracy scale. Medical Decision Making, 27, 672–680. Galesic, M. & Garcia-Retamero, R. (2011). Graph literacy: a cross-cultural comparison. Medical Decision Making, 31, 444-457. Baker, W. D. (2006). The meaning and the measure of health literacy. Journal of General Internal Medicine, 21, 878-883. Schwartz, L. M., Woloshin, S., & Welch, G. (2005). Can patients interpret health information? An assessment of the medical data interpretation test. Medical Decision Making, 25, 290–300. Lipkus, I. M. (2007). Numeric, verbal, and visual formats of conveying health risks: Suggested best practices and future recommendations. Medical Decision Making, 27, 696-713. Zikmund-Fisher, B. J., Smith, D. M., Ubel, P. A., & Fagerlin, A. (2007). Validation of the Subjective Numeracy Scale (SNS): Effects of low numeracy on comprehension of risk communications and utility elicitations. Medical Decision Making, 27, 663–671. Gaissmaier, W., Wegwarth, O., Skopec, D., Müller, A.-S., Broschinski, S., & Politi, M. C. (in press). Numbers can be worth a thousand pictures: individual differences in understanding graphical and numerical representations of health-related information. Health Psychology. Advance online publication. doi: 10.1037/a0024850 Zikmund-Fisher, B. J., Fagerlin, A., Ubel, P. A. (2008). Improving understanding of adjuvant therapy options by using simpler risk graphics. Cancer, 113, 3382-3390. Lipkus, I. M., Peters, E., Kimmick, G., Liotcheva, V., & Marcom, P. (2010). Breast cancer patients' treatment expectations after exposure to the decision aid program adjuvant online: the influence of numeracy. Medical Decision Making, 30, 464-473.
SECTION 6:
EMERGING ISSUES / EMERGING RESEARCH AREAS
Interactive, Web-based Formats
The increasing prevalence of computers, tablets, and mobile devices creates new opportunities
for interactive, web-based formats for communicating probability information. The literature in
this area is sparse, and we are aware of no published studies that have examined use of such tools
in actual patient decision aids. Several experimental studies suggest, however, that web-based
formats offer both opportunities and pitfalls. For example, in one study, participants presented
with a treatment scenario were better calibrated in their perceptions of medication side effects
when they created a bar graph of the risk instead of just viewing one [Natter at al (1)]. Another
study found that a web-based, game-like, interactive risk graphic in which participants clicked in
a matrix until they uncovered a risk event had the effect of reducing disparities in risk
perceptions between high and low numeracy participants [Ancker et al. (2)]. Such exercises
could be seen as methods of increasing patients’ active processing of risk information, which
may lead to improved risk understanding. Indeed, the game-like interactive task elicited stronger
emotional responses [c et al. (3)].
However, there is also considerable reason for concern about interactive risk graphics. In 2002,
Tversky, Morrison, and Betrancourt reviewed the literature on animated graphics of all types and
noted that “the research on the efficacy of animated over static graphics is not encouraging”
[Tversky et al. (4), p. 247]. More recently, research participants who used an interactive
pictograph applet to visually graph provided risk numbers had significantly worse knowledge
and made poorer decisions than participants who viewed static graphs [Zikmund-Fisher et al. (5)]. Even without interactivity, animated graphics can use motion cues to reinforce gist messages. However, the evidence here is also mixed. While one study found a dynamic scattered icon display increased recipients’ subjective uncertainty about a risk [Han et al., (6)]. Another study tested various types of animation in both grouped and scattered icon displays and found that they failed to improve participants’ ability to identify a dominant treatment option and sometimes significantly impeded performance [(unpublished but presented here: Zikmund-Fisher et al. (7)]. In short, interactive web-based risk communication formats allow educators to use additional cue in risk communications. However, evidence is lacking to determine whether these techniques allowed by new technologies provide a net positive experience. Preliminary evidence suggests that unless the motion cues reinforce the most critical gist message (e.g., the accumulation of risk over time), there remains significant risk that interactive or animated formats may degrade knowledge versus evidence-based static formats. Natter HM, Berry DC. Effects of active information processing on the understanding of risk information. Ancker JS, Chan C, Kukafka R. Interactive graphics for expressing health risks: development and qualitative evaluation. J Health Commun. 2009;14(5):461-75. 3. Ancker JS, Weber EU, Kukafka R. Effects of game-like interactive graphics on risk perceptions and decisions. Med Decis Making. 2011;31(1):130-42. 4. Tversky B, Morrison JB, Betrancourt M. 2002. Animation: can it facilitate? International Journal of Human-Computer Studies, 57, pp. 247-262. 5. Zikmund-Fisher BJ, Dickson M, Witteman HO. Cool but counterproductive: interactive web-based risk communications can backfire. Journal of Medical Internet Research 2011;13(3):e60. doi:10.2196/jmir.1665 6. Han PKJ, Klein WMP, Killam B, Lehman T, Massett H, Freedman AN. Representing randomness in the communication of individualized cancer risk estimates: Effects on cancer risk perceptions, worry, and subjective uncertainty about risk. Patient Education and Counseling. 2012 Jan;86(1):106–13. 7. Zikmund-Fisher BJ, Dickson M, Swanson M, Fuhrel-Forbis A, Exe N, Kahn V, Witteman H. Does adding motion to icon array risk graphics help? Oral presentation at the International Shared Decision Making Conference, Maastricht, Netherlands, June 20, 2011. APPENDIX:
ORIGINAL CHAPTER C

Original Authors
Alex Barratt
University of Sydney, School of Public Health University of Wales, College of Medicine, Swansea University of Sydney, School of Public Health University of Sydney, School of Public Health Dartmouth Medical School, Dartmouth Collage, New Behavioural Sciences, University of Leeds Department of Social Medicine, Vrije Universiteit, Amsterdam Research Psychologist, Geneva University Hospitals
Original Rationale/Theory
A key objective of patient decision aids is to provide information to help patients understand the
possible benefits and harms of their choice, and the chances that these will occur. Since no
intervention is 100% effective in all patients without harms (including side-effects), probabilities
must be presented in patient decision aids. However, presenting risk information (probabilities)
is problematic because most individuals -- including patients and professionals -- have difficulty
in processing and accurately evaluating probabilities and statistics. The evidence suggests that
individuals would rather use a heuristic such as someone else's evaluation of the risks than attend
to the figures in order to make a decision. Some strategies for effectively communicating
probabilities in health have been proposed (see, for example, Schwartz, 1999), but few have been
tested empirically in patient decision aids. Therefore, recommendations in this document are
largely made on theoretical grounds, borrowing heavily from work in clinical epidemiology and
evidence based health care, psychology (prospect theory, Tversky & Kahneman, 1974;1981),
risk communication and risk perception research (Loewenstein et al., 2001; Slovic et al., 2002),
and decision theory (theory of expected utility, Neumann & Morganstern).

Presenting Numbers

Although many patients prefer to read words rather than numbers, numerical probabilities
improve the accuracy of understanding. Event rates (natural frequencies) are the recommended
way to present these probabilities. Event rates for all relevant options and for each relevant
outcome should be given, and appropriate time frames and denominators should be provided. For
example, a patient decision aid on stroke prevention in atrial fibrillation should give the number
out of 100 men who will have a stroke over 10 years if they take warfarin, the number out of 100 men who will have a stroke over 10 years if they take aspirin, and the number out of 100 men who will have a stroke over 10 years if they take no treatment. For situations in which risks are small, such as screening and other preventive interventions, denominators of 1000 or 10,000 may be needed. Events rates are intuitively interpretable because they are natural frequencies with clearly stated
reference classes. Some patient decision aids use other presentation formats including relative
risk reduction, absolute risk reduction, number needed to screen, or number needed to treat.
These may help when patients have to compare many options, because they allow summarization
of data but they are less likely to be well understood. Furthermore, none of these formats
(relative risk reduction, absolute risk reduction, number needed to screen, or number needed to
treat) make the baseline risk of disease as explicit as simply presenting event rates for all
intervention options being compared.
Constant denominators (e.g. 1 in 100, 5 in 100) rather than constant numerators (e.g., 1 in 100, 1
in 20) are more readily understood (Woloshin et al., 2000). For information to be meaningful, it
is important to present the timeframe over which events occur, and to use a timeframe that
patients find useful for planning health management -- for example, “Imagine 1000 patients.
Over the next 10 years, 150 of them will die of …”. Although lifetime risk is often used, 10 year
time frames are often more appropriate (Woloshin et al., 2002).
Visual Aids

Presenting event rates with visual aids such as 100 faces diagrams, bar charts, human figure
representations, or flow diagrams may aid accurate understanding of probabilities. By using
more than one presentation format, patients are able to choose the format that works best for
them. As well, analogies may be especially useful for presenting small risks – e.g. one person in
a football stadium crowd, etc (Edwards, 2003). Any visual aids to be used should be pilot tested
for understanding, and developers should take care to avoid using misleading images (such as
graphs with misleading scales) or using different scales within the same patient decision aid.
There is evidence that the formats which are perceived most accurately and easily by patients are
vertical bars, horizontal bars and systematic ovals. Pie charts and random ovals lead to slower
and less accurate estimates (Feldman-Stewart et al., 2000).
Probabilities For Tests And Screening Decisions
The mortality benefit from screening should be presented as the probability of death with and
without screening; e.g. the probability of dying of breast cancer in 1000 women who regularly
participate in screening and in 1000 women who decline screening. It is very important that the
survival times are NOT used as these are likely to be affected by lead time bias (Barratt et al.,
1999; Welch et al., 2000).
Patient decision aids for screening should also present the probability of having the target condition detected with and without screening, because many cancer screening programs lead to over-detection of disease. Disease aids should therefore alert readers to the possibility of screening leading to detection and treatment of disease that might never have caused symptoms had it not been for screening. For example, the chance of having breast cancer or prostate cancer diagnosed is substantially higher in screened compared to unscreened populations because some or many of these cancers would never have become symptomatic (and therefore diagnosed) in the absence of screening. Patient decision aids about tests or screening programs also need to present information about the
chances of receiving a false positive (false alarm) or false negative result. Although these data
have traditionally been presented as specificity and sensitivity, these are not readily understood.
Such conditional probabilities should be avoided and natural frequencies (event rates) used
instead. For example, “over 10 mammography screening rounds, 160 out of 1000 women
participating in screening will experience a false positive result” is more readily interpreted than
the specificity (the proportion of patients who test positive among those who do not have
disease) of mammography screening.
Screening may lead to a cascade of events (including follow-up tests and treatments), and the
probability of each of these events occurring should also be presented.

Tailoring Probabilities
Whenever possible, individualised risks should be used. Although there is little evidence
specifically examining the degree to which individualised risk information facilitates patients’
understanding and decisions, it is likely that personally relevant risks will be evaluated more
accurately in accord with a patients' values than less relevant risk information. For example,
individualized risk estimates (using tables, computerized algorithms, or risk estimates for groups
of patients) depending on important risk factors such as age, gender, family history, smoking
status might be used. As a minimum, it should be clear to the user of the patient decision aid
whether the probabilities apply to them based on their gender, age, medical history, or other risk
factors.

Framing Probabilities
The way information is framed can affect preferences and decision making (Edwards et al.,
2001; Tversky et al., 1981). Thus, patient decision aid developers should be aware of potential
framing effects. Framing effects are minimized if visual aids such as 100--faces diagrams are
used, because they show the number of patients experiencing the outcome and the number of
patients not experiencing the outcome for each option being considered all at once. Simply
giving the percentage (x %) of patients who experience an event (e.g., death) does not achieve
this as clearly, because the reader has to do mental arithmetic (100-x) to calculate the percentage
who do not experience it (e.g., survive). Event rates presenting both positive and negative frames
can be used, but may lead to information overload. An alternative is for writers to acknowledge
explicitly the frame used and encourage patients to reformat the information for themselves.
Formats such as relative risk reduction, absolute risk reduction, and numbers need to treat can be
misleading, because they do not make explicit the baseline risk of the target condition. For
example, a 50% reduction in risk sounds very impressive, but it might refer to a treatment that
reduces the risk of death from 40 out of 100 to 20 out of 100 OR to a treatment that reduces the
risk of death from 4 out 10,000 to 2 out of 10,000. Relative risk reduction generally is more
impressive -- and potentially misleading -- than absolute risk reduction, particularly for rare events. Probabilities in Context
Disease-specific probabilities (or the benefits of various disease-specific interventions) are hard
to understand in isolation. Therefore, patient decision aids need to help patients put disease- (or
intervention-) specific information into context. One way is to provide estimates of the 10-year
chance of developing or dying from various diseases (or dying from any causes) for men/women,
smokers/non-smokers at various ages. Other anchors, such as commonly and not so commonly
occurring events, have been used.

Conveying Uncertainty
It’s very important to acknowledge uncertainty in probability estimates. Often the uncertainty is
large, especially if evidence is scarce or events are rare. It’s probably wise to do simple things
such as rounding off numbers (to avoid false illusions of precision), using phrases like "our best
guess is.”, give ranges, or provide 95% confidence intervals.
Even with the best evidence from large studies (thus with high accuracy and precision), the issue
of stochastic uncertainty remains (Edwards, Elwyn, Mulley, 2002). Essentially, we never quite
know who are the patients who are going to be affected, and who the treatment is going to be
most useful for. One way to deal with this uncertainty might be to say: "If 100 patients like you
are given no treatment for five years, 92 will live and eight will die. Whether you are one of the
92 or one of the eight, I do not know. Then, if 100 patients like you take a certain drug every day
for five years, 95 will live and five will die. Again, I do not know whether you are one of the 95
or one of the five." (Skolbekken, 1998)
Despite these limitations from uncertainty, practitioners generally feel that we can still try to
make decisions about what the best treatment plan is for an individual person, based on what
happens to these groups of patients in the studies. Hence the value, it is thought, of presenting the
information about benefits and harms to aid the decision making process. Both sources of
uncertainty should be acknowledged in comprehensive discussions of risks in patient decision
aids.

Evidence for Probabilities Used
To enhance transparency and allow patients and practitioners to see for themselves where the
probabilities come from, a technical appendix or something similar should be provided. This can
outline the data sources, the populations from which the probabilities were obtained, and any
calculations or modeling that was done to derive the probabilities in the patient decision aid.
Developers may want to include a decision analyst or other experienced modeler on their team to
help obtain useful probability estimates. In some instances, developers may use decision analysis
to structure the patient decision aid. In such cases, if the probabilities used in the decision
analysis are presented, they should be presented in accordance with these criteria.
Original Evidence

RCTs Involving Patients Facing Actual Choices
Of 29 individual patient decision aids evaluated in the 34 RCTs included in the Cochrane
Review, 19 were available for content review (O’Connor et al., 2003). Of these, 19, 17 (89%)
patient decision aids contained some sort of information about outcome probabilities. There were
some differences in the way this information was provided:
 11 of 19 (58%) patient decision aids provided numerical data with outcomes reported as “x out of 100” and/or percentages (with consistent denominator of 100).  5 of 19 (26%) patient decision aids provided numerical data with outcomes reported as “x out of y” (denominators were not necessarily consistent).  4 of 19 (21%) patient decision aids provided graphical display of the data using pie  3 of 19 (16%) patient decision aids provided graphical display of the data using 100  1 of 19 (5%) patient decision aids provided numerical data using a tabular format.
The Cochrane Review identified 7 randomized controlled trials that evaluated the effect of
patient decision aids on patients’ perceived probabilities of outcomes: 4 of these compared a
patient decision aid to usual care and 3 compared a simpler to a more detailed patient decision
aid (O’Connor et al., 2003). Perceived outcome probabilities were classified according to the
percentage of individuals whose judgments corresponded to the scientific evidence about the
chances of an outcome for similar patients.
All 7 studies (100%) showed a trend toward more realistic expectations in patients who
received a detailed patient decision aid (i.e., included descriptions of outcomes and
probabilities) compared to those who did not receive patient decision aids with this
information included. However, only 6 of the studies had the power to detect a statistically
significant difference (RR ranged from 1.3 to 2.3).

Original References
Barratt AL, Irwig L, Glasziou P, Cumming R, Raffle A, Hicks N, Gray JAM, Guyatt GH. Users’ guides
to the medical literature. XVII How to use guidelines and recommendations about screening. JAMA
1999; 281: 2029-2034.
Edwards A, Elwyn G, Covey J, Matthews E, Pill R. Presenting risk information- a review of the effects
of “Framing”and other manipulations on patient outcomes J Health Communication 2001 6 61-82.
Edwards, A., G. Elwyn, and A.G. Mulley, Explaining risks: turning numerical data into
meaningful pictures. British Medical Journal, 2002. 324: p. 827-830.
Edwards A: Communicating risks through analogies [letter]. British Medical Journal 2003, 327:749
Feldman-Stewart D, Kocovski N, McConnell BA, Brundage MD, Mackillop WJ. Perception
of quantitative information for treatment decisions. Medical Decision Making 2000;20:228-
238.
Gigerenzer G, Edwards A. Simple tools for understanding risks: from innumeracy to insight. BMJ 2003;327:741-4. Huff D. How to lie with statistics. WW Norton & Co. New York: Reissued 1993. Loewenstein G, Weber EU, Hsee CK, Wech N. Risk as feelings. Psychological Bulletin 2001;127:267-286. O'Connor, A.M., Stacey, D., Entwistle, V., Llewellyn-Thomas, H., Rovner, D., Holmes-Rovner, M., Tait, V., Tetroe, J., Fiset, V., Barry, M., Jones, J. Decision aids for people facing health treatment or screening decisions [Cochrane Review]. In: The Cochrane Library, 2003: Issue 2. Oxford: Update Software. Skolbekkan JA. Communicating the risk reduction achieved by cholesterol reducing drugs. BMJ 1998;316:1956-1958. Slovic P, Finucane M, Peters E, MacGregor DG. The affect heuristic. In T. Gilovich, D Griffin & D Kahneman (Eds). Heuristics and biases (pp. 397-420). New York: Cambridge University Press, 2002. Schwartz LM. Woloshin S. Welch HG. Risk communication in clinical practice: putting cancer in context. Journal of the National Cancer Institute Monographs 1999; (25):124-133. Tversky A, Kahneman D. Judgement under uncertainty: heuristics and biases. Science 1974; 185:1124-1131. Tversky A. Kahneman D. The framing of decisions and the psychology of choice. Science. 211(4481):453-8, 1981 Jan 30. Users Guides to the Medical Literature A manual for evidence based practice. Jaeschke R, Guyatt G, Barratt A et al. Therapy and understanding the results. Measures of association (pp. 351-368). American Medical Association 2002. Von Neumann J, Morgenstern O. Theory of games and economic behaviour. Princeton: Princeton University Press, 1947. Welch HG, Schwartz LM, Woloshin S. Are increasing 5-year survival rates evidence of success against cancer? JAMA 2000;283:2975-2978. Woloshin S, Schwartz LM, Byram S, Fischhoff B, Welch HG. A new scale for assessing perceptions of chance: a validation study. Medical Decision Making 2000;20: 298-307. Woloshin S, Schwartz LM, Welch HG. Risk charts: putting cancer in context. Journal National Cancer Institute 2002;94: 799-804.

Source: http://ipdas.ohri.ca/IPDAS-Chapter-C.pdf