The road to misunderstanding your health is paved with numbers

We rely on professional advice when making decisions about prenatal testing or cancer screening or judgments about test results, such as an HIV test. But there is a need to be wary about what your doctor is telling you.

Research has revealed that the probabilities underpinning these tests are often misunderstood not only by lay people but doctors as well.

Gerd Gigerenzer has argued that the ability to interpret numbers correctly depends on both people’s level of numeracy and how the numbers are presented.

When presented as conditional probabilities - 11% - it is much more difficult to make sense of numbers than when they are presented in natural frequencies - 11 out of 100.

The test

German researchers tested the numeracy of doctors by providing them with the task to estimate the likelihood that a woman with a positive screening mammogram actually has breast cancer.

For one group of doctors the likelihood was formulated in conditional probabilities and for the other group, it was presented in natural frequencies.

The doctors were asked to imagine that they conducted a breast cancer screening program.

About the women participating in the program, the first group was told: “The probability that one of these women has breast cancer is 0.8%. If a woman has breast cancer, there is a 90% probability that she will have a positive mammogram. If a woman does not have breast cancer the probability that she will still have a positive mammogram is 7%. Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer?”

The same task in natural frequencies for the second group: “Eight out of every 1000 women have breast cancer. Of these 8 women with breast cancer, 7 will have a positive mammogram. Of the remaining 992 women who don’t have breast cancer, some 70 will still have a positive mammogram. Imagine a sample of women who have positive mammograms in screening. How many of these women actually have breast cancer?”

Do you think you can get it right?

Natural frequencies

If you think in natural frequencies, you have only to consider the 7 patients who have a positive test and the disease and the 70 patients with a positive test and no disease.

That means 7 + 70 would have a positive mammogram while only 7 of these with a positive mammogram have breast cancer.

7 divided by 77 is about 9%. So, 9% of the women with a positive mammogram have actually breast cancer.

Conditional probability

If you try to formulate the task in conditional probabilities you get a much more complicated equation.

You have to multiply the probability that a woman of the screening program has breast cancer with the probability that she will have a positive mammogram (.008 x .90).

This has to be divided by sum of the probability that a woman has a positive mammogram and has breast cancer (.008 x .90) and the probability that a woman has a positive mammogram but no breast cancer (.992 x .07).

The equation looks like this: .008 x .90 divided by (.008 x .90 + .992 x .07). The result is again approximately 9% but most people will not be able to solve this without a calculator.

In the test, the doctors in the group solving the first case - conditional probabilities - got it wrong. Most of them highly overestimated the probability and believed that the probability would be 90%.

In the second case - natural frequencies - most doctors got it right but there were still some who believed that the probability would be 70% or 80%.

The results

These results illustrate two major insights from risk research - innumeracy is not only a problem for lay people but for professionals as well. So both would gain from data presented in natural frequencies.

The other insight from decision-making research that when people have to decide under time pressure or with limited time available, they refer to their intuition or trust in information given by others rather than on rational calculation.

The research shows that such intuitive strategies are useful in many cases for making a decision when not enough certain knowledge is available or there’s no time to carefully balance all possible options.

This is the way people operate in many high risk occupations such as fire fighters, stock market traders or doctors working in Emergency.

But in the case of health tests, most of us would like to know how much uncertainty is actually involved in our doctor’s judgment.

A lesson for patients

You might be confident about your doctor but can you be sure that he always gets the numbers right?

It is worth asking about evidence to prevent you from jumping to conclusions with expensive consequences. Only when you ask you will find out how good you doctor really is.

Jens Zinn does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.