This week’s New England Journal of Medicine has a fascinating study that followed 115,000 health professionals for upwards of 30 years to see if regularly eating nuts – walnuts, peanuts and others – was associated with fewer deaths. The answer is “yes,” and you can read details from the source here. Today’s blog uses this study to teach how to read statistical charts for fun and health.
The lesson is not entirely geeky. Once you understand the basic law of large numbers — and the downside of trying to draw conclusions from small numbers — you’ll be armed to make sense out of lots of the statistics that bombard us every day.
Here’s my annotation of the basic layout of this forest plot. You’ll see that the three vertical dotted lines — one for men, one for women, and one “pooled” — are key to understanding it. They show the dividing line between good (on the left of the dotted line) and bad (on the right). The further the blue dot gets from the vertical line, the stronger the effect. If the blue dot is right on the dotted line, that would mean the ratio of deaths in the nut-eating group was the same as among the nut abstainers. Most of the time, though, you see the blue dot on the left, which means fewer deaths among the nut eaters.
By following the blue dots, you can see similar numbers divided by the type of death — on this excerpt: cancer, heart disease and breathing disorders — and sub-divided between men and women. Remember, these are ratios between deaths in the nut-eating group and deaths in the non-nutters. (In this particular chart, they lumped in anyone who ate at least two one-ounce portions of nuts per week among the nut eaters. Other data in the study showed that the death ratio inched over in favor of the nut eaters with more and more nut consumption up to seven times a week.)
Now here’s the next key thing to notice – the thin horizontal lines that cut through the blue dots. It’s called a “confidence interval,” something baffling and off-putting to us non-statisticians, but easy to grasp once you see it in a picture.
These horizontal lines are measures of how statistically strong the results are — the wider spread of the horizontal line, the weaker the finding, and vice versa: the narrower the spread, the stronger the chance that the blue dot is in the right place.
Look at the example I circled at the bottom of the forest plot chart: this is the ratio for respiratory disease deaths comparing tree nut eaters to non-nut consumers. The basic result is 0.90, a 10% reduction. But see how the horizontal line stretches all the way from 0.74 – 26% fewer deaths – to 1.09 – 9% MORE deaths. Which is the right number? We can’t tell. A statistician can only tell you that if the same test was run 100 times, it would fall in this range 95 times. The fact that one end of the interval falls on the other side of the dotted line of neutrality means that that result is not considered “statistically significant,” at least if you’re trying to guess whether nut eating is associated with fewer deaths from breathing issues.
Why do statisticians use the 95% “confidence interval”? It’s a more precise way of rendering the concept that to be statistically meaningful a “P” value needs to be 0.05 or less: i.e., 5% of the time, or one in twenty, this result would come about by random chance. The “confidence interval” gives you a lot more data than the simple “P” number.
Back to the law of large numbers. The reason the numbers at the top of the chart have narrower “confidence intervals,” shorter horizontal spreads, is that they are crunching very large numbers. This study reported on 115,000 adults followed for 30 years. When you lump all comers, that’s a lot of years. But when you start slicing out particular causes of death, and separating tree nuts from peanuts, you get fewer and fewer person-years and more and more chance that the results are statistically flukey.
“Forest plots” are commonly used by medical scientists to crunch together data from different studies to ask big questions like one in the news this week: Do cholesterol-lowering statins lengthen life and who should use them? Here’s a forest plot done by the Canadian group Therapeutic Letter on that issue:
Notice how the dots cluster near the “no effect” dividing line, with some really wide “confidence intervals.” But when all the studies are put together at the bottom, the statins win out over placebo (albeit not by much).
Back to the nuts: By the way, bear in mind this isn’t necessarily cause-and-effect. It could be that eating nuts will add years to your life. It could also be that people who eat a lot of nuts are “health nuts” who do all sorts of things to prolong their lives, so nut eating is just a marker of longevity and not a cause. Meantime, though, it can’t hurt to chomp down a few walnuts.