Not Different, But Not The Same

Dec 06 2012 Published by under [Medicine&Pharma]

A study compares two treatments for a disease. A pre-agreed outcome is measured, and the results are compared in the two groups. The p value for the statistical text is 0.75, so the treatments are equivalent.

Not necessarily.

Lactate and Frog Mortality

A lovely article in Advances in Physiology Education leads us through these concepts, especially when using a continuous variable to distinguish categories. Their example uses plasma lactate and mortality in overheated frogs which, as shown in the figure, shows a significant difference in median value between survivors and nonsurvivors, albeit with a great deal of overlap. How can we use these data to predict mortality in frogs?

The receiver-operator characteristic curve (ROC) is the answer. This technique was used to see how operators of radar receivers distinguished signals from background.

ROC

The curve is generated by plotting sensitivity vs 1-specificity. As shown in the lower graph, the area under the curve should be >50% (the value for the dotted line of no predictive value). The closer the value is to 100%, the better the test's discrimination value.

The piece goes on to demonstrate effects of sample size on analyses and a consideration of the power of a negative study. Power is the chance of failing to find a difference in samples when one exists; it must be considered before one concludes that two things are the same.

These concepts confuse a lot of people in science and medicine. As more laypeople directly access the biomedical literature, we must be prepared to discuss these issues with a broad audience. This article provides some concrete examples to help walk readers through the issues.

No responses yet

Leave a Reply