A study compares two treatments for a disease. A pre-agreed outcome is measured, and the results are compared in the two groups. The p value for the statistical text is 0.75, so the treatments are equivalent.
A lovely article in Advances in Physiology Education leads us through these concepts, especially when using a continuous variable to distinguish categories. Their example uses plasma lactate and mortality in overheated frogs which, as shown in the figure, shows a significant difference in median value between survivors and nonsurvivors, albeit with a great deal of overlap. How can we use these data to predict mortality in frogs?
The receiver-operator characteristic curve (ROC) is the answer. This technique was used to see how operators of radar receivers distinguished signals from background.
The curve is generated by plotting sensitivity vs 1-specificity. As shown in the lower graph, the area under the curve should be >50% (the value for the dotted line of no predictive value). The closer the value is to 100%, the better the test's discrimination value.
The piece goes on to demonstrate effects of sample size on analyses and a consideration of the power of a negative study. Power is the chance of failing to find a difference in samples when one exists; it must be considered before one concludes that two things are the same.
These concepts confuse a lot of people in science and medicine. As more laypeople directly access the biomedical literature, we must be prepared to discuss these issues with a broad audience. This article provides some concrete examples to help walk readers through the issues.