Health news websites — including a Harvard health blog — buzzed last week with news that yoga can help stroke survivors improve their balance and quality of life. But a closer look reveals problems with the study’s design and calls those conclusions into question.
In the study, e-published last week by the journal Stroke, researchers randomly assigned stroke survivors to one of two groups. One group participated in twice-weekly, hour-long yoga sessions, while the control (or comparison) group didn’t do yoga or any other kind of rehab. The scientists reported that after eight weeks of yoga, patients’ balance improved, they weren’t as afraid of falling, they felt more independent and had better quality of life.
Sounds great, right?
Hang on. Researchers found no significant difference in outcomes between the yoga group and the control group, according to the study. Instead, they based their positive conclusions on a method called within-group comparison: They looked just at the yoga group to see how much participants improved. The problem with this approach is the yoga group could have improved for any number of reasons. Research has showed that simply participating in a study can affect how ill people experience and describe their systems. That’s why the yoga group needed to be compared to the control group — and when the researchers did that, they came up short. An Oxford University statistician called within-group comparisons “conceptually wrong, statistically invalid, and consequently highly misleading.”
Second, the physical therapist who evaluated the study participants knew who did yoga and who didn’t. She even helped out during the yoga sessions. In well designed medical studies, researchers often don’t know the specific theory the study is testing, much less which group a patient was assigned to. Failing to “blind” researchers this way can lead to observer bias – without intending to, the physical therapist could have evaluated the yoga participants differently than the control group.
The final word? Yoga might help stroke survivors, but we can’t conclude that based on this study.