The New York Times has this article up on its web site:
Fourteen years ago, a leading drug maker published a study showing that the antidepressant Paxil was safe and effective for teenagers.
On Wednesday, a major medical journal posted a new analysis of the same data concluding that the opposite is true.
That study — featured prominently by the journal BMJ — is a clear break from scientific custom and reflects a new era in scientific publishing, some experts said, opening the way for journals to post multiple interpretations of the same experiment.
It comes at a time of self-examination across science — retractions are at an all-time high; recent cases of fraud have shaken fields as diverse as anesthesia and political science; and earlier this month researchers reported that less than half of a sample of psychology papers held up.
“This paper is alarming, but its existence is a good thing,” said Brian Nosek, a professor of psychology at the University of Virginia, who was not involved in either the original study or the reanalysis. “It signals that the community is waking up, checking its work and doing what science is supposed to do — self-correct.”
Of course, scientologists (who generally oppose all psychiatry and psychiatric medications) will have a field day. So will all the voices out there — from climate deniers to vaccine opponents — who will say, “A-ha! This proves it! Scientists don’t know what they are doing with Paxil so how can we trust them on global climate change and childhood vaccinations?”
These kinds of scientific about-faces are confusing for many members of the general public.
First: butter is good for you. Then it was bad for you. Now it’s not bad for you again except if you eat too much of it. To many non-scientists it seems as of scientists have no idea what they are doing and you just can’t trust any of it.
I always tell people who are put off by all this switching and changing that the first thing they can do to put themselves on the road to being a scientifically aware person is this: All of this is perfectly normal. Science makes mistakes. Other scientists correct those mistakes.
Think of when you were first learning how to do something relatively simple — say, drive a car. You were likely not perfect the first time you got behind the wheel. You drove too slow or too fast. You ran over a curb. You could not parallel park. You mowed over your mother’s prized rose bushes.
But you got better at it the more you did it. This is called trial and error and it’s the best way many people learn. Just do it yourself, make mistakes, and learn from those mistakes. Nobody said, “Well, you made too many mistakes. This is proof you are an unreliable driver and we cannot ever trust you to do any better.”
Broadly speaking, science is no different. Think of how simple driving a car is now for you. Yet you made mistakes in the beginning anyway. Now think about a mathematically and logistically complex scientific experiments can be, involving highly technical measurements and calculations taken over years with thousands of variables. Then that mountain of data has to be analyzed by (one hopes) very smart people with years of education and training. But they are still, after all is said and done, just humans.
Does anyone really expect that there will never be any errors in work like this? It’s naive to think so.
But then someone else comes along — other scientists — and they take a new look at old data (as with Paxil) or they design a new study which uses better, more modern methods than an older study.
This trial and error, the catching of old mistakes, is a sign that all is working as it should in the world of science. And some science is easier to quantify, which probably partially accounts for much of the high error rates in the psychology papers mentioned above.
The hard sciences — chemistry, much of physics, etc. — are relatively easy when it comes to getting more exact measurements. You have known constants and specific end points which you can measure. In the simplest terms, say you wanted to test a substance to see if it turns water blue. You gets the substance, some water and you put the substance into the water. Does it turn blue? That is the end point. Simple.
But the social sciences, psychology and psychiatry included, can be difficult to quantify because you are relying on self-reported behaviors and feelings. Does this drug make you feel less nervous? Well, that would depend on the person and how they define “less nervous.” Some people are nervous all the time. Some people tell researchers what they think that researcher wants to hear.
Many people — a very great number of people — are susceptible to the placebo effect. You give someone a pill that is nothing but sugar and tell them it’s to increase their energy. Viola! Many people will say they have more energy because they believed the pill is an energy pill. Just believing something is enough to make some people feel as if it is true. Some of those people might even show physical symptoms which suggest they are less sluggish despite the fact that they were only given a sugar pill. The mind’s effect on the body can be incredibly powerful.
Also, it can be difficult to design experiments with humans when it comes to psychiatric issues because you are playing science with people’s mental health, broadly speaking. Suicidal people, for example, deserve the best treatment available. They cannot be dropped into some study where some suicidal people are given something scientists think might help them not kill themselves, and others are given nothing, just to compare the suicide rates of the two groups.
However, advances in brain imaging are moving at an incredible pace. The day is coming when brain scans can be used on a regular, relatively low-cost basis to actually measure whether a person is happy or depressed or telling a lie. They are already being used to measure what effects advertisements have on the brains of the people who see them and judge which types of ads are better at causing the positive feelings that businesses hope will cause you to take that extra step to actually purchase their product.
The point of all this is that if you feel confused about issues where science says one thing one year, and a totally different thing five years later, relax. This is how it is supposed to work. Eventually the system — coming to an initial conclusion and then having other scientists all from all over eventually test your methods, data and results — is supposed to correct itself.
Just as science did with Paxil.
And a few mistakes don’t call into question all of science, as many conservatives are trying to convince people in order to cast doubt on global climate change and other issues.