Scott Alexander of Slate Star Codex, a psychiatrist research blog, gets entertainingly agitated about research on a gene named 5-HTTLPR, which was originally pronounced to be the key part of the genetics behind depression:
In 1996, some researchers discovered that depressed people often had an unusual version of the serotonin transporter gene 5-HTTLPR. The study became a psychiatric sensation, getting thousands of citations and sparking dozens of replication attempts (page 3 here lists 46).
Since then, genetics has advanced a long ways, and now studies trying to replicate the 5-HTTLPR effect have been failing, due to a better understanding of how to apply statistics to genetics. This all leads up to this description of the research on 5-HTTLPR:
First, what bothers me isn’t just that people said 5-HTTLPR mattered and it didn’t. It’s that we built whole imaginary edifices, whole castles in the air on top of this idea of 5-HTTLPR mattering. We “figured out” how 5-HTTLPR exerted its effects, what parts of the brain it was active in, what sorts of things it interacted with, how its effects were enhanced or suppressed by the effects of other imaginary depression genes. This isn’t just an explorer coming back from the Orient and claiming there are unicorns there. It’s the explorer describing the life cycle of unicorns, what unicorns eat, all the different subspecies of unicorn, which cuts of unicorn meat are tastiest, and a blow-by-blow account of a wrestling match between unicorns and Bigfoot.
And the first thing that pops to mind? Research on the nature of God. So many conclusions about a creature for which there’s no evidence.
Ah, well, I’m just tired and feeling a bit snitty. My thanks to Mr. Alexander for his condemnation of hundreds of studies. I have no idea if he’s right, but he’s certainly fun.