Scientists definitely need to do a better job conveying the meaning of their results to the media, but honestly most journalists have little interest in nuance. They want something sexy that will draw a lot of readers.

And that’s the problem—maybe economists and scientists and people who have studied and trained on interpreting research studies can synthesize that data and draw conclusions, but most people really don’t understand what “significant” means in terms of research. Its irresponsible to expect that people reading a news article can understand the subtleties involved rather than being swayed by the headline and sound bites. Ignorant and painful and can lead to real harm when people misunderstand/misinterpret medical findings.

A lot of economists don’t do a bonferroni or even s holm test for multiple comparisons, but would instead advocate that the authors be transparent about how many comparisons were made and allow readers to draw their own conclusions, and yes, a lot of research is crap. But even worse is the ignorance of the media and the way in which they present results. So painful!

Yes, and I’d defend (?) the .05 cutoff lump in journal articles by noting that plenty of studies are designed to include enough subjects (but not more ) to generate detectable results at that point — though the tea leaves of power calculations are an issue in their own right. Still, interesting points and sounds like a good read.