@Jessica- interesting question – I hadn’t seen the issues on Wansink’s work. I know that there is potentially a lot of fuzzy research out there. Getting results with statistical significance becomes so important that people are tempted to look at huge troves of data and find anything significant (which might still be noise) and report that. But then it might not be replicable.

As for me and what I write about other people’s research, I think if it looks like BS or is one small study then I would try to say as such. If I’ve seen multiple studies pointing different ways (as with drinking water) I’d try to say as such too. I forget who said that statistical significance is the start of the conversation, not the end, but it’s a good point.

One question that came to mind when you mentioned Brian Wansink — I’m curious what your reaction has been to questions raised over his lab’s research? Not to imply that you were wrong to cite him; more, your work involves a lot of drawing from relatively new studies in psychology and behavioral economics, and it could be that a particular consensus at the time that you write A has changed by the time you write B. How do you balance getting potentially useful information to readers quickly with keeping abreast of potential self-corrections or debates in the research? (I’m a graduate student who right now is having a very hard time getting anything resembling statistical significance out of her dissertation research, and having doubts that said statistical significance will make for actually useful work, so this has been on my mind lately.)

Great tips!