What should we know?

I decided a while ago that I didn't want this blog to be a bad science blog. There are plenty of those, I really like them but as the market's a little swamped I thought I'd just talk about stat-mech and hope that someone thinks it's interesting as well. Last weekend, however, I went to a talk by Ben Goldacre in Bath and so these things were brought to mind.

The thrust of the talk was that we, the public, are being misled and lied to by the media when it comes to science. He has compelling examples whereby the media would print unpublished stories from a discredited scientist but ignore several published articles that say the opposite. These examples are clear cut, the media are willing to lie for a good story. Even a well educated member of the public has no chance if information is being withheld.

What if it's less clean cut? Could the blame be shared in some cases? Take this story, the Durham fish oil trial (also mentioned in the talk, I don't have anything new). Uncritically reported by the media this "trial" had no control group, no predefined measure of success and more than a whiff that they knew what the outcome would be before it started. I need go no further describing it. The reasons why this "trial" was of zero scientific value are laid bare for anyone to see. The problem when one accepts what the article is saying (trial will prove fish oil works) without asking the huge question "where the hell's the control group?".

Anyone can ask this question. I expect people to ask this question. The concept of a control group is not difficult and everyone should understand it. In fact a full double blind trial is also easy to understand even if you didn't expect it to be necessary. There are certain things that I believe we should all just know about. Some good starting ones would be

1. Double blind trials. For me I wouldn't have guessed they need to be double blinded, it's great that scientists don't exclude themselves from ruining their own experiments.
2. Statistical significance. Small scale experiments can be good, but you need to be able to say when things could have been chance.
3. Pattern recognition. Related to significance. People are pattern recognition machines, we see things where they are not.

If you ask questions about these things then it'll be a lot harder to slip things past you. If not, you can be taken for a ride. There are few other areas of our lives where we leave ourselves so open to abuse. None of these things are too difficult to understand. It's certainly easier than buying a car...

Anyway, back to physics next time. There's lots I want people to know about physics but that's another fight for another time.