Five-Star science?

OK, we all know how to tell the difference between really good scientific research and over-hyped rubbish. Don’t we? Well, perhaps not all of us. If you have 25 years experience judging the value of scientific papers, you probably don’t need much help. But if you’ve  only just started, or if scientific papers are only a small part of what you do, you might be screaming for some help. Well help is on its way.

It comes from the Science Media Centre whose two Fionas (Fox and Lethbridge) explain: “ Too often science which is at a very early stage – like a conference abstract or preclinical trials in mice – is reported with the same prominence and excitement as large studies conducted in humans and published in top journals.  And all too often we see a single observational study, which can never show that X causes Y, reported as if a link is proven and awarded the same front page prominence as a gold standard RCT or meta-analysis.” They quote a statistic that really worried me: in a survey by the Academy of Medical Sciences, only 37% of the public said they trusted evidence from medical research.  The AMS decided that a labelling system for press releases might help and the SMC developed one. 

How’s it work? Somewhere near the headline of a health or medical press release there will be two or three “ratings”. The first would describe what kind of study is being reported: randomised controlled study, opinion piece, case report whatever. The Second shows who or what the study was done on: animals, humans, cells. The third shows whether it has been peer-reviewed or not. 

OK, it’s not perfect. It’s not going to stop those “coffee causes cancer” headlines. But it might help a desperate reporter to keep a rocky story out of the paper. It might help a good PR to negotiate a reasonable headline with a pushy scientist. What’s not to like – and that’s what people thought when it was trialled. So watch out for the star ratings which will be used from July.