Science needn't hide its mistakes

An ex-Nature editor says that peer review is dead? I hope that isn’t what you’d be left thinking from my Comment in the Guardian (below). We need peer review. It is flawed in all kinds of ways – I think, with the “replication crisis” in the scientific literature, we’re starting to appreciate just how flawed – but it remains a valuable way of determining what is publishable. My point is that there are also other valid ways of presenting science these days, arXiv certainly being one of them. Some folks worry about how bad science might get an airing if peer review isn’t seen as an obligatory gatekeeper – but my God, have you ever looked at what gets published already? Peer review is mostly a good way of making mediocre science less error-prone, not of preventing the dissemination of grandiose dross. I’d prefer to think that people can be taught to understand that, if something hasn’t been peer-reviewed, it should be approached with a pinch of salt and with the knowledge that one needs to hear the assessments of other experts too. It’s possible that a blanket insistence on only announcing peer-reviewed work would make people less likely to get taken in, not more so, because they would come to realise that science is always contingent and liable to be wrong, whether it is peer-reviewed or not, and that peer review isn’t a guarantee of veracity. The filtering process in science is a many-staged one, which includes the post-publication assessment of peers and the longer-term sieve of history – it’s not something that happens, or ought to happen, all at once. With blogs, preprint servers, social media and so forth, this is much more true now than it was 20 years ago, and we need to recognize that.

As for the BICEP2 results themselves, it does seem that the team was rather hasty and sloppy in not waiting for the Planck data but apparently basing their assessment of the dust issue on preliminary findings presented at a conference. But this is no great sin. I’m pleased that people have been able to see that scientists like this are grappling with these huge and difficult questions, and that there are ways we can look for answers, and that sometimes we’ll get wrong ones. Our best protection against oversold and misleading claims is to admit that scientists can make blunders, because they are just people doing their best to figure out these difficult and amazing questions, not priests handing down answers written in stone. So anyway: here it is.

_______________________________________________________________________

It was announced in headlines worldwide as one of the biggest scientific discoveries for decades, sure to garner Nobel prizes. But now it looks very likely that the alleged evidence of both gravitational waves and the ultra-fast expansion called inflation in the Big Bang has literally turned to dust. Last March a team using a telescope called BICEP2 at the South Pole claimed to have read the signatures of these two elusive phenomena in the twisting patterns of the cosmic microwave background radiation, the afterglow of the Big Bang. The latest results from an international consortium using a space telescope called Planck show that BICEP2’s data is very likely to have come not from the microwave background at all, but from warm dust scattered through our own galaxy.

Some will regard this as a huge embarrassment, not only for the BICEP2 team but for science itself. As the evidence against the earlier claims has been mounting over the past months, already some researchers have criticized the team for making a premature announcement to the press before their work had been properly peer-reviewed.

But there’s no shame here. On the contrary, this episode is good for science. This sequence of excitement followed by deflation, debate and controversy is perfectly normal – it’s just that in the past it would have happened out of the public gaze. Only when the dust had settled would a sober and sanitized version of events have been reported, if indeed there was anything left to report.

That has been the Standard Model of science ever since the media first acknowledged it. A hundred years ago, headlines in the New York Times had all the gravitas of a papal edict: “Men of Science Convene” and so forth. They were authoritative, decorous, and totally contrived. That image started to unravel after James Watson published The Double Helix, his scurrilous behind-the-scenes account of the pursuit of the structure of DNA. But even now, some scientists would prefer the mask to remain, insisting that results are only announced after they have passed “peer review”: that is, been checked by experts and published in a reputable journal.

There are many reasons why this will no longer wash. Those days of deference to patrician authority are over, probably for the better. We no longer take on trust what we are told by politicians and leaders, experts and authorities. There are hazards to such skepticism, but good motivations too. Few regret that the old “public understanding of science” model – spoon-feeding facts to the ignorant masses – has been replaced with attempts to engage and include the public.

But science itself has changed too. Information and communications technologies mean that, not only is it all but impossible to keep hot findings under wraps, but few even try. In physics in particular, researchers put their papers on publicly accessible preprint servers before formal publication so that they can be seen and discussed, while specialist bloggers give new claims an informal but often penetrating analysis. This enriches the scientific process, and means that problems can be spotted and debated that “peer reviewers” for journals might not notice. Peer review is highly imperfect anyway – a valuable check, but far from infallible and notoriously conservative.

It is because of these new models of dissemination that we were all able to enjoy the debate in 2011 about particles called neutrinos that were alleged to travel faster than light, in defiance of the theory of special relativity. Those findings were announced, disputed, and finally rejected, all without any papers being formally published. The arguments were heated but never bitter, and the public got a glimpse of science at its most vibrant: astonishing claims mixed with careful deliberation, leading ultimately to a clear consensus. How much more informative it was than the tidy fictions that published papers often become.

Of course, there will always be dangers in “publication by press conference”, especially if the findings relate to, say, human health. All the more reason for us to become more realistic, informed and grown-up in assessing science: to listen to what other experts say, to grasp the basic arguments, and not just to be seduced by headlines. Researchers who abuse the process will very quickly feel the heat.

Aren’t some premature announcements just perfidious attempts to grab priority, and thereby fame and prizes? Probably – and this exposes how distorted the reward systems of science can be. It’s time we stopped awarding special status to people who, having more resources or leverage with editors or just plain luck, are first past a post that everyone else is stampeding towards. Who cares? Rewards in science should be for sustained creative thinking, insight, experimental ingenuity, not for being in the right place at the right time. A bottle of bubbly will suffice for that.

What, then, of gravitational waves? If, as it seems, BICEP2 never saw them bouncing from the repercussions of the Big Bang, then we’re back to looking for them the hard way, by trying to detect the incredibly tiny distortions they should introduce in spacetime as they ripple past. Now the BICEP2 and Planck teams are pooling their data to see if anything can be salvaged. Good on them. Debate, discussion, deliberation: science happening just as it should.