Why I stopped caring about peer review, and learned to love the work

Peer review is often upheld as one of the revered checks and balances that ensures some kind of quality in science; but, in an increasingly competitive world, peer review in journals has become ineffective way of getting to the best science, and it is not always used with honorable intent. Today, if the proliferation of thought and ideas is your goal, the best chance you have is simply to publish work openly online and wait for peer review to come.

Some time in the mid-2000s, just before I left a hard won university position as full tenured Professor, I decided that I no longer cared about academic journal publications. Indeed, I had lost faith in the system of academic quality assurance altogether. If that sounds like the bitter rebuttal of someone who failed, that is not quite true. In fact, I did relatively well in what I consider to be a decaying system. However, I didn't like what I had experienced myself, and felt for those who were starting their careers, who are judged by how many publications they can get. I feel as though my own small success is in spite of rather than because of peer review, and the real world relevance of my work was recognized by the world at large even before it was grudgingly recognized by academia.

Today, thanks to the Internet, the sharing of ideas is a free and open process, and readers can form their own judgements about the value of ideas, whether flawed or polished. A good idea does not have to be stamped and labelled to be approved, and science is done in communities, without the central control of anonymous judges and juries. Preprint archives (originally distributed on paper by SLAC) have now become ArXive.org and ResearchGate provides indexed community based services, more efficiently than gross amalgamations of diverse fields by journals on a monthly schedule.

The idea of peer review

The idea of peer review is that smart people look at what you've done and some kind of democratic vote takes place on whether this is worthy of publication.

  1. Is the work of sufficient written quality?
  2. Is the scientific method good enough?
  3. Does this advance the state of the art?

This is elementary stuff. 1. is work that can be done by any editor; 2. is a boring task often delegated to students who are not fit to make the judgement, or involves the merest `does the author refer to my work?' or `is there any reason why I have to accept this?' This suggests bribing reviewers by referring to just their work, and appealing to their narcissism. 3. is just too easy to say `no' to.

To say it like it is, there is essential bribery and corruption at the heart of the peer review process, as there is in any realm of human power. The result is not always a quality assurance. Nevertheless, this is the standard by which scientists are judged in the world of academia (and funding agencies).

The reality of peer review

The problem is the sheer volume of papers being written today. Scientists are overloaded. Students without experience are often given the task of reviewing papers. Scientists are judged not by quality but by quantity, and this leads to an over production of repetitive noise. This means everyone is asked to review weak papers all the time, which leads to fatigue and a lack of interest. The short cut becomes: is this written by someone I know and trust? Science is hard and it takes time, and there just aren't that many good ideas to be able to publish 5 papers a year about.

Like most social climbing measures, this leads to a split between the haves and the have-nots, which tends to be perpetuated, because the people who are trusted to make the judgements are the establishment of haves.

I have been relatively lucky in my career. My work has earned modest if sometimes grudging respect, sometimes even enthusiasm. I have been lucky to find the respect of an honest and open-minded community. I have worked hard, but have not had to compromise many times to get work accepted. But, when you submit to the peer review process, there is no guarantee that those who are able to see something of value in your work will be offered the change to see and comment on it. In fact, as the volume of publication journals increases, the chances get smaller, and the process less reliable.

As someone who has flouted conventional boundaries for much of my career, I have had to endure painful reviewing processes that took 3-4 years to get certain work past the establishment, and accepted. It was tempting to give up, but I was somewhat determined by the importance of the results. This was on two occasions because established professors (especially in conservative American old-boys clubs like the ACM and IEEE, and the Ivy League schools from Oxford and Cambridge, to MIT, Stanford, Berkeley, etc) decided I was not one of them and tried every possible argument to have the papers blocked. Sad, but a common story. Having become full professor (a process which requires a significant body of publications and a vote from an international committee of people who are already professors), I could afford to be self-righteous in this matter. So I played the game for long enough, no longer. Still, my sympathies go to younger researchers starting out today.

The idea that, if you are merely persistent enough, you will get published begs a different question: what is the value of the process? Sometimes the work does indeed improve as a result of trying to parry the comments of reviewers. More often than not, this is not so much due to their comments as to the extra time spent thinking about it. If we worried less about publishing quantity instead if quality...

You reputation is yours to grow, not others to prune

Science rarely waits for publications to appear anymore. The process is so painfully slow that when a publication appears, either everyone has already read it, or no one is going to. So much nonsense gets into print that it is scarcely a real accolade to be in print.

It takes months or years for a paper to reach print, and who can afford academic journals anyway? So, why would we not think: what if I just self-published my idea in a blog or a website? I could simply ask a smaller circle of people to comment honestly. And it would be searchable by anyone who might stumble across it with the help of modern search engines. Use the archives and the new social sharing sites. Technology can take prejudice out of the equation.

It is up to each of us to exercise best-effort, to listen and improve -- to not waste others' time. So why not wait until we have real ideas, questions, conjectures that inspire others, then write them carefully and put them out there to stimulate a research community, instead of trying to shut down others as competitors (or less important institutions). That is what is now possible, without an old guard to get in the way.

I do research because I love it. And guess what: I love it much more now that I don't care about the publications. It feels more honest and less stressful. I am less inclined to cut corners to meet a deadline.

So is self-publishing better? In most cases, your work will be ignored, so no different from a journal publication. But at least the work will be timely, and available to a wider audience, where Google can find it, and where it might actually be the thing that inspires someone else on their own journey.

MB Oslo Wed Dec 10 19:19:14 CET 2014