Ryan Marcus, assistant professor at the University of Pennsylvania. Using machine learning to build the next generation of data systems.
    ____                       __  ___
   / __ \__  ______ _____     /  |/  /___ _____________  _______
  / /_/ / / / / __ `/ __ \   / /|_/ / __ `/ ___/ ___/ / / / ___/
 / _, _/ /_/ / /_/ / / / /  / /  / / /_/ / /  / /__/ /_/ (__  )
/_/ |_|\__, /\__,_/_/ /_/  /_/  /_/\__,_/_/   \___/\__,_/____/
      /____/
        
   ___                   __  ___
  / _ \__ _____ ____    /  |/  /__ ___________ _____
 / , _/ // / _ `/ _ \  / /|_/ / _ `/ __/ __/ // (_-<
/_/|_|\_, /\_,_/_//_/ /_/  /_/\_,_/_/  \__/\_,_/___/
     /___/
        
   ___  __  ___
  / _ \/  |/  /__ ___________ _____
 / , _/ /|_/ / _ `/ __/ __/ // (_-<
/_/|_/_/  /_/\_,_/_/  \__/\_,_/___/
        

How not to complain about peer review

A cartoon of a scientist's shiny machine and a beat-up paper. Caption: I can’t believe reviewer 2 didn’t comment on the flux initiators! Designed and captioned by me, rendered by Gemini.

I think many complaints about peer review and publication decisions stem from a basic misunderstanding that goes something like this: I did good science, I wrote it up, and now my peers will read the paper and decide whether the underlying science is good.

The mistake is treating “the science” as the product and the paper as its container. The paper is the product, with the purpose of communicating the new insights and intuitions you gained from doing the science. For the reader, that communication is the contribution.

If three knowledgeable peers read a paper and all miss its central result or big idea, that is evidence that the paper, in its current form, is not making the intended contribution. The reviewers may be inattentive, mistaken, random, or unfair – we have plenty of evidence suggesting this happens. But blaming reviewers for failing to recognize your brilliance is rarely the most productive response. The better framing is: how can I meet my peers where they are and write this in a way that advances the field’s collective understanding?

I worry that many complaints about peer review reflect an underlying, unspoken, and unpleasant view of scholarship: peers are obligated to recognize the value of your work, but you have no corresponding obligation to make that value accessible to them. Would you want to read papers written by people who think that way? If not, it’s worth asking yourself if you’ve consciously or unconsciously held this view before — I know I have.

This is mostly orthogonal to impact, which can absolutely come from something other than a paper. If you build a great system that is widely used, that is real and valuable impact – but that doesn’t mean everyone should read a paper about it. A paper about such a system should communicate the new insights, techniques, or principles that others can learn from and build upon.