Journal Articles and Journal Reviewing

Readers might recall my AAEA president's column from a couple weeks ago, where I highlighted some potentially worrying trends in the academic publishing world associated with agricultural economics.  Since then, I've run across at least two other papers that touch on related themes and problems.  

The first is a paper in the Journal of Economic Perspectives by Jonathan Berk, Campbell Harvey and David Hirshleifer.  The main purpose of the article is to provide some advice to academics on how to referee journal articles.  Their main message, with which I completely agree, is that referees often pay too much attention to minor technical issues and too little attention to the importance of the problem.  Their advice:

Do not dismiss papers that attack larger issues merely because flaws can be found. The important question that you need to assess is whether the flaws actually invalidate the contribution. If the flaws do not rise to this level and you judge the contribution to be important enough to warrant publication, then you should recommend publication. All papers have flaws, and no amount of revision removes all uncertainties. There is always need for further research to provide deeper perspectives. Try to ask yourself the following question: Flaws and all, would I have been pleased to have written such a paper? If yes, that gives a strong hint that it should be strongly considered for publication, flaws and all.

At the beginning, the authors highlight what they see as the main problem:

The review process for academic journals in economics has grown vastly more extensive over time. Journals demand more revisions, and papers have become bloated with numerous robustness checks and extensions . . . Even if the extra resulting revisions do on average lead to improved papers—a claim that is debatable—the cost is enormous. We argue that much of the time involved in these revisions is a waste of research effort.

It seems that my colleague, Wade Brorsen, agrees (at lest with some of these points).  Wade's Western Agricultural Economics Association presidential address was recently published. After documenting the increase in paper and complexity, in his usual frank fashion, Brorsen has the following to say:

One thing that stands out in table 1 is the increase in the length of manuscripts. The increased length is not all bad—since it can sometimes mean more robustness checks and more detail that will help a few readers—but the increased length can also be costly. Not necessarily to authors; as the saying attributed to Blaise Pascal goes: “I didn’t have time to write a short letter so I wrote a long one instead.” The cost is on the reader because it takes more time to read the paper. The cost can also be on the science since the paper may not be read if it is too long. I advocate twenty manuscript pages of text as a target. I select this length because it is roughly my own attention span. Anything much longer and I am not going to read it.

Brorsen also takes issues with the use of impact factors, the pursuit of the interesting over the important, the use of certain statistical techniques, the lack of simplicity, and more.  He ends as follows:

I have suggested several changes that our profession needs to make, such as reducing the length of manuscripts and reducing complexity. I am not the first to make such suggestions. The reviewers and editors are us. If we want to change what we value, we can.