Dan Rigby, Michael Burton, and I just published an article in the American Journal of Agricultural Economics on the impact of academic journals - as seen through the eyes of the academics who write journal articles.
Motivating the work is the fact that more emphasis is being placed on the "impact" or our academic work. This can be see most directly in places like the UK where funding directly follows measures of impact. At my own University, we have to write annual "impact statements", and it is commonplace in promotion and tenure decisions for candidates to have to document "impact." One of the most common metrics used to identify impact is the Impact Factor of the journal in which an author's article appears. This impact factor is calculated by measuring citations to articles published in a journal in the two years following the publication date. There are many critiques of the use of the Impact Factor, and my own research with Tia Hilmer shows that using the impact factor of a journal to measure the impact of a particular article is potentially misleading: some articles published in low Impact Factor journals receive many more citations than some articles published in high Impact Factor journals.
In our current research, we wanted to know what academics themselves think of the impact of different journals, were "impact" can mean several different things. We surveyed agricultural and environmental economics who were members of at least one of the seven largest agricultural economics associations throughout the world. We asked respondents to tell us which (of a set of 23 journals) they thought 1) would "most/least enhance your career progression, whether at your current institution or another at which you would like to work" and 2) "The journal whose papers you think have most/least impact beyond academia (i.e., on policy makers, business community, etc.).” We compared the journal rankings based on these two measures of impact to each other and to the aforementioned Impact Factor based on citations data.
We find:
“We find no significant correlation between the journal scores based on the two criteria, nor between them and the journals’ impact factors. These results suggest that impact beyond academia is poorly aligned with career incentives and that citation measures reflect poorly, if at all, peers’ esteem of journals. ”
My favorite part of the paper are a set of graphs Dan put together plotting the various measures of impact against each other. Here's one showing a journal's Impact Factor vs. respondent's perception of the career impact of publishing in the journal.
