Strong editors
Frequent commenter The Digital Drake offers the following suggestions on a journal-rating/reporting system:
“Creating a reporting mechanism where authors can rate and answer relatively simple questions about their experiences with various journals seems worthwhile. Anti-gaming and anti-senseless-grudge guards would need to be in place; partnering with an academic-identity enterprise such as ORCID or an article-identity enterprise such as CrossRef might be wise, to start.”
In practice, I think what’s most likely to work here is a strong editor (or editors). You need an editor who maintains a forum where they, and contributors, can report on their experiences with scholarly journals, to inform people who want to know about good and bad forums to publish, review, edit, and read work in. “Good” and “bad” can be measured on a number of axes: how credible their review process is, how long their turnaround time is, how accommodating they are to open access and other reader concerns, and how reasonable or predatory their pricing or contracts are. All of these concerns are of potential interest to the scholarly community.
You need an editor who has the time and the obsession to keep up the forum; an editor who has a good sense of forum moderation (and can decide, for instance, when to require attribution of a post to a known academic, and when anonymous posts should be allowed; and be able to detect and deal with trolls, defamers, shills, and sock-puppets). You need an editor well-enough connected into the world of scholarship to attract a critical mass of reviews and reports from scholars. And an editor whose sponsor, if any, will back them up when they get heat. (Because, as we’ve seen over the last couple of years, anyone who poses a serious and credible threat to questionable publishers’ business is likely to get lawsuit threats, and possibly actual lawsuits.)
The better-known publisher-watch projects that I know of seem to be based on the strong-editor model, including Writer Beware (two dedicated editors, backed by SFWA), and, yes, Scholarly Open Access (one dedicated editor, supported by his library). While I have problems with the judgment of the latter’s editor in some important respects, his tenacity at investigating and reporting on questionable publishers, day in and day out, has had a lot to do with the visibility and attention his work has gotten. If others want to advance a different view of abusive publishers (open access or not), they’re probably going to need to be willing and able to put in at least as much effort.
(I am not volunteering here. I already have my own editorial obsessions that take quite enough of my time, thank you very much; and there are other professionals much more qualified than I to collect and assess relevant information on journals and publishers.)
While this is a big task, I don’t think it’s beyond the grasp of an editor, or a small group of editors, cultivating a larger community of contributors. When it comes down to it, the scope of scholarly publishing is quite a bit smaller than the scope of Amazon or Yelp. A few thousand reports a year could cover the ground pretty well, and there are a number of forums online now moderated by one person or a small team than handle that scale well.
It’s by no means certain that the right people, with the right backing, will show up for this. But it’s seems to me a real possibility, especially if there are enough people and institutions that would find the results of value.
Who is willing to step up to this? The Loon cannot either; a major project struck her on the back of the head recently (it hurt), so her time is if anything oversubscribed for the next year or so.