Post Academic

So on the heels of finding the HuffPo College photo gallery/poll about academic publishers comes a well-circulated and oft-blogged-about story in the Monday’s New York Times about an online, open peer-review process experiment undertaken by Shakespeare Quarterly published by the Folger Shakespeare Library.  To give credit where credit’s due, the Chronicle actually reported on what Shakespeare Quarterly (SQ) is doing last month, but you know it’s really big new when The Gray Lady reports on it.  On the whole, we’ve been pushing for innovation in academic publishing on this blog, so this is a welcome development that bears observation.

"Folger Shakespeare Library" by AgnosticPreachersKid (Creative Commons license)

Here’s how the open, online peer reviewing apparently worked: Contributors to a special issue of SQ were given a choice to have their submissions assessed according to a standard blind review or have them posted online at MediaCommons and commented upon by a group of invited experts and “self-selected” readers who register to the site.  From what I can gather, it looks like the online reviewers basically post comments on the submitted essays like you would add comment bubbles on MS-Word track changes.  Guest editor Katherine Rowe of Bryn Mawr calculates that 41 reviewers–invited and party-crashers–posted 350 comments for the four article and three book reviews in the issue.  Any commenter had to be registered, putting her/his good name and reputation behind the criticisms and/or suggestions.

More on the open peer review process below the fold…

Those of us in academia are probably well aware of the conundrums that the project was meant to address.  A good starting point for the experiment would be to keep reviewers honest by getting them to offer helpful advice and fair judgement, while forcing authors to put forth a good, well-written effort that wouldn’t embarrass them in front of a much bigger and much more public audience than just a few anonymous peer reviewers.  According to the NYT:

Many professors, of course, are wary of turning peer review into an “American Idol”-like competition. They question whether people would be as frank in public, and they worry that comments would be short and episodic, rather than comprehensive and conceptual, and that know-nothings would predominate.

But it actually seems like the opposite happened.  The Chronicle story interviewed one of the guinea-pig submitters, Associate Prof Ayanna Thompson for Arizona State, who explains how academic power plays could still appear in what people presume is the virtual Wild West of the comment thread.  Tenured bigwigs (felt like they) had more latitude in commenting, while scholars earlier in their careers were more reticent, sometimes refraining from participating lest they received some collateral criticism from the more prominent folks.  As Prof. Thompson notes:

“The people who were commenting online were very established, senior scholars. In fact, several junior scholars notified me offline that they had comments, but that they did not want to post them in case they contradicted the senior scholars,” she said. “So while the process was supposed to democratize the review process, it did keep some of the old hierarchies in place.”

Perhaps, though, peer pressure and the threat of public shaming wouldn’t hurt anyone, putting all involved on their best behavior–especially the senior figures in the field, who wouldn’t want reveal themselves as Potemkin professors.

The other big issue that any online experiment would address is what we’ve called the time-lag problem.  Apparently, the results here are promising in terms of the time frame, since the articles were posted over basically a two-month period, from March 2010 to May 2010.  And considering that the issue will be in print a few weeks from now in mid-September, that means, what, aseven-month turnaround time?  That’s actually pretty impressive, because sometimes it feels like it takes seven months for an invited blind reviewer to find an afternoon to give a submission a thumbs up or down!

What’s most interesting to me, as a real-life editorial type now who’s trying to think about how to best make use of digital media, is where the attempt to innovate occurs in the chain of events.  I always thought that going online could facilitate publication and circulation at the end of the production process, but it seems like SQ used social networking in the middle of things to promote the more scholarly and potentially interactive dimensions of publishing, enlisting experts to provide comments and assessment.  It’s smart, too, because production (I’ve learned) goes as fast as it can, once you have material to work with.

Sure, we could offer some nitpicky little criticisms–er, suggestions–of the SQ project from what we know about it, but nothing is going to happen if folks don’t try something new, whatever impediments and unforeseen difficulties they might run into.  In the end, whatever the glitches and hiccups it might’ve encountered this go-around, the online open-review process sounds like it could be more about the journey than the destination.

We’ll chime in on what we think are some more things to brainstorm about next time!