Post Academic

Brainstorming the online peer review process

Posted in Publish and Perish by Arnold Pan on August 31, 2010
Tags: ,

"Brainstorming" by Agripolare (Public Domain)

It’s not much of a surprise that I would be thinking more about the online peer review process we discussed last week, since that’s what I do many hours a week now.  Again, whether or not it was wholly successful isn’t really the issue in my mind, but it’s that the folks at Shakespeare Quarterly and MediaCommons sought to innovate peer review and academic publishing.  Like I mentioned last time, I’ve always been thinking about production and distribution when it came to imagining what digital media had to offer, and less about how scholarship and collegiality might also benefit.  So the SQ experiment was definitely illuminating on that front.

What follows, then, are some things that could be brainstormed about the next time someone tries something like this, to build on what SQ and MediaCommons tried on this go-round:

Incorporating responses: One of the outcomes of the project was that there was so much feedback that authors found it took longer to process the comments, both in terms of time and page length.  According to the journal’s editor David Schalkwyk in the piece that appeared in the Chronicle, editors and authors had to spend a good amount time keeping track of how the discussion of the articles went, which also led to more lengthy revisions.  Think of it this way: Don’t you feel indebted to incorporate all the comments that people who’ve really taken the time to read your writing offer you?  Well, multiply that by about 10 times, with the suggestions being public, so that there’s a record to check your changes against.  Getting input is good, but there’s a limit to it, logistically for the editor and mentally for the writer.

More brainstorming below the fold…


Academic publishing goes online–and mainstream?

So on the heels of finding the HuffPo College photo gallery/poll about academic publishers comes a well-circulated and oft-blogged-about story in the Monday’s New York Times about an online, open peer-review process experiment undertaken by Shakespeare Quarterly published by the Folger Shakespeare Library.  To give credit where credit’s due, the Chronicle actually reported on what Shakespeare Quarterly (SQ) is doing last month, but you know it’s really big new when The Gray Lady reports on it.  On the whole, we’ve been pushing for innovation in academic publishing on this blog, so this is a welcome development that bears observation.

"Folger Shakespeare Library" by AgnosticPreachersKid (Creative Commons license)

Here’s how the open, online peer reviewing apparently worked: Contributors to a special issue of SQ were given a choice to have their submissions assessed according to a standard blind review or have them posted online at MediaCommons and commented upon by a group of invited experts and “self-selected” readers who register to the site.  From what I can gather, it looks like the online reviewers basically post comments on the submitted essays like you would add comment bubbles on MS-Word track changes.  Guest editor Katherine Rowe of Bryn Mawr calculates that 41 reviewers–invited and party-crashers–posted 350 comments for the four article and three book reviews in the issue.  Any commenter had to be registered, putting her/his good name and reputation behind the criticisms and/or suggestions.

More on the open peer review process below the fold…