Suggestions for enhancements in version 2

[osborne]

1. I understand the you plan to separate control over editorial and administrative tasks in version 2. I think that is a very good idea---it's not good now that any editor can change the design of the journal. My preference would be also to limit the actions one editor can take regarding a paper being handled by another editor. Currently, one editor can assign referees to a paper being handled by another editor, which seems to me inappropriate.

2. I would like not to have to specify in advance, as part of the journal setup, the number of referees per paper. Some papers will have no referees, others will have one, two, three, four, or maybe more. It would seem to me to be a better design to allow any number of referees (with there always being an "add a referee" button available).

3. Most referees of the journal we're planning will want to submit reports as pdf files. (They will need to include some math in their reports, which text doesn't easily accommodate.) I realise that I could use the functionality of the "annotated version of paper" upload feature, although the program may treat this file differently from the way in which a referee's report should be treated. Thus I'd like referees to have the option of uploading their reports as pdf files.

3.1 If the uploading of referee reports as pdf files is allowed, they should be stripped, upon receipt, of the author-identification that some pdf-creation programs add (so that the author of the paper, who sees the report, can't find out the identity of the referee). See for example http://economics.ca/cje/en/pdfclean.php (which is maintained by Werner Antweiler at UBC).

4. When a referee is reminded about an overdue report, it would be useful to keep a log of the reminders (so that the editor remembers whether she has sent one...).

5. We expect at least many of our papers (like most papers in economics) to go through more than one round of refereeing. That is: paper is submitted -> gets refereed -> editor sends back to author for revision -> author sends back revised paper -> editor sends revised version to referees (probably the same ones as on the first round, but possibly new ones) -> and so on (possibly through several rounds). The current system doesn't seem to easily accommodate this workflow, though maybe I don't fully understand it. When an author submits a new version of a paper, the editor has available the option to make this available to all reviewers of the first version, but not, as far as I can see, the option to make it available to some, but not others, and possibly some referees who did not see the first version. Are the referees notified and given new due dates for reports? It seems that they are not ... does the system allow that?

6. It would be useful to keep (and make available to editors) statistics on referees---how long they took, how good a job they did. (The latter will require editors to rate referees' reports as they come in; allowing comments by editors on referees would be good too.)

[kevin]

1. I understand the you plan to separate control over editorial and administrative tasks in version 2. I think that is a very good idea---it's not good now that any editor can change the design of the journal. My preference would be also to limit the actions one editor can take regarding a paper being handled by another editor. Currently, one editor can assign referees to a paper being handled by another editor, which seems to me inappropriate.

We have split what currently falls under the "editor" role into four separate user classes:
- Site administrator: Handles creation of journals and any site-wide settings.
- Journal manager: Manages setup, people, etc., of a single journal.
- Editor: Has access to all submissions, assigns section editors to submissions, publishes issues.
- Section editor: Has access only to assigned submissions.

2. I would like not to have to specify in advance, as part of the journal setup, the number of referees per paper. Some papers will have no referees, others will have one, two, three, four, or maybe more. It would seem to me to be a better design to allow any number of referees (with there always being an "add a referee" button available).

We intend to make the setting for the number of reviewers to only act as a default; the number of reviewers for any particular submission can be more or less than the default.

4. When a referee is reminded about an overdue report, it would be useful to keep a log of the reminders (so that the editor remembers whether she has sent one...).

We intend to have an option to save emails sent out by the system related to a particular submission to be stored and viewable alongside that submission.

5. We expect at least many of our papers (like most papers in economics) to go through more than one round of refereeing. That is: paper is submitted -> gets refereed -> editor sends back to author for revision -> author sends back revised paper -> editor sends revised version to referees (probably the same ones as on the first round, but possibly new ones) -> and so on (possibly through several rounds). The current system doesn't seem to easily accommodate this workflow, though maybe I don't fully understand it.

There are many changes in the works related to the review process that should accommodate a more iterative workflow.

6. It would be useful to keep (and make available to editors) statistics on referees---how long they took, how good a job they did. (The latter will require editors to rate referees' reports as they come in; allowing comments by editors on referees would be good too.)

An optional rating option for editors to rate the quality/timeliness is already planned.

[osborne]

Excellent! Any estimate of when version 2 will be available?

[osborne]

Another request:

7. Referees be reminded automatically when they are late. One possibility is to send reminders that are completely automated. I prefer that each editor get a warning when referees are overdue, and be given the option of sending a reminder (which s/he can customize). It would be good if each editor could have a "to do" page, where tasks are listed in order of some kind of priority: e.g. referee reports that are overdue, papers for which reports have been received but no decision has been made, papers that have been sent back for revision and not yet returned. (I realise that referees can now be reminded manually, but there is no warning that they are overdue, and no ordering of priorities (that I can see).)

[blake]

I really like OJS, but the seven points osborne has identified are really importnant!

Thanks.

[blake]

Now that my first manuscripts are completing initial review, it is clear that the reviewer workflow really needs work. I hope these are the changes coming in 2.0. Specifically, the following are serious issues that anyone should know about before starting serious work with OJS:

1) If a reviewer is obstinant and just sends their review in by email, the editor MUST have the ability to enter thier information into the system. Right now I cannot enter their recommendation for them or event that they have sent in the review - I have already sent out embarassing reminders for completed reviews.

2) It is very rare that a paper gets "ACCEPTED" or "ACCEPT with Minor Revisions". In most cases a paper requires an additional review cycle. But the only options for the editor are "ACCEPT" or "DECLINE". Neither one is appropriate in this case. I am having to "DECLINE" people's papers and then send them an email urging them to resubmit their revised paper as a brand new manuscript.


Thanks

Blake

[gmkrishn]

From what I read so far, Version 2.0 (to be released by the end of 2004) will allow a paper to have multiple rounds of refereeing. It would be useful if the initial few rounds can be given a different name (chosen at journal setup time). So, if I choose to name the first round X, second Y and the rest Z, then in my refereeing model I have one round of X, one of Y and many Z's.

Please consider this feature in version 2.0.... Here is why I (and hopefully others) will find it useful.

In our journal, we hope to have a pre-refereeing round before the referees look at the paper. In the pre-refereeing round some experts go through the paper superficially, and tell whether the paper should be rejected or not (with the journal's goals in mind). This round for example, will quickly handle off topic submissions, or papers whose claim (even if true) is not considered important enough to be published.

So if my suggestion is implemented, I will call the first refereeing round "pre-refereeing" and the others will be as usual. As far as OJS is concerned I just have multiple refereeing rounds. And the implementation just needs one additional table (roundnum, Name).

- Murali

[John]

Another request:

7. Referees be reminded automatically when they are late. One possibility is to send reminders that are completely automated. I prefer that each editor get a warning when referees are overdue, and be given the option of sending a reminder (which s/he can customize). It would be good if each editor could have a "to do" page, where tasks are listed in order of some kind of priority: e.g. referee reports that are overdue, papers for which reports have been received but no decision has been made, papers that have been sent back for revision and not yet returned. (I realise that referees can now be reminded manually, but there is no warning that they are overdue, and no ordering of priorities (that I can see).)

We are establishing an automated reminder system for reviewers (configurable by the editor), and we currently have an "Active Assignment" table for editors that shows them, on one page, exactly what stage each manuscript they've been assigned is at, with dates for each step, so the editor can judge where there are holdup besides the reviewers, and notify accordingly (copyedtiors, etc.).

[John]

1) If a reviewer is obstinant and just sends their review in by email, the editor MUST have the ability to enter thier information into the system. Right now I cannot enter their recommendation for them or event that they have sent in the review - I have already sent out embarassing reminders for completed reviews.

2) It is very rare that a paper gets "ACCEPTED" or "ACCEPT with Minor Revisions". In most cases a paper requires an additional review cycle. But the only options for the editor are "ACCEPT" or "DECLINE". Neither one is appropriate in this case. I am having to "DECLINE" people's papers and then send them an email urging them to resubmit their revised paper as a brand new manuscript.

We are adding a feature that enables the editor to record the decision and comments (and noting it was the editor that did so). Also, we are including expanded options for the editor's decision, including Resubmit for Review, Pending Revisions (when it is the Editor's call alone). These decisions are being recorded and dated as part of a much more complete record keeping.

[maulsby]

Hi Folks,

Thanks for creating the Online Journals System - amazing work!

The editorial team at Postcolonial Text includes two layout editors, but the Editing / Request Galleys function assumes only one editor.

Would you please consider generalizing it so that either:
1) Galleys work goes into a queue shared by multiple Layout Editors, who can then "seize" a galley for their personal queue
or
2) An Editor can select a Layout Editor to whom s/he will assign the Galleys

I imagine that you already have something like this for assigning a Proofreader.


Thanks,
David

[kevin]

Layout editors are a distinct user role in 2.0 and so are now assigned to submissions in the same manner as other types of users.

[ramon]

Dear OJSers,

The problem with automatic reminders is that, if the reviewer is out of the office (here in Brazil it´s very common, because of University strikes, vacations, and the such!), he will not get the reminder.

Of course, the editor needs to do their job, which is to take care of submissions, but what happens when the editor´s out of the office as well??

Is there a simple way to recover a rejected paper because it was not reviewed?? I mean, I don´t want to bother the author, because it´s not their fault, as they have filled all the metadata correctly.

Changing the dates and the status should do it, right?

thkns

[kevin]

I don't really understand how a paper that was not reviewed would become rejected in OJS without user action. An editor has to specifically reject a paper.

That said, there's no way in the OJS interface to change the editor decision if an incorrect decision was entered accidentally, but resetting the appropriate database fields will work.

[ramon]

Kevin,

I´m not sure what is happening, but this is what the editor is complaining about:
A paper was submitted and a reviewer was assigned to it. The review was not made and the editor did not do anything (apparently). The deadline expired and the document was then arquived and rejected.

If OJS doesn´t do this automatically, then this means someone is doing this then??

[kevin]

OJS does not do that automatically. An editor must explicitly choose "Decline" under Editor Review and submit their decision for an article to be rejected.