OJS OCS OMP OHS

You are viewing the PKP Support Forum | PKP Home Wiki



conceptual error in acceptance rate calculation

Are you responsible for making OJS work -- installing, upgrading, migrating or troubleshooting? Do you think you've found a bug? Post in this forum.

Moderators: jmacgreg, btbell, michael, bdgregg, barbarah, asmecher

Forum rules
What to do if you have a technical problem with OJS:

1. Search the forum. You can do this from the Advanced Search Page or from our Google Custom Search, which will search the entire PKP site. If you are encountering an error, we especially recommend searching the forum for said error.

2. Check the FAQ to see if your question or error has already been resolved.

3. Post a question, but please, only after trying the above two solutions. If it's a workflow or usability question you should probably post to the OJS Editorial Support and Discussion subforum; if you have a development question, try the OJS Development subforum.

conceptual error in acceptance rate calculation

Postby jmir » Thu Aug 02, 2007 7:49 am

There seems to be a conceptual error in the acceptance- (also decline-) rate calculation in OJS.

In my opinion (and understanding what an "acceptance rate" should measure), when rejecting an article, the acceptance rate should go DOWN. However, I noted that if I reject an article, the acceptance rate doesn't change at all!

Why is this? It's because in OJS (I saw this in JournalStatisticsDAO.inc.php) the acceptance rate is defined as submissionsAccept / numSubmissions.
I.e. the denominator includes ALL submitted manuscripts, even those where no final decision has been made yet. Thus, rejecting an article keeps the acceptance rate unchanged, which is counterintuitive to most people.

The common understanding of the acceptance rate is that as denominator you count only those articles where a final decision reject or accept has been made by the editor, i.e. submissionsAccept / (submissionsAccept + submissionsDecline).

The difference between the two definitions is substantial for a journal where we have a substantial (and changing) backlog of undecided articles / articles under revision. The acceptance rate is then mainly affected by the size of the backlog / new submissions rate, rather than the editor actions.

The reason why this is so important is that the acceptance rate is commonly seen as a metric which tells authors how selective the journal is, and also as an indirect measure for the journals' quality (low acceptance rate = better (as more selective) journal).
However, if the acceptance rate is measured the way OJS defines it, a low acceptance rate is potentially an indication of a long backlog of undecided articles, thus the exact opposite of a quality criterion! If you want to know how many are undecided, then look at the undecided rate, but don't let the number of undecided articles confound the acceptance or rejection rate calculations.

I welcome some discussion on this, but in my opinion this should be fixed. As a minimum, people (authors, editors) must be warned that the OJS acceptance rate is calculated the way it is, which is different from how most journals would calculate their acceptance rate.
jmir
 
Posts: 74
Joined: Mon Feb 26, 2007 12:16 am

Re: conceptual error in acceptance rate calculation

Postby asmecher » Thu Aug 02, 2007 11:56 am

Hi Gunther,

Agreed -- the denominator should be the number of accepted and rejected submissions, not the total number of submissions. This will be corrected in OJS 2.2, and you can patch your existing install with the patch linked at http://pkp.sfu.ca/bugzilla/show_bug.cgi?id=2966 -- you'll have to modify templates/manager/statistics/statistics.tpl by hand, as the patch is generated against CVS but you're running a modified release of 2.1.1. That part is an easy single-line change, though.

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 9050
Joined: Wed Aug 10, 2005 12:56 pm

Re: conceptual error in acceptance rate calculation

Postby jmir » Sat Aug 04, 2007 5:15 am

Just for the record: The acceptance rate given on the JMIR homepage (http:/www.jmir.org) is now the corrected acceptance rate (currently about 50% - was 33% before I made the correction).

Personally, I also think (and might modify our code accordingly unless OJS adds this to future releases anyway) that acceptance rates of articles decided over the past 12, 6, 3 month periods should be disclosed by journals (to allow authors to see some more recent trends), but this may be a modification we might have to hack into the system. Any hints on how to do this query are welcome...

The current acceptance rate disclosed on the JMIR homepage is calculated over the lifetime of the journal (since we use OJS, anyway), which is not so meaningful for authors, who are more interested in more recent trends.
jmir
 
Posts: 74
Joined: Mon Feb 26, 2007 12:16 am

Re: conceptual error in acceptance rate calculation

Postby asmecher » Sat Aug 04, 2007 12:57 pm

Hi Gunther,

The stats page should allow the user to jump back and forth between stats for different years. Is this what you mean?

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 9050
Joined: Wed Aug 10, 2005 12:56 pm


Return to OJS Technical Support

Who is online

Users browsing this forum: Baidu [Spider] and 3 guests