We are moving to Git Issues for bug tracking in future releases. During transition, content will be in both tools. If you'd like to file a new bug, please create an issue.

Bug 977 - Statistics and Reports generator
Statistics and Reports generator
Status: RESOLVED FIXED
Product: OJS
Classification: Unclassified
Component: Journal Management
2.1.1
All All
: P1 critical
Assigned To: Alec Smecher
: 952 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2004-06-07 21:33 PDT by Kevin Jamieson
Modified: 2006-05-15 13:25 PDT (History)
2 users (show)

See Also:
Version Reported In:
Also Affects:


Attachments
Stats Page Design (55.00 KB, application/msword)
2006-04-16 22:15 PDT, John Willinsky
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin Jamieson 2004-06-07 21:33:01 PDT
15. STATISTICS
A. ADD NEW FINAL ROW TO JOURNAL MANAGEMENT IN JOURNAL ADMINISTRATION
CHANGE....
Statistics | View the journal's review and publication statistics.
TO...
Stats & Reports | View journal statistics and generate summary reports for journal.

B. ADD TO STATISTICS REPORT ABOVE NO. OF REGISTERED AUTHORS AND TO SETUP 4.7
No. of submissions

C. ADD BELOW STATS TABLE
Report Generator

For submissions received from... 
Year: [  ] Month: [  ]  to Year: [  ]  Month: [  ] 

Include the following in [Choose One]  
                          [Author Report | Editor Report | Reviewer Report |
Section Report]
[ ] Author(s)
[ ] Affiliations
[ ] Title
[ ] Section
[ ] Date submitted
[ ] Editor
[ ] Reviewers
[ ] Status (Accepted, declined, undecided)
[ ] Date decided

[Generate Report]
Comment 1 Kevin Jamieson 2004-06-07 21:40:45 PDT
Could you clarify the functionality required for the "Report Generator"?

My interpretation of the current specs is that this function would only be
available to the journal manager and would display a HTML page listing all
submissions within the specified date boundaries, displaying only the checked
off columns for each submission. But this interpretation does not explain what
the "Author Report" / "Editor Report" / "Reviewer" Report selection is for, so
please clarify.

It is also not clear whether these reports would simply be displayed, or would
be exported to some tab-delimited/XML format to allow importing into another
spreadsheet or database application.
Comment 2 John Willinsky 2004-06-07 23:05:42 PDT
ADDITIONAL DETAIL PROVIDED...

15. STATISTICS
A. ADD NEW FINAL ROW TO JOURNAL MANAGEMENT IN JOURNAL ADMINISTRATION
CHANGE....
Statistics | View the journal's review and publication statistics.
TO...
Stats & Reports | View journal statistics and generate summary reports for journal.

B. ADD TO STATISTICS REPORT ABOVE NO. OF REGISTERED AUTHORS AND TO SETUP 4.7
No. of submissions

C. ADD BELOW STATS TABLE, THIS WOULD NOT BE MADE PUBLIC BUT WOULD GENERATE AN HTML 
PRINT-READY PAGE IN THE FORM OF AN EXTENDED TABLE. EACH ENTRY IN THE TABLE RUNNING DOWN 
THE PAGE WILL BE NUMBERED AND WOULD CONSIST OF A SUBMISSION INFORMATION (DEPENDING ON 
CATEGORIES SELECTED).  THE "JOURNAL" WOULD GENERATE FOR EVERYTHING SUBMITTED AND/
PUBLISHED IN THE DESIRED PERIOD, WHILE "EDITOR" WOULD ORGANIZE THE SUBMISSIONS BY EDITOR, 
AS WOULD "REVIEWER" AND "SECTION"

Report Generator

For submissions received from... 
Year: [  ] Month: [  ]  to Year: [  ]  Month: [  ] 

Include the following in [Choose One]  
                          [Journal Report | Editor Report | Reviewer Report | Section Report]
[ ] Author(s)
[ ] Affiliations
[ ] Title
[ ] Section
[ ] Date submitted
[ ] Editor
[ ] Reviewers
[ ] Status (Accepted, declined, other)
[ ] Date decided

[Generate Report]
Comment 3 John Willinsky 2004-12-08 21:44:49 PST
Also needed in Reviewers (for the Journal Manager) at the bottom of the table of names.

Public Acknowledgement of Reviewers
[  ] Include in the next scheduled issue of the journal an entry at the beginning of the Table of Contents  
(above the initial Section) entitled Reviewers for [Name of journal], which will be linked to an 
alphabetical list of the names of all reviewers who have completed a review since [YYYY] [MM] [DD].
Precede this list with the following note of gratitude:  [textarea]  
Comment 4 Kevin Jamieson 2004-12-08 22:10:09 PST
What does that last item have to do with statistics? That sounds like something
that should be integrated with issue management, as an option when
creating/publishing the issue (although to me makes more sense as something the
editor should do manually, and paste into the issue description field).
Comment 5 John Willinsky 2004-12-08 22:42:57 PST
It is a common function and good practice, thus one it would be good to remind editors about. I placed 
it in this bug because the Report Generator would be picking up this sort of information. I placed it 
under Reviewers as that seemed a natural home, but it is not ideal there, so other ideas are welcome.

I would improve it, thusly...

Public Acknowledgement of Reviewers  (Last published 2004-01-01)
[  ] Include in the next scheduled issue of the journal an entry at the end of the Table of Contents 
entitled Reviewers for [Name of journal], which will be linked to an alphabetical list of the names of all 
reviewers who have completed a review since [YYYY] [MM] [DD].
Precede this list with the following note of gratitude:  [textarea]  
Comment 6 Kevin Jamieson 2004-12-08 23:35:49 PST
Please file that feature request in a separate bug entry, it is not relevant to
the statistics and reports generator.

I suggest reconsidering building such a specialized feature into the system,
however. IMO, this would be more appropriate to handle manually (using the
reports generator to get the information and paste into a submission, as with
other front-matter). I fear "feature-creep" with things like this.
Comment 7 John Willinsky 2004-12-08 23:59:37 PST
All good points.
Ignore references to Public Acknowledgement of Reviewers in this bug report. 
Comment 8 Kevin Jamieson 2004-12-24 00:14:15 PST
Suggest we defer this feature past the initial release, due to deadline constraints.
Comment 9 John Willinsky 2004-12-24 00:22:22 PST
Yes, it is a perfect candidate for deferral. An excellent feature for an upgrade.
Comment 10 Alec Smecher 2005-11-14 15:25:03 PST
*** Bug 952 has been marked as a duplicate of this bug. ***
Comment 11 Alec Smecher 2005-11-22 09:42:02 PST
John, do the various report types (Journal Report, Section Report, ...) only
affect the sort order of the resulting report?

Also, there's no information here on what fields should be included in the
Statistics section (besides mention of # of submissions and authors). I'm
including the following information so far:

No. of submissions  	 2 (0 published)
No. of issues 	0 (0 published)
Users 	Total 	4 (1 signed up for notifications)
Authors 	1
Reviewers 	1
Comment 12 John Willinsky 2005-11-23 23:09:25 PST
1. UNDER MANAGEMENT PAGES LIST...
Report Generator

2. FORMAT
Report Generator

From [2004]-[12] to [2005]-[12].

[ ] Generate journal statistics.
 
[ ] Generate a (()) Journal Report, ( ) Editor Report, ( ) Reviewer Report, or ( ) Section Report
      Include the following information in the report:
      [ ] Author(s)
      [ ] Affiliations
      [ ] Submission title
      [ ] Section
      [ ] Date submitted
      [ ] Editor(s)
      [ ] Reviewers
      [ ] Days to review [use - for incompletes]
      [ ] Status (Accept, decline, revise, undecided)
      [ ] Days to decision

[Generate Statistics and Report]

So for Reviewer, for example, the report for the time period indicated would for each reviewer list the 
full name of Author, Affiliation, Title, etc, depending on what was selected. If Journal Report was 
selected, then the report would list every submission. In every case, order would be by ID # which is 
included by default for every submission. The width of the format will be a challenge, and it is tempting 
to go for a PDF turned landscape with column titles of two lines (but open to other ideas on this).

The stats, which have not yet been specified, would need to include for the time period indicated

         Journal Statistics
2004-12-12  |  2005-12-12
Issues published:
Items published:
Submissions:
  Accept (%):
  Decline (%): 
  Revise (%):
  Undecided (%):
Reviewers:
Reviews per reviewer:
Reviewers' score:
Days per review:
Days to publication:

Comment 13 John Willinsky 2005-11-23 23:10:59 PST
Inconsistencies in dates of period should be resolved in favor of day in FORMAT.
Comment 14 John Willinsky 2005-11-23 23:44:17 PST
Missing Reviewer Score and Nation (to be added to profile) from list of options
      [ ] Author(s)
      [ ] Affiliation
      [ ] Country
      [ ] Submission title
      [ ] Section
      [ ] Date submitted
      [ ] Editor(s)
      [ ] Reviewers
      [ ] Reviewer Scores
      [ ] Days to review [use - for incompletes]
      [ ] Status (Accept, decline, revise, undecided)
      [ ] Days to decision
Also on Editor and Reviewer Reports, need to include Affiliation and Country with each Editor or 
Reviewer name reported out on.
Comment 15 John Willinsky 2005-11-24 08:14:02 PST
FURTHER THJOUGHT AND DETAIL (May be over-extended into web-log analysis).

Report Generator

From [2004]-[12]-[31] to [2005]-[12]-[31].

Journal statistics: [ ] Summary  [ ] Graphs (monthly) 
    Graphs includes Submissions, Acceptance, Review Period, Registered Users, Unique Visitors.

Reports: [ ] Journal Report, [ ] Editor Report, [ ] Reviewer Report, [ ] Section Report
    Report includes details of all relevant submissions and editorial processing.

[Generate Statistics and Report]

--------

Journal Statistics
2004-12-12  |  2005-12-12
Issues published:
Items published:
Submissions:
  Accept (%):
  Decline (%): 
  Revise (%):
  Undecided (%):
Reviewers:  [Total registered]
   No. assigned:  [Total reviewers assigned]
   No. of reviews:  [Average per reviewer]
   Editor score:  [Average score if present]
   Days taken:  [Average days taken to complete]
Days to publication:
Registered users:  [Total for system]
   Country distribution:
Unique visitors:
   Country distribution:

Reports [again with some work on how to lay this out]

Reports Include the following items, grouped by the subject of report {with affiliation and country listed 
for editors in Editor Report and similarly for Reviewers in Reviewer Report]:
Submission ID#; Author(s); Affiliations; Country; Submission title {partial]; Section; Date submitted; 
Editor(s); Reviewers [name on its own line with Reviewer recommendation, Editor score; Days to review]; 
Decision [Accept, decline, revise, undecided]; Days to decision; Days to publication
Comment 16 Alec Smecher 2005-11-28 19:41:56 PST
John,

- For "Registered Users", rather than using the total for the entire system, we
should include only users who are enrolled with this journal.

- I think "Unique Visitors" would be better left to a web server log analysis tool

- We might eventually add chart generation, but I think we've got bigger fish to
fry at the moment (and if the report generates CSV we can leave this to a
spreadsheet)

- I've implemented the range chooser for Statistics as follows:
 <<  2005  >>
where << and >> are links to the previous and next year, respectively.

- Generating PDFs is outside the scope of what we should be doing -- a much more
flexible and usable way of solving the space problem would be an option to
generate a CSV file containing the report.
Comment 17 Alec Smecher 2005-12-08 10:57:54 PST
Implemented additional report types. This needs testing on a proper data set.
Comment 18 John Willinsky 2006-01-02 19:48:00 PST
In discussions with an editor it became clear that more important than overall acceptance/rejection was 
the rejection rate for those sections which are peer reviewed. Also, revise was ambiguous and 
"resubmit" is the ruling that matters.

Journal Statistics
2004-12-12  |  2005-12-12
Issues published:
Items published:
Total submissions:
  Accept (%):
  Decline (%): 
  Resubmit (%):
  Undecided (%):
Journal Section: [Title of Section, with this repeated for each section]
  Submissions:
    Accept (%):
    Decline (%): 
    Resubmit (%):
    Undecided (%):
Reviewers:  [Total registered]
   No. assigned:  [Total reviewers assigned]
   No. of reviews:  [Average per reviewer]
   Editor score:  [Average score if present]
   Days taken:  [Average days taken to complete]
Days to publication:
Registered users:  [Total for system]
  
Comment 19 John Willinsky 2006-01-07 14:46:11 PST
Journal funders (e.g. SSHRC) require more usage detail

Journal Statistics
2004-12-12 | 2005-12-12

Issues published:
Items published:
Total submissions:
  Accept (%):
  Decline (%): 
  Resubmit (%):
  Undecided (%):
  Days to publication:
Journal Section Title1: [Repeated for each section]
  Submissions:
    Accept (%):
    Decline (%): 
    Resubmit (%):
    Undecided (%):
    Days to publication:

Reviewers:  [Total registered]
   No. added:
   No. assigned:  [Total reviewers assigned]
   No. of reviews:  [Average per reviewer]
   Editor score:  [Average score if present]
   Days taken:  [Average days taken to complete]

Users
Registered users:  [Total for system]
  No. added:
Readers:
  No. added:
Subscriptions [Only triggered if Subscriptions activated]
  Type Title1:
    Number added:
  Type Title2:
    Number added:
Comment 20 Alec Smecher 2006-01-10 11:10:19 PST
Additional stats implemented as specified, except that cases like:

Readers: [number]
 Readers Added: [number]

have been condensed to:

Readers: [number] ([number] new)
Comment 21 John Willinsky 2006-04-08 08:55:27 PDT
CHANGE...
Days to publication:
TO...
Average Processing Times (days)
        Review Period [Revision Version date to first Editor Decision date]
        Publishing Period [Submission date to publication date, with accepted papers/days].

AS PER BELOW...

Journal Statistics
2004-12-12 | 2005-12-12

Issues published:
Items published:
Total submissions:
  Accept (%):
  Decline (%): 
  Resubmit (%):
  Undecided (%):
  Average Processing Times (days)
     Review Period [Revision Version date to first Editor Decision date]
     Publishing Period [Submission date to publication date, with accepted papers/days].

Journal Section Title1: [Repeated for each section]
  Submissions:
    Accept (%):
    Decline (%): 
    Resubmit (%):
    Undecided (%):
    Average Processing Times (days)
        Review Period: [Revision Version date to first Editor Decision date]
        Publishing Period: [Submission date to publication date, with accepted papers/days].

Reviewers:  [Total]  (Added)
   No. assigned:  [Total reviewers assigned in period]
   No. of reviews:  [Average per reviewer]
   Editor score:  [Average score if present]
   Days taken:  [Average days taken to complete FROM REQUEST TO COMPLETE]

Users
  Registered users:  [Total]  (Added)
  Readers:  [Total]  (Added)

Subscriptions [Only triggered if Subscriptions activated]
  Type Title1:   [Total]  (Added)
  Type Title2:  [Total]  (Added)
Comment 22 John Willinsky 2006-04-16 22:15:20 PDT
Created attachment 72 [details]
Stats Page Design
Comment 23 John Willinsky 2006-04-16 22:28:01 PDT
After much thought and reworking, attached is a simplified but much more effective stats page 
(apologies for the number of times it has taken to get this far).
Need to add "Days to review" and "Days to publish" to Reports. With "days to review," the calculation is 
now to be made from the date of submission to the editor's initial decision, unless a "Review Version" is 
uploaded, at which point, the calculation is from RV to editor decison.

I would like to consider adding Top 10 Articles, based on views of full-text, which we are already 
capturing for the abstract, and which we were offering in OJS1.

To do so, add to the bottom of the statistics list.
[ ] Most-viewed items         Top 10

And 

Select the sections for calculating this journal's peer-reviewed statistics, and the number of items to list 
for most-viewed items.

  Sections                             Most-viewed items toi display  [10] 
  [Articles
  Clinical Reports
  Book Reviews
  Interviews
  Editorials]                

[Record]


Comment 24 Alec Smecher 2006-05-15 13:25:09 PDT
Implemented as in attached layout, except that the top-10 list has been deferred
in favour of counter-compliant usage stats (see bug #2125).