Making OJS 3 Awesome with Usability Testing

August 1st, 2016 by  | 2 Comments

One of our major goals with OJS 3.0 is to deliver an outstanding user experience for our thousands of journals and the many readers, authors, reviewers, editors, and librarians who use them. Since OJS 2.0 was first released way back in 2005, we’ve made continual improvements based on user feedback posted on the support forum, heard during workshops and presentations, and received during the hundreds of conversations we’ve had in-person and online with community members from around the world.

The development team has worked hard to incorporate all of those suggestions into the upcoming version of the software. Watch for an upcoming blog post that will describe some of the major changes you’ll see in OJS 3 (watch for the announcement of an exact date very soon).

In addition to listening to community input, for the past year we have also been running usability tests on iterative versions of the software, first with wireframes, and later with prototypes of the software. facep Watching real users walk through a series of tasks that you thought were easy to do, and watching them stumble or in some cases hit a wall, has been highly informative. Audible groans of “how could we have missed that?” became common.

We were always clear, with the users doing the testing and with ourselves, that it was the software (and our work) that was being tested, not the users themselves (we never call it “user testing”, always “usability testing” or “software testing”). If they had trouble completing a task, that was on us, not them. The development team sincerely believes that OJS 3 is significantly improved because of what these tests revealed and the improvements we were able to make.

Because PKP doesn’t have a huge budget, we’ve used “guerilla” usability testing techniques, involving a volunteer tester sharing her screen via Google Hangout, and two PKP’ers – one leading the session and the other taking notes. Sonya Betz of the University of Alberta Libraries (a PKP Development Partner) deserves a huge thank you for helping to organize and run the tests! The testing session is recorded and the tester is kept anonymous. The recording and notes are shared with the PKP development team, who summarize the findings and identify required changes — some critical, some minor, and some for future consideration. Although low-cost, the results are well worth the effort.

Putting the findings from the usability tests into practice has been successful through the addition of Nate Wright to the PKP development team. Nate brings a wealth of front-end development experience to the team and has jumped into the OJS 3 project with both feet!

We’ll be running at least one more round of usability tests before the upcoming release, and plan to make them an ongoing part of our development cycle. The richness of the data revealed is too important to ignore.

In addition to thanking Sonya, we’d also like to acknowledge all of the volunteer editors, editorial assistants, librarians, authors, reviewers, system technicians, and others who took time from their busy schedules to participate in the sessions. We’d also like to thank the California Digital Library, who conducted the preliminary tests a few years ago and showed us how important and valuable usability testing is for making better software.

Stay tuned for an announcement of the official release date of OJS 3 in the next couple of days!


2 Comments on "Making OJS 3 Awesome with Usability Testing"

  1. Vlad Alex says:

    Can’t wait to see OJS3! Keep it up, and thank you!

  2. Zhihua says:

    Thank you very much for your great contribution.

Comments are now closed.