The Johns Hopkins University has released its Open Society Institute-funded survey and evaluation of open-source electronic publishing systems.

OJS is one of the projects investigated in the study. From the report:

Strengths
OJS runs on multiple platforms, including Windows, and it is not Web server dependent, i.e., it runs on either Apache or IIS. It is easy to install and had the best, most comprehensive and clear documentation of any of the systems under consideration. It provides support for multiple discrete publications, all from within a single instance of the application. Each publication is separately skinnable. It appears to be highly extensible via a well-defined plugin API. It has a large deployment and an active developer and user community. OJS models the entire scholarly publications process, from author-initiated account generation and article submissions, through peer-review, editing, copy-editing, production, publication, and archiving. It includes well-thoughtout administrative roles and default workflow. Its selection of bibliographic "reading tools" is interesting and useful.

Weaknesses
Based on this review, potential improvements for OJS would be support for an outside authentication mechanism, e.g., CAS, SiteMinder, WebAuth, Shibboleth; perhaps, like Hyperjournal and Topaz, integration with external RDF repositories; and the facility for using an external repository for persistent storage. Such additions are probably suitable for development as plugins, yet might be central enough for the main developers of OJS to consider making more closely coupled as part of the application architecture.