The 5-Point Journal Integrity Initiative

A New Standard for Public Access to Research

John Willinsky, Public Knowledge Project, Stanford University

This project has two closely related objectives in advancing the public’s and the professions’ ability to assess the trustworthiness of the research that each community encounters in their lives and work at a growing pace:

   (a) It provides a means of checking the extent to which research articles and preprints are trustworthy in their compliance with scholarly publishing standards (Table 1). The project will develop internal and third-party verification systems for this compliance, the results of which are communicated in ways that constitute the second objective.

Table 1. Examples of the scholarly publishing standards for which internal and third-party verification systems will be built, with the standards selected for Phase I determined through the consultation and research process described below. 

Standard	Level	Checksa	Metrics	Additional information
1. Research status	Article	Crossref, Publons, Retraction Watch	Versions, downloads	Whether the research is (a) peer-reviewed research, moderated preprint; final draft; or working paper; (b) the version; (c) a research article, letter, editorial, opinion piece, systematic review, etc.; (d) corrected, withdrawn or retracted; and (e) open access or paywalled.
2. Editorial oversight	Journal	ORCiD	Percent of editorial team displaying iDs	ORCiD is a trusted source of academic identities, which provides editors, board members, and reviewers of the journal with an iD that links to their profiles.
3. Peer review	Journal	ORCiD, Publons	Reviewers, time period, and rounds	Availability of open reviews, and article types subject to peer review.
4. Data deposit	Article	Dryad, Dataverse, Figshare	Data set size; presence of tools	Data availability policies and statements for journals, as well as existence of a dataset for replicability and other analytical tools (e.g., Jupyter Notebooks) for articles.
5. Sponsorship	Article	Crossref	No. of funders; average funded	Funders with link from Crossref Funder Registry, as well as author conflict of interest policies and declarations.

   (b) It communicates the verification to the public. The verified extent of a research study’s compliance with scholarly publishing standards will be compiled on a Publication Facts label (based on the FDA Nutrition Facts label), which will be linked to the study and will provide the compliance metrics, along with detailed explanations of each standard and metric (Fig. 1). The labeling is intended to inform and educate the public and the professions on research’s standing, as well as the trusted standards of scholarly publication and dissemination. A Preprint Facts label would be similar, except that “Editor ORCiD iD” would be “Moderator ORCiD iD” and “Peer Review” would be “Moderation,” with a measure of whether the moderator approves the initial posting and subsequent versions. 

Figure 1. This Publication Facts label is an example of a communication strategy to facilitate the assessment of research publishing trustworthiness. 

Verification Systems for Standards Adherence

The goal will be to develop verification systems that involve a two-way interaction that is managed by the network between the journal or article with a trusted third-party industry organization, such as ORCiD. The trusted exchange of information that, for example, verifies an editor’s identity (ORCiD) and that the editor indeed has been appointed (journal designation) is presented here  (Fig. 2). This ensures that the editorial oversight standard can be configured through an API arrangement between journal/preprint platforms and a trusted identity management organization such as ORCiD.

Figure 2. Using the journal PLOS One as an example, the proposed two-way authentication between a journal and a third-party industry organization can be demonstrated in four steps.

These two-way third-party designs will form a basis for initiating agreements with the relevant organizations (e.g., Crossref, DOAJ, ORCiD, Publons) in establishing these authentication services for journals. From the communications and education perspective, the research team will draft the components of the Publication/Preprint Facts label reflecting the publishing standards, measures, and explanations to be incorporated by the graphic designer into a variety of test designs and formats for assessment. 

Table 2. Members of the Research Publishers Panel to review feasibility of verification strategies, publication of measures, and use of the Publication/Preprint Facts label. 

Name	Title	Organization or Publication
1. Richard Gallagher	President & Editor-in-Chief	Annual Reviews
2. Jennifer Gibson	Head of Open Research	Dryad
3. Samantha Hindle	Content Lead	bioRxiv / medRxiv
4. Veronique Kiermer	Chief Scientific Officer	PLOS
5. Janine Latham	Journals Manager	Berghahn Books
6. Daniella Lowenberg	Dryad Product Manager	Dryad
7. Catriona MacCallum	Director of Open Science	Hindawi
8. Alice Meadows	Community Engagement	National Information Standards Organization
9. David Ross	Vice President Open Research	SAGE
10. Seth Truger	Digital Media Editor	JAMA Network Open

PKP Team for Developing Integrity Pieces for OJS and OPS

1. The software developers will work within PKP’s platforms to create three levels of integration across the five points. The first will be to build features into the software that are active by default or that can be activated in the setup of the platform; the second will be to design plugins that can be activated by platform managers and publishers; and the third will also be plugins that call on third-party services, such as DOAJ. 
   
2. The graphic designer will establish a common look and feel to all components of the five points to build trust and assurance, without intimidation. The use of iconography will be intended to add to the ready awareness of each points’ contribution to integrity,
   
3. The communications specialist will not only write up the explanations intended for the public but with revisions following the UI/UX testing with various segments of the public, including public librarians, library patrons, undergraduate and high school students, as well as translators.
   
4. The user interface and user experience specialist (UI/UX) will assess the technical and graphical design effectiveness, as well as wording, of the internal and public-facing elements introduced through the five points, with a relevant set of users, including publishers, editors, and authors, as well as those listed in #3 above.
   
5. The project manager will ensure that the proper personnel are assigned and employed, while the work is efficiently coordinated and conducted to PKP standards, with quality control checks and UI/UX testing and revisions, as well as soliciting test platforms from publishers to participate in the trials and testing.