OJS OCS OMP OHS

You are viewing the PKP Support Forum | PKP Home Wiki



HOW-TO: Import and Export Data Using OJS

Are you an Editor, Author, or Journal Manager in need of help? Want to talk to us about workflow issues? This is your forum.

Moderators: jmacgreg, michael, vgabler, John

Forum rules
This forum is meant for general questions about the usability of OJS from an everyday user's perspective: journal managers, authors, and editors are welcome to post questions here, as are librarians and other support staff. We welcome general questions about the role of OJS and how the workflow works, as well as specific function- or user-related questions.

What to do if you have general, workflow or usability questions about OJS:

1. Read the documentation. We've written documentation to cover from OJS basics to system administration and code development, and we encourage you to read it.

2. take a look at the tutorials. We will continue to add tutorials covering OJS basics as time goes on.

3. Post a question. Questions are always welcome here, but if it's a technical question you should probably post to the OJS Technical Support subforum; if you have a development question, try the OJS Development subforum.

HOW-TO: Import and Export Data Using OJS

Postby jmacgreg » Thu Mar 20, 2008 6:52 pm

Expanded documentation on importing and exporting data into OJS is now available, culled from previous forum posts, READMEs and developer documentation:

http://pkp.sfu.ca/files/docs/importexport/index.html (HTML)

The Table of Contents is as follows:

Preface
Introduction
Articles and Issues XML Plugin
-Importing Articles and Issues
-Exporting Articles and Issues From the Web
Using the Web-based Expedited Submission Process
Users XML Plugin
-Importing Users
-Exporting Users From the Web
CrossRef XML Export Plugin
-Exporting Crossref Information From the Web
Erudit Article Export Plugin
-Exporting Erudit Information From the Web
PubMed XML Export Plugin
-Exporting Pubmed Information From the Web
Using the Plugins from the Command Line
Appendix A. Example Articles and Issues Import file
Appendix B. Example Users Import XML File

If you have any further tips, tricks or suggestions, please post them here. This is an evolving document, and comments are welcome.

Cheers,
James
jmacgreg
 
Posts: 4162
Joined: Tue Feb 14, 2006 10:50 am

Re: HOW-TO: Import and Export Data Using OJS

Postby jmacgreg » Wed May 07, 2008 10:36 am

A collection of relevant links to this article will be kept and updated in this post:

Import error with function getFilePath:
User has problem with exported XML filesize, and posts a fix.
viewtopic.php?f=8&t=3086&p=11165#p11165
jmacgreg
 
Posts: 4162
Joined: Tue Feb 14, 2006 10:50 am

Re: HOW-TO: Import and Export Data Using OJS

Postby dlove » Tue Sep 16, 2008 3:23 pm

I was just wondering if anyone has developed a user-friendly form (web or desktop) that allows a non-technical user to enter in back issues. I'm looking for a way for one of our users to enter in a large amount of back issues in both French and English. The form would either push the metadata directly into the Db or create an XML document that could then be imported using the OJS Articles & Issues XML Plugin. If no form exists are there any plans to develop a plugin?

Thanks.
dlove
 
Posts: 11
Joined: Wed Oct 31, 2007 2:43 pm

Re: HOW-TO: Import and Export Data Using OJS

Postby jmacgreg » Wed Sep 17, 2008 5:37 pm

Hi dlove,

I don't know of any other easier-to-use import plugins than the ones that currently ship with OJS. Whiel writing XML files may seem to be a large barrier to entry at first, I think that once you give it a try you may find it easier than anticipated. This can be of great help especially when you are dealing with a large amount of back issues -- you can reuse code (ie., not fill in a web form over and over again) and even string multiple issues together in one file.

Cheers,
James
jmacgreg
 
Posts: 4162
Joined: Tue Feb 14, 2006 10:50 am

Re: HOW-TO: Import and Export Data Using OJS

Postby dlove » Thu Sep 18, 2008 10:24 am

Hi James, thanks for the response. I think for us it boils down to our users wanting to add back issues without having to look at or create the XML. I agree it's not difficult to create the XML file but we have had requests for a user-friendly interface. We might end up creating a '1st step' interface that allows the user to enter in the information which then generates the XML file. Once that step is done the XML file can then be edited directly if needed before it's uploaded. Thanks again.

Darren
dlove
 
Posts: 11
Joined: Wed Oct 31, 2007 2:43 pm

Re: HOW-TO: Import and Export Data Using OJS

Postby asmecher » Thu Sep 18, 2008 10:38 am

Hi Darren,

We've had a fair amount of interest from a number of sources in making a plugin to allow for one-step, easy submission of articles with the goal of quickly building back-issues; however, none of this has yet paid off in the form of code. If you do have any resources to put into this, we'd be happy to help.

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7710
Joined: Wed Aug 10, 2005 12:56 pm

Re: HOW-TO: Import and Export Data Using OJS

Postby dantonucci » Wed Sep 24, 2008 3:18 pm

We are looking to implement OJS. We are working on establishing work flows and estimating time for initial start-up tasks. We are looking at loading 40 years of back issues. Has anyone had experience using the XML coding to import a large number of issues and can you estimate the amount of time it takes to get the data in OJS? We will be using some staff and grad students to upload the back issues.

Dana
dantonucci
 
Posts: 1
Joined: Wed Sep 24, 2008 3:13 pm

Re: HOW-TO: Import and Export Data Using OJS

Postby mnason » Thu Sep 25, 2008 12:39 pm

Our experience at UNB in this field is with batch uploaded issues. However, our journals are marked up with XML before being imported/converted to OJS's native.dtd. This initial markup usually takes a good while but it spans more than just metadata.

If one were to record metadata for, and upload, a backrun of 40 issues, depending on the issue, this could take a good chunk of time. Metadata would probably take something like a handful of minutes per article to copy over the relevant metadata to native.dtd at the issue level. If the issues are large this could take a while. I'm assuming it's PDF too... so those would have to be split up and actually available in web-accessible folders. Each article in native.dtd has to link to the individual files.

Anyway, the upload process itself (from command line) is pretty speedy. You tell the php program to go, you wait anywhere from 20-90 seconds (depending on file size) and then the issue is up in OJS.
mnason
 
Posts: 7
Joined: Mon Feb 27, 2006 10:09 am
Location: Fredericton, New Brunswick

Re: HOW-TO: Import and Export Data Using OJS

Postby alally » Tue Oct 14, 2008 1:42 pm

A quick question. I have tried to upload a test file from my desktop. The xml and pdf files are in a folder in My Documents [C:\Documents and Settings\person\My Documents\OJS test]; I included this path in the xml document, but I still get a "Missing a galley file" message. My question is: does OJS expect that the items will be on a server? And, more importantly, does OJS assume that the files will be on the same server as OJS?

Thanks.
alally
 
Posts: 3
Joined: Thu Oct 09, 2008 4:01 pm

Re: HOW-TO: Import and Export Data Using OJS

Postby jmacgreg » Wed Oct 15, 2008 9:53 am

Hi Alally,

Yes, if you choose to point to external files they have to be on a public server -- the XML file is essentially telling your OJS server to grab files from a public machine. The files don't have to be on the same server as OJS; furthermore, if they are, you still have to point to the web directory to grab them (for example, www.example.com/file.xml instead of /www/html_docs/file.xml).

Cheers,
James
jmacgreg
 
Posts: 4162
Joined: Tue Feb 14, 2006 10:50 am

Re: HOW-TO: Import and Export Data Using OJS

Postby asmecher » Wed Oct 15, 2008 10:56 am

Hi all,

FYI, there are two ways to import XML files -- the command line tool (tools/importExport.php) and the web-based tool. If you're using the command line tool, it's possible to use local files on the same machine. If you're using the web-based tool, you'll need to specify full URLs to external files. This is done for security reasons. See docs/IMPORTEXPORT for details.

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7710
Joined: Wed Aug 10, 2005 12:56 pm

Re: HOW-TO: Import and Export Data Using OJS

Postby Flying_Monkey » Thu Oct 23, 2008 9:08 am

Hi people,

I am currently migrating my journal to OJS. I am not a programmer but I know the basics and can alter code according to instructions. I created my own theme and stylesheets with no difficulties. I was starting to really enjoy the process.

But then I came to importing our past issues. I have been really, really struggling with the Import-Export functions. I've read the guide (above) and I have created XML files from the sample tempate provided. That seemed fine. But now I have a problem. How do you actually get the data in the XML file onto the site?

I have tried uploading my XML file, and I get the following message:

"One or more errors occurred during import. Please check to ensure that the format of the import file correctly matches the specification. Specific details of the import errors are listed below.

* This plugin does not support the supplied root node "". Please ensure that the file is correctly formed and try again."

What? The file is exactly based on the sample provided! Can someone please, please and fairly urgently tell me what I need to do because I am going crazy here....:(

Can I say for the future, to any programmers here, that there is an urgent need for a simpler uploading solution - as more people start using OJS who are even less geeky then me, and particularly for those who created online journals previously and now want to migrate several volumes of issues to an OJS site. OJS seems fantastic if you are starting from scratch, rather more daunting otherwise.

Thanks :)
Flying_Monkey
 
Posts: 13
Joined: Thu Oct 23, 2008 8:54 am

Re: HOW-TO: Import and Export Data Using OJS

Postby jmacgreg » Thu Oct 23, 2008 1:25 pm

Hi Flying_Monkey,

Have you been able to validate your XML file? It's worth validating it to catch any errors. If it validates, please feel free to email me a copy of it to look over (to jmacgreg at gmail dot com). Please also include your OJS version number.

On a general note -- the XML import function, while tremendously useful for users who have a large amount of back issues to import, is also considered an advanced tool. We are in the process of updating the importexport documentation, but due to the vast difference of import scenarios we have to support, I don't foresee the process itself becoming any simpler in the near future.

Cheers,
James
jmacgreg
 
Posts: 4162
Joined: Tue Feb 14, 2006 10:50 am

Re: HOW-TO: Import and Export Data Using OJS

Postby Flying_Monkey » Sun Oct 26, 2008 5:40 am

I think it's fine now, James - I have spent some time validating the XML files, and I think I know exactly what I am doing with that aspect at least! There are now other problems however, but those are questions for a different thread...

Thanks,

David.
Flying_Monkey
 
Posts: 13
Joined: Thu Oct 23, 2008 8:54 am

Re: HOW-TO: Import and Export Data Using OJS

Postby mgscano » Fri May 01, 2009 1:34 am

Hey David,
I was having the same error. Find out it was caused by a non-breaking space code left in the abstract, i.e. the infamous  . Once removed everything went smooth (except for the CDATA trick in emails which simply does not work and generates the empty issue in the database.

About the XML validation, I use the native.dtd on the site for validation, but sounds quite weird most of the fields DO NOT validate, even using the sample.xml from that same folder.

The import plugin needs documentation, for sure. I'm not a professional programmer, but I've always been skilled enuff to make ANY CMS work the way it was meant to work. OJS is a real deal, indeed. :-)

bye,
Marco
mgscano
 
Posts: 72
Joined: Tue Jan 27, 2009 3:22 am
Location: Brescia, Italy

Next

Return to OJS Editorial Support and Discussion

Who is online

Users browsing this forum: No registered users and 9 guests