OJS OCS OMP OHS

You are viewing the PKP Support Forum | PKP Home Wiki



Problems Importing with Users XML Plugin

Are you responsible for making OCS work -- installing, upgrading, migrating or troubleshooting? Do you think you've found a bug? Post in this forum.

Moderators: jmacgreg, michael, John

Forum rules
What to do if you have a technical problem with OCS:

1. Search the forum. You can do this from the Advanced Search Page or from our Google Custom Search, which will search the entire PKP site. If you are encountering an error, we especially recommend searching the forum for said error.

2. Check the FAQ to see if your question or error has already been resolved. Please note that this FAQ is OJS-centric, but most issues are applicable to both platforms.

3. Post a question, but please, only after trying the above two solutions. If it's a workflow or usability question you should probably post to the OCS Conference Support and Discussion subforum; if you have a development question, try the OCS Development subforum.

Problems Importing with Users XML Plugin

Postby tshore » Thu Nov 15, 2012 8:26 pm

I spent last night importing 1000+ users into my new OCS site - 50 users at a time!!! Very painful. The site is running OCS 2.3.5 and the users were exported - using the same plugin - from a 2.3.3 site. My exact problem is described in this OJS forum post:

http://pkp.sfu.ca/support/forum/viewtopic.php?f=2&t=2943&start=15#p19352

which I don't think was ever resolved.

It seemed odd to me that this guy's import was cutting out after exactly 10 records and mine was stopping after 50. I suspect that this error might have something to do with the # of items displayed on a page:
Code: Select all
; Number of items to display per page; overridable on a per-conference basis
items_per_page = 25

I really only suspected this because they were both such round numbers and I had this variable set to 50. Changing this variable didn't help, however.

Even if this is probably not the case, I'm hoping that the developers can offer a solution so that no one else will have to endure what Charles and I went through in order to get users imported into our new sites.

Thank you.
tshore
 
Posts: 263
Joined: Fri Nov 18, 2005 12:48 pm
Location: Hamilton, Ontario

Re: Problems Importing with Users XML Plugin

Postby JasonNugent » Fri Nov 16, 2012 8:31 am

Hi tshore,

Did you try doing this from the command line? You can use the import/export plugin to import users:

Code: Select all
php tools/importExport.php UserImportExportPlugin usage


this bypasses the per-page restriction.

Regards,
Jason
JasonNugent
Site Admin
 
Posts: 791
Joined: Tue Jan 10, 2006 6:20 am

Re: Problems Importing with Users XML Plugin

Postby tshore » Fri Nov 16, 2012 9:11 am

No. Is this documented somewhere?

Regardless, you're going to file this as a bug, correct? Pretty frustrating that someone had made a forum post about this error 3 years ago, and there was never any followup.
tshore
 
Posts: 263
Joined: Fri Nov 18, 2005 12:48 pm
Location: Hamilton, Ontario

Re: Problems Importing with Users XML Plugin

Postby tshore » Wed Jan 16, 2013 1:49 am

Ok, so I'm setting up another conference and I find myself back at this same unresolved thread. Tonight I have to import about 1200 users. After first having to track down a bug on the export end of things, now I'm faced - again - with a bug while importing.

Can somebody please file this as a bug, at the very least...
tshore
 
Posts: 263
Joined: Fri Nov 18, 2005 12:48 pm
Location: Hamilton, Ontario

Re: Problems Importing with Users XML Plugin

Postby asmecher » Wed Jan 16, 2013 12:21 pm

Hi tshore,

Are you seeing the same error message that was reported on the other thread ("DB error:column 'first_name' cannot be null")? If so, have you tried validating your XML before importing it?
I'm not sure yet, but I suspect that items_per_page is a red herring (or if not, that you're experiencing a different problem than the fellow on the other thread).

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7736
Joined: Wed Aug 10, 2005 12:56 pm

Re: Problems Importing with Users XML Plugin

Postby tshore » Wed Jan 16, 2013 12:31 pm

Yes, still get the same problem of only being able to import 50 records at a time.

I ended up doing it from the command line for this conference (based on Jason's suggestion above). That turned out to also be painful and a long, drawn out procedure.

I didn't validate my XML, but, considering it was generated by the OCS export user plugin, I didn't think that this would be necessary. Also, the fact that I can reliably import 50 records at a time seems to indicate that that XML is just fine.

I will provided details of the problem with the command line import and the solution/hack I came up with to get past it.
tshore
 
Posts: 263
Joined: Fri Nov 18, 2005 12:48 pm
Location: Hamilton, Ontario

Re: Problems Importing with Users XML Plugin

Postby tshore » Wed Jan 16, 2013 1:26 pm

Ok, here's the command that I used:
Code: Select all
php tools/importExport.php UserImportExportPlugin import users-5.xml 2013

The current directory was the main OCS directory and I uploaded the XML file to that directory. 2013 is the path (schedConfPath).

However, I was still getting an error:
Code: Select all
An unknown scheduled conference path "2013" was specified.

This is because, in order to lookup the scheduled conference, the function getSchedConfByPath (in classes/schedConf/SchedConfDAO.inc.php) needed to know the conferenceId. It looks like it should be able to manage without it, but this was not the case. I assume this was because it was being called from the command line.

So, to get it to work, I temporarily hacked line 214 in tools/importExport.php by supplying the conferenceId=1 to getSchedConfByPath.
Code: Select all
-      $schedConf =& $schedConfDao->getSchedConfByPath($schedConfPath);
+      $schedConf =& $schedConfDao->getSchedConfByPath($schedConfPath, 1);

I suppose that in order for the import users to work from the command line, maybe the solution would be to accept the conference ID as another parameter.
tshore
 
Posts: 263
Joined: Fri Nov 18, 2005 12:48 pm
Location: Hamilton, Ontario

Re: Problems Importing with Users XML Plugin

Postby asmecher » Wed Jan 16, 2013 1:38 pm

Hi tshore,

Can you confirm specifically that you're getting the error message I quoted above?

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7736
Joined: Wed Aug 10, 2005 12:56 pm

Re: Problems Importing with Users XML Plugin

Postby tshore » Wed Jan 16, 2013 2:00 pm

Yes, it was the exact same error. Although I can't double-check because everythings already imported.
Code: Select all
DB error:column 'first_name' cannot be null
tshore
 
Posts: 263
Joined: Fri Nov 18, 2005 12:48 pm
Location: Hamilton, Ontario

Re: Problems Importing with Users XML Plugin

Postby asmecher » Wed Jan 16, 2013 3:54 pm

Hi tshore,

I can't duplicate this error with OCS 2.3.5, the current stable branch, or the current unstable branch. I've tested with 60-user imports and all the users come in without trouble.

Without being able to duplicate the error it's difficult to help debug, but my suspicion is that you're trying to import users that don't have a first name. Technically OCS requires a first name to be specified. (There are work-arounds that can be used to trick OCS into accepting users without first names, and if your source conference has used those, it may be that it's generating a user export without first names and the target conference is rejecting it.)

This doesn't explain why importing users in smaller batches works while performing the whole import doesn't. Is it possible that the process you used somehow avoided importing a record without a first name specified? Did you try the entire import in one file when using the command-line based import?

Note that I've filed and entry for the inability to specify a scheduled conference path on the command line as you noted above. There's a patch at http://pkp.sfu.ca/bugzilla/show_bug.cgi?id=8082.

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7736
Joined: Wed Aug 10, 2005 12:56 pm

Re: Problems Importing with Users XML Plugin

Postby tshore » Wed Jan 16, 2013 8:47 pm

No users without a first name in the file.

Yes, entire file worked fine using the command line interface.

Perhaps you can try a larger import file? I'll try to attach the file I used last night.

Didn't work: Sorry, the board attachment quota has been reached.

Perhaps it will let me send it to you in a PM.
tshore
 
Posts: 263
Joined: Fri Nov 18, 2005 12:48 pm
Location: Hamilton, Ontario

Re: Problems Importing with Users XML Plugin

Postby asmecher » Thu Jan 17, 2013 10:04 am

Hi tshore,

Unfortunately I'm still not able to duplicate this problem. I started with a fresh OCS 2.3.5 installation, created a new conference and scheduled conference, applied the patch for bug #8082, and tried first the command-line import:
Code: Select all
$ php tools/importExport.php UserImportExportPlugin import /tmp/users-5.xml import test_sched_conf
This had the expected result:
Code: Select all
The following users were successfully imported into the system:
...followed by a list of 1187 usernames.

Checking the XML you sent for the number of users it includes:
Code: Select all
xmllint --format /tmp/users-5.xml | grep "<user>" | wc -l
1187
...and the database to see how many users it contains:
Code: Select all
mysql> select count(*) from users;
+----------+
| count(*) |
+----------+
|     1187 |
+----------+
(Both include the "admin" user, which is the administrator.)

Next, I restored my installation to its fresh state, and tried the same process via a web import. Importing many users that way results in some very long pages, but again, everything behaved as expected.

There are still many variables at play here -- your PHP version (mine is 5.3.3), your MySQL version (mine is 5.1.50), the upgrade and maintenance history of your OCS install, etc. If any of this process differs substantially from what you've been doing, please let me know.

If you're able to duplicate the error message, try turning on the "debug" option in config.inc.php just before you do it. This will result in any requests to OCS also including database statements, which may be relevant. (Note that you'll only want to turn this on briefly.)

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7736
Joined: Wed Aug 10, 2005 12:56 pm

Re: Problems Importing with Users XML Plugin

Postby NoS » Mon Mar 04, 2013 4:42 am

Hi there,

I'm facing the exact same problem, but with OJS 2.3.8. (but I think it's better to add this to this thread than to start a new one in the ojs-section.)

Server:
OS platform Linux
PHP version 5.3.3
Database driver mysql
Database server version 5.1.61
Update history: 2.3.6, 2.3.7, 2.3.8
No Hacks at all.

Code: Select all
DB error:column 'first_name' cannot be null


when I try to import more than 50 users with user-import-plugin (UI, I have no access to the server or config file). No problems occur when I split the users in numerous files.

Please help!

Regards,
Nora
NoS
 
Posts: 3
Joined: Wed Feb 08, 2012 8:08 am

Re: Problems Importing with Users XML Plugin

Postby asmecher » Mon Mar 04, 2013 9:22 am

Hi Nora,

Have you validated the XML document? Are you sure that all users have a specified first name (which is a required field)?

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 7736
Joined: Wed Aug 10, 2005 12:56 pm

Re: Problems Importing with Users XML Plugin

Postby NoS » Sat Mar 16, 2013 6:40 am

Hi Alec,

all the users were generated via OJS 2.3.8 GUI. I guess the XML is fine, then, including first names, that I checked scrolling through the list.

Regards,
Nora
Last edited by NoS on Tue Apr 02, 2013 2:32 am, edited 1 time in total.
NoS
 
Posts: 3
Joined: Wed Feb 08, 2012 8:08 am

Next

Return to OCS Technical Support

Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 2 guests