PKP Bugzilla – Bug 8043
memory exhausted on mEDRA/datacite plugin
Last modified: 2012-12-03 09:45:56 PST
I've obtained an out of php memory while selecting "Export all unregistered issues, articles and galleys" option in mEDRA and datacite plugin.
I tried to apply this patch
but this didn't solve the problem.
I have 256M on memory limit in php.ini, I could increase it further, but maybe there's some bug, I thought.
Giuseppe, try also patching bug #8031. If you could post the error message too (including the file and line number), that would help.
I tried to apply that patch too, but the problem there is yet.
this is the error I obtain :
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 24 bytes) in /var/www-ojs/lib/pkp/classes/core/PKPApplication.inc.php on line 388
Giuseppe, if you split the group of articles in half, do you still get an errorm message? If so, can you narrow in on a particular article that causes the problem, or does it only seem to affect large exports?
With a smaller list, i don't obtain the error, i think it only seem to affect large exports
Thanks for the details, Giuseppe; that sounds like a legitimate out-of-memory error. The more data you export, the more memory will be used putting them together. The only solution I can recommend is either to increase your memory limit or export smaller batches.
it seems that the paging bug fixed in #7967 still affects the function namend
» Export all unregistered issues, articles and galleys
while it was resolved in » Export specific articles
So at the moment we can not export smaller batches.
Andrea, did you apply the patch in bug #7967 against both copies of DOIExportPlugin.inc.php? Depending on your OS, there may be two separate copies of this file and the patch needs to be applied to both. (This distribution quirk has been fixed elsewhere but remains for OJS 2.4.1.)
(In reply to comment #7)
> Andrea, did you apply the patch in bug #7967 against both copies of
Yes, we did.
I can see paginated lists that I can not see in
where I see only never-ending lists without any button at the bottom
Thanks for the details, Andrea; I'll pass this along to the group working on this plugin.
The "Export all unregistered objects" feature is meant for continuous registration of new publication objects. Users usually wish to be able to select all those objects with one click without having to page through results and register object pages separately. Therefore it is on purpose that paging has not been implemented for this view.
The list of "unregistered objects" is not meant for the initial registration of a large journal. If you try to do so you'll need very large junks of memory and you will probably get further problems when trying to register such a large amount of objects all at once with DataCite or mEDRA.
It is recommended to do the initial registration of a journal via the other lists and then use the "unregistered objects" list only for later updates.
That said, it would be entirely possible to implement paging for the list of unregistered objects although I'm not convinced this is really what everybody wants. Unfortunately I do not currently have the resources to do so.
BTW: I made an in-depth analysis of OJS memory consumption recently in another context (Lucene search). I found out that in the case of large internal result sets as necessary for export or re-indexation of whole journals, the currently used PHP MySQL-extension together with ADOdb are the main culprits of huge memory leaks. As long as OJS has to rely on these tools I don't see a way around the memory issues as closing intermediate MySQL result sets does not have the effect of releasing internally acquired memory, too.
One more comment: Do you know that you can partially automate initial export via the command line export tool? Both, registering and exporting objects via command line is possible. A small script accessing your database may help you to avoid manual work when registering large amounts of publication objects.
No, I didn't know, thanks!
Is there any documentation about semi-automatic exports?
Hm, there may be better documentation. The only thing I know of is
php tools/importExport.php will give you general usage information.
php tools/importExport.php DataciteExportPlugin usage
specific information for DataCite.
Probably Alec can tell you more.
There is some general documentation in docs/IMPORTEXPORT, though nothing there specific to mEDRA/datacite.