OAI_PMH v2.0 - Flow Control

Open Harvester Systems support questions and answers, bug reports, and development issues.

Moderators: jmacgreg, michael, John

Forum rules
The Public Knowledge Project Support Forum is moving to http://forum.pkp.sfu.ca

This forum will be maintained permanently as an archived historical resource, but all new questions should be added to the new forum. Questions will no longer be monitored on this old forum after March 30, 2015.
Posts: 3
Joined: Thu Mar 03, 2005 9:57 am
Location: Tucson, AZ

OAI_PMH v2.0 - Flow Control

Postby robackja » Thu Mar 03, 2005 10:10 am

Does the PKP harvester support OAI protocol's flow control http://www.openarchives.org/OAI/2.0/openarchivesprotocol.htm#FlowControl?

Archives such as http://arxiv.org use HTTP 503 Retry-After replies to limit the speed at which harvester can harvest.

I personally think it is nothing but annoying and stupid, but none-the-less it is there.

What seems to be happening is after the first ListRecords request is made, the webserver returns 503, with a specified number of seconds to wait before requesting again. PKP just seems to stop there, only harvesting the first set of records returned from ListRecords.

I am currently doing a research assistantship at the University of Arizona and I can certainly provide help in adding this to PKP. Regardless, I will have to add it to our local copy of PKP, as we want to be able to harvest sites like http://arxiv.org. :)

Posts: 338
Joined: Tue Oct 14, 2003 8:23 pm

Postby kevin » Thu Mar 03, 2005 11:01 pm

Currently the PKP Harvester does not support flow control. This is a feature we will consider adding in a future version, when we next have time to work on improvements to the harvester.

If you do implement this yourself and make the code available, we'd be happy to merge it in with ours.

Return to “Open Harvester Systems Support and Development”

Who is online

Users browsing this forum: No registered users and 2 guests