Harvester Throttle

[thornleyk]

Hi All,

May I suggest a new Configuration parameter?

It seems as Digital Commons sites have a rather annoying "feature" where multiple requests by the same IP are denied if followed in quick succession.

This mainly occurs on the resumption token. I have got around this by adding:

usleep(2000000);

After the declaration of function handleResumptionToken in OAIHarvesterPlugin

It would make sense if this could be configured in the config.inc file and turned on or off

What do you guys think??

Cheers
Kris

[asmecher]

Hi Kris,

Strange -- were you able to find that particular behavior documented anywhere? I'd rather work with Digital Commons to correct the problem, rather than adding a work-around to the Harvester.

Regards,
Alec Smecher
Public Knowledge Project Team
---
Don't miss the First International PKP Scholarly Publishing Conference
July 11 - 13, 2007, Vancouver, BC, Canada
http://ocs.sfu.ca/pkp2007/

[thornleyk]

To replicate try this:

In web browser go to:

http://digitalcommons.massey.ac.nz/cgi/ ... b=Identify

and then once displayed quickly press refresh. This will simulate what the harvester is presented with.

Cheers
Kris

[asmecher]

Hi Kris,

I see your point. I'll file an entry in Bugzilla for the next release recommending a throttle option in the configuration file.

Regards,
Alec Smecher
Open Journal Systems Team

[asmecher]

Hi all,

FYI, the bugzilla entry for this issue is at http://pkp.sfu.ca/bugzilla/show_bug.cgi?id=2822. The new feature will be released with the next version of the Harvester.

Regards,
Alec Smecher
Public Knowledge Project Team