Harvester2 as an OAI data source?

[obi]

Hi,

is it possible to extend the current release of the Harvester2 to serve as an OAI data source; extracting data from it into another Harverster2? Or has anyone developed data provider functionality into Harvester2?

The reason is that we are working on a thematic harvesting system using Harvester2. Some of the sites we would like to harvest their data takes up to two days to harvest. We were thinking of using a Harvester2_A to harvest all these sites, and another harvester (Harvester2_b) in the same server to filter out the records/items we would like to harvester from Harvester2_A.

remote sites--->Harvester2_A(provider)--->filter--->Harvester2_B

Thanks in advance.

Obi

[mj]

Hi Obi,

We actually had a conference call yesterday to discuss the next round of Harvester functionality we'd like to develop, and acting as an OAI data source was definitely on the list. You can have a look at some of the ideas we were tossing around at: http://pkp.sfu.ca/wiki/index.php/Harvester_Roadmap

MJ

[obi]

Hi MJ,
When do you anticipate this functionality will be available in Harvester2?

Regards
Obi

[mj]

Hi Obi,

We're still in the planning stages of a roadmap for the entire PKP suite, so I suspect we won't be able to start working on this specific feature for a while yet -- I'd roughly guess a couple of months, at the earliest. That being said, we certainly try to prioritize features based on demand among the community, so this will likely be implemented before other things. I'm afraid I that's the best estimate I have.

MJ

[hydra]

I'll second this feature request.

How is the planning and development of the new release going?

Christian

[asmecher]

Hi hydra,

We're working actively on rewriting a number of pieces of the harvester at the moment, with a few major features in mind. One of these is OAI data source support. We'll also be improving performance significantly. No date for the next release yet, but stay tuned.

Regards,
Alec Smecher
Public Knowledge Project Team