Harvesting the harvesters

[Scott]

Are there any plans to provide a Data Provider as part of the Harvester distribution, or has anyone implemented this? This would be useful for example if you had a number of institutions harvesting their own data for internal/local purposing, but with the option of harvesting or being harvested by other harvesters.

[asmecher]

Hi Scott,

This is a feature we've considered, and which may make it into the distribution in a few releases; unfortunately I can't be more specific than that until we've firmed up a release schedule.

Regards,
Alec Smecher
Public Knowledge Project Team

[thornleyk]

Hi All,

Our project http://www.oarinz.ac.nz is looking at using the PKP harvester2 as a integral part of the network. The role would be like a hub -> sub hub -> Institutional Repository. I really like the architecture of the Harvester and the plugin flexibility is great. To that end I am developing a OAI out plugin that exposes the harvested data to be harvested by other harvesters.

The project also is looking at devloping a number of other plugins

General Plugin -> SRW/U exposure

Post Process Plugin -> Validation of harvest against predefined standards e.g "At least one Subject" this will feedback info to Repository Admins about how they can improve data in their repositories

Post Process Plugin -> export to local directory FOXML formatted data for ingest by Fedora Repository.

Let me know if these are of any interest to the PKP project

Cheers
Kris
[/list]

[asmecher]

Hi Kris,

Yes, this is certainly of interest -- if you have code, questions, or anything else you'd like to ask us, let me know. You can contact the PKP team at pkp-support@sfu.ca.

Regards,
Alec Smecher
Public Knowledge Project Team

[sfkeller]

thornleyk,

I got a similar impression about the architecture of the code and consider contributing myself a metadata editor as well.

=> What is the time frame for your extensions (1. harvester2 as data provider, 2. SRW binding).

=> What else have you in mind (e.g. OpenSearch? Adding a metadata editor?)

-- Stefan

[thornleyk]

Our project is aiming to have the OAI out, SRW/U exposure and FOXML output plugins in workable format by of this year 2006. And more stable versions by early 2007. This will be made available for download on our web site http://www.oarinz.ac.nz .

As for OpenSearch hmmmm... interesting I will have to consult with the rest of the project team. Our primary goal is the alignment with international standards to increase accessibility so this would be very good.

We are also developing SRW/U exposure for Fedora(http://fedora.info).

[sfkeller]

When mentioning OpenSearch I only wanted to complete the list of potential protocols... Personally, I'm inclined to parsimonious (RESTfull) approaches like OAI-PMH for harvesting and SRU for distributed search.

-- Stefan

P.S. The OARiNZ site slogan mentions "mapping": Does this include metadata about maps and geospatial information about New Zealand? If yes, you should look at a distinct extension of DC information model called DClite4G at this OSGeo page http://tinyurl.com/kfkyv.

[thornleyk]

Sorry "mapping" is in relation to documentation and modeling of the New Zealand Research information space