New version of Harvester2

[djib]

Hello,

i'd like to know if you expect to edit a new release of your harvester soon, or if this part of your development project are stopped for the moment.

thanks

djib

[asmecher]

Hi djib,

We've been working hard on OJS and OCS, but we haven't forgotten the Harvester. At the moment, there are one or two minor unreleased features in CVS, such as support for static OAI respositories. Are there any features in particular that you've got in mind for a follow-up release?

Regards,
Alec Smecher
Public Knowledge Project Team

[ramon]

Hello Alec,

We're using PKP Harvester for our OASISBR repository.
We'll be updating with the latest version this week or the next, because the current version is not working well.
Too many "hacks" to the software created an unmanageable system.
I've "integrated" it with Joomla! for an easier method.

Here are the features we'd like to see in any harvester:

1. Repositories list categorized by listSets, which would enhance the use of the software.
2. Allow the owner of the repository to update the repository information, avoiding replicated repository info
3. Improve the set of checks to verify repository OAI-PMH stantdards compliance, such as content really being UTF-8, correct XML structure. There's an OAI tool that checks repositories for validity available. There may be a way to send the user, or send the admin to that tool, to check and report back errors.

[asmecher]

Hi Ramón,

Picking up the conversation from here and from http://pkp.sfu.ca/support/forum/viewtopic.php?f=4&t=2986... Points 2 and 3 here are definitely on our list of important features. Could you give me a bit more description of point 1?

Also, any information on the slow query you reported in the other thread would be great.

The Harvester hasn't received much development attention for a while -- that's in part because one of the PKP team members who works with the harvester was busy with another project. However, he's returned to the team, and for other reasons we're also going to be looking at extending the harvester over the coming months, so it will be receiving more attention. In particular, I'll be working on some optimization for large collections.

Whereas OJS and OCS have very clear and specific uses, the Harvester is more of a toolkit, and we've seen it showing up in very unusual places where developers have found it useful. We don't get as much feedback on it as on our other applications, so any feedback here is very welcome.

The pendrive/VM approach is particularly intriguing. Could you describe this further? What software tools are you using?

Regards,
Alec Smecher
Public Knowledge Project Team

[ramon]

Hi Alec and all PKP Team,

First of all, glad to see the support team growing!!

We are evaluating a Zopen Archives Harvester product for Zope/Plone, which categorizes the content.
We are not sure how it works yet, because it won't install in newer versions of the CMS.
Here's a demo of the search, which apparently is run on the XML collected.

What we are assuming is it might use the ListSets to categorize content, but it may be implemented differently.

The idea behind categorizing content is to separate the types of repositories harvested: journals, conferences, digital libraries, institutional repositories, whatever classification people may want to come up with.

So, when viewing the list of repositories, you could have something like this example, which is a modification made in DSpace, with CSS+Javascript, when listing categories. If you click the "»" symbol, you'll open the list (useful when hierarchy is deep!).

[ramon]

Hi,

Try this link.
It's been a while, since I've been "reassigned" to other projects, I don't work with the harvester projects anymore.
I'm mostly working with our institution's new website.

However, I haven't seen any harvester do what DOAJ does.
I believe it's implemented with sets.

For journals, I still believe there's a need to define a specific metadata format for journals, with more fine-grain information such as general manager, editorial and review board members, total submissions per month /year, total published articles, where the journal is indexed, and whatever info may seem important to qualify journals (not to prevent them from being harvested, but to allow researches to verify journal quality, managers to identify what they can do to improve, etc...)