Harvesting less records than real

[josipkp]

Hi,

We are trying to harvester Actas from DSpace at UMinho Portugal.
Browsing the DSpace community, there is 42 itens.
The same itens harvesting by hand:
http://repositorium.sdum.uminho.pt/oai/ ... l_1822_831

Using Harvester2 we just collect 3 records: the last 3 records visible in the hand harvesting.
What could be happen?

Thanks in advance,
Josi Perez

[asmecher]

Hi Josi,

Are the harvest from / to dates blank when you submit the harvest form, and are any specific sets selected?

Regards,
Alec Smecher
Public Knowledge Project Team

[josipkp]

Hi,

It was the first harvesting: no dates, but yes, we select 2 specific sets.
When the harvester didn't harvest all records expected using the web interface, we tried to harvester using harvester/tools.

The command used:
php harvest.php 50 flush verbose set=hdl_1822_831 set=hdl_1822_830

Harvesting just the set hdl_1822_830 results zero records, but there are around 50.
No success trying to divide the range:
php harvest.php 50 flush verbose set=hdl_1822_831 from=2000-01-01 (3 records)
php harvest.php 50 flush verbose set=hdl_1822_831 from=2000-01-01 until=2009-12-30 (3 records)

Changing the data granularity results error message.
Any suggestions?

Thank you for your support.
Josi Perez

[asmecher]

Hi Josi,

What version of the harvester are you using?

Regards,
Alec Smecher
Public Knowledge Project Team

[josipkp]

Hi,

Using Harvester2 version 2.3.0.0

Thank you for your efforts.
Josi Perez

[asmecher]

Hi Josi,

I managed to harvest 70 records from the two sets. Is it possible that you have several records from different archives that share the same identifier?

Regards,
Alec Smecher
Public Knowledge Project Team

[josipkp]

Gotcha!
We decided to divide DSpace at UMinho in more than one set, creating categories (Thesis, Journals, Books and so) and I kept the entire Uminho while not harvested each category.

Thank you,
Josi Perez

[josipkp]

Hi,

I need a help to harvester a journal from SCielo.
I received the following error message when collecting from tools/harvester.php:
Errors/Warnings:
Missing or empty metadataPrefix

I tried to divide in small pieces of time and observed that the error occurs when the interval of time have more than 30 records, the number send by SCielo on each page.
For example:
http://www.scielo.br/oai/scielo-oai.php ... 2009-01-01
is the first page and have resumptionToken.
Executing the command manually in a web page we have response:
http://www.scielo.br/oai/scielo-oai.php ... 2009-01-01
but these records I can not found on Harvester2. On each interval I can just to collect 30 records.

Can you help me to discover the solution?

Thanks in advance,
Josi Perez
Univerciencia.org

[otuften]

Hi,
it seems to me to be an error in the OAI gateway scielo-oai.php.
The script demands metadataPrefix in addition to resumptionToken, but the OAI guidelines say that resumptionToken is an exclusive argument and should thus be the only in addition to the verb.
So PKP harvester sends only verb=ListRecords and resumptionToken=..., which should be accepted.

Regards
Olav