OJS OCS OMP OHS

You are viewing the PKP Support Forum | PKP Home Wiki



Proxy problems with Harvester?

Open Harvester Systems support questions and answers, bug reports, and development issues.

Moderators: jmacgreg, michael, John

Forum rules
Developer Resources:

Git: You can access our public Git Repository here. Comprehensive Git usage instructions are available on the wiki.

Bugzilla: You can access our Bugzilla report tracker here.

Search: You can use our Google Custom Search to search across our main website, the support forum, and Bugzilla.

Questions and discussion are welcome.

Re: Proxy problems with Harvester?

Postby asmecher » Wed Oct 17, 2007 10:43 am

Hi Fred,

Glad to hear it's debugged, if not resolved yet. We'll be working on the platform code that OJS, OCS, and Harvester all use, and one of the things we'll be paying attention to is better error logging. In the meantime, good luck with the Inquisition...

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 8419
Joined: Wed Aug 10, 2005 12:56 pm

Re: Proxy problems with Harvester?

Postby fredriley » Thu Oct 18, 2007 6:23 am

Alec

I've written a long justification to our sysadmins for opening port 3128 to allow Harvester to work. Before I send it off, I'd be grateful if you could briefly answer 3 quick Qs which I'm sure they'll ask:

1. Does Harvester have to use port 3128, or can it use any HTTP port? (I have tried setting ports 80 and 8080 in the config.inc.php proxy section but no dice.)
2. Is the traffic on the port both outgoing and incoming?
3. Are any other ports used by Harvester? Other than 80 for normal web traffic, that is.

Cheers

Fred
fredriley
 
Posts: 27
Joined: Fri Sep 14, 2007 10:47 am

Re: Proxy problems with Harvester?

Postby asmecher » Thu Oct 18, 2007 9:31 am

Hi Fred,

1. Harvester can be configured to use a proxy on any port.
2. Harvester uses standard protocol for HTTP proxy requests, i.e. all indexing requests are initiated by Harvester. No incoming connections need be accepted.
3. No, nothing else is used.

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 8419
Joined: Wed Aug 10, 2005 12:56 pm

Re: Proxy problems with Harvester?

Postby fredriley » Thu Oct 18, 2007 10:27 am

asmecher wrote:Hi Fred,

1. Harvester can be configured to use a proxy on any port.


Including port 80? So if I were to use the following in config.inc.php Harvester should work?

; The HTTP proxy configuration to use
http_host = 128.243.220.20
http_port = 80

I did give that a bash but no luck. I also tried just specifying the proxy host without a port, also no good. By default, what port do Harvester requests go out on? Questions, questions... :cry:

Cheers

Fred
fredriley
 
Posts: 27
Joined: Fri Sep 14, 2007 10:47 am

Re: Proxy problems with Harvester?

Postby asmecher » Thu Oct 18, 2007 10:45 am

Hi Fred,

If this is a machine that you can run a web browser on (even a text-mode one like Lynx), I'd suggest hammering out your proxy settings using that, as it'll be quicker to test with and less of an unknown app than Harvester. Once you've got the proxy settings figured out for a browser, use the same settings for Harvester.

OAI requests use HTTP as the underlying protocol, so the technology is exactly the same.

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 8419
Joined: Wed Aug 10, 2005 12:56 pm

Re: Proxy problems with Harvester?

Postby fredriley » Mon Oct 22, 2007 3:54 am

asmecher wrote:Hi Fred,

If this is a machine that you can run a web browser on (even a text-mode one like Lynx), I'd suggest hammering out your proxy settings using that, as it'll be quicker to test with and less of an unknown app than Harvester. Once you've got the proxy settings figured out for a browser, use the same settings for Harvester.

OAI requests use HTTP as the underlying protocol, so the technology is exactly the same.

Regards,
Alec Smecher
Public Knowledge Project Team


Hmm. That's got me scratching my head again. The machine on which I've installed Harvester is our university's main multi-user Unix system, which runs Lynx, Firefox (in X Windows), and even Mosaic (a blast from the past!). What you're saying is that if they work then Harvester should work, as Harvester can also use port 80. In which case I'm absolutely stumped as to why it won't work. This is getting messy with all the recent tweaks - I'll try installing it clean on another account on the same machine.

Cheers

Fred
fredriley
 
Posts: 27
Joined: Fri Sep 14, 2007 10:47 am

Re: Proxy problems with Harvester?

Postby asmecher » Mon Oct 22, 2007 9:16 am

Hi Fred,

What proxy settings are the various browsers using? Are they connecting directly via port 80 (e.g. a direct connection to the remote server), or is there a proxy configuration that accesses the proxy using port 80?

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 8419
Joined: Wed Aug 10, 2005 12:56 pm

Re: Proxy problems with Harvester?

Postby fredriley » Wed Oct 24, 2007 9:48 am

asmecher wrote:Hi Fred,

What proxy settings are the various browsers using? Are they connecting directly via port 80 (e.g. a direct connection to the remote server), or is there a proxy configuration that accesses the proxy using port 80?


I've looked in the connection settings of Firefox on the Unix system, and it uses the proxy config scrip at http://wwwcache.nottingham.ac.uk/proxy.pac, the same as the copy of Firefox on my local machine.

Having failed completely in getting Harvester to run on the multi-user system, I installed it on a Windows PC in our office running Apache, mySQL and PHP (courtesy of the excellent XAMPP package) with all root permissions. The installation went fine, then after I edited the config.inc.php to add:

http_host = 128.243.220.20
http_port = 3128

to the [proxy] section, it harvested records fine. Woo-hoo! I thought, at last I've got it working on a server which others can access. Sadly, my enthusiasm was dampened when I found that, although anyone inside our nottingham.ac.uk domain could access it at http://128.243.175.211/harvester2/, nobody outside the domain could get to it - if you give it go it'll time out, and neither can 128.243.175.211 be pinged. Almost certainly our sodding sysadmins have blocked incoming HTTP to anything other than 'approved' IP addresses, so I've sent them a sharp email asking them to 'approve' the test machine.

Put simply, I think that all this trouble has been caused by some mysterious settings on our university web server which I'm not privy to. It frustrates the almighty hell out of me, and means that I'll probably have to try to install harvester on my personal machine at home in order to demo it to work colleagues.

Has the pkp.sfu.ca server been up and down these last couple of days? It's been unreachable for much of today, and a fair bit of yesterday.

Cheers

Fred
fredriley
 
Posts: 27
Joined: Fri Sep 14, 2007 10:47 am

Re: Proxy problems with Harvester?

Postby asmecher » Wed Oct 24, 2007 10:23 am

Hi Fred,

Sorry to hear it's been a frustrating experience, but at least we've confirmed that proxy support for the Harvester does work. You might try downloading that proxy configuration file and looking through it by hand -- but make sure you fetch it from the server itself, e.g. from a SSH session using "wget", as it's probably dynamically generated. (For example, I can download it from here, and it recommends using a direct connection to the Internet; someone on an internal subnet will get different contents if they fetch the same file, depending on where they're coming from.)

We've had some power issues on campus that caused some downtime for pkp.sfu.ca; hopefully it'll be smooth sailing going forward.

Regards,
Alec Smecher
Public Knowledge Project Team
asmecher
 
Posts: 8419
Joined: Wed Aug 10, 2005 12:56 pm

Previous

Return to Open Harvester Systems Support and Development

Who is online

Users browsing this forum: No registered users and 1 guest

cron