OJS OCS OMP OHS

You are viewing the PKP Support Forum | PKP Home Wiki



Lucene issue

Are you responsible for making OJS work -- installing, upgrading, migrating or troubleshooting? Do you think you've found a bug? Post in this forum.

Moderators: jmacgreg, btbell, michael, bdgregg, barbarah, asmecher

Forum rules
What to do if you have a technical problem with OJS:

1. Search the forum. You can do this from the Advanced Search Page or from our Google Custom Search, which will search the entire PKP site. If you are encountering an error, we especially recommend searching the forum for said error.

2. Check the FAQ to see if your question or error has already been resolved.

3. Post a question, but please, only after trying the above two solutions. If it's a workflow or usability question you should probably post to the OJS Editorial Support and Discussion subforum; if you have a development question, try the OJS Development subforum.

Lucene issue

Postby piotreba » Thu Nov 28, 2013 2:44 pm

Hello,

I have problems with starting solr. First of all, what should be folder permissions to properly run solr? When I try to start solr I get the following message: "We did not find the location of the OJS files directory or the files directory is not writable." My files folder is writtable, although it is outside the OJS installation (in the hidden from the public access area).


Thanks,
Peter
piotreba
 
Posts: 157
Joined: Mon Dec 13, 2010 3:29 pm

Re: Lucene issue

Postby swing » Mon Dec 02, 2013 5:21 am

Hi Peter,

The OJS files directory, the lucene folder inside of it and all subfolders and files therein should be readable and writable for the user that runs Solr. Also, always start and stop the Solr as/with the same user.
How do you start your Solr -- as which user? Are all the folders and files readable and writable for that user? Is you path to the OJS file directory in your config.inc.php absolute and correct?

Best,
Bozana Bokan
swing
 
Posts: 144
Joined: Tue Oct 09, 2007 2:59 am

Re: Lucene issue

Postby piotreba » Wed May 07, 2014 2:40 pm

My files directory resides outside of the OJS installation, but the folder is writtable. What you mean by "lucene folder inside of it"? Should I have a "lucene" folder inside my files directory? Path to the files directory is absolute and correct as all files are properly placed there and read.

The error results from running the chkconfig.sh script.
piotreba
 
Posts: 157
Joined: Mon Dec 13, 2010 3:29 pm

Re: Lucene issue

Postby swing » Thu May 08, 2014 3:33 am

Hi Peter,

Ah OK, the check if the files directory exists and is writable is done first, and because it fails the other checks are not done/executed.

OK, lets go step by step, to find where is the problem:
Could you please insert the variable '$OJS_FILES' in the error message in the file plugins/generic/lucene/embedded/bin/script-startup, e.g. like this:
"We did not find the location of the OJS files directory $OJS_FILES or the files directory is not writable."
Then the files directory should be displayed in the error message and you could check if this is the right/correct files directory -- eventually there is a problem when the script tries to read the files directory from the file 'config.inc.php'.
It doesn't matter if the files directory is outside the OJS directory -- this is surely not the problem.
If the displayed file directory is correct, then probably something is wrong with the permissions. The directory should be writable for the user that executes the script 'chkconfig.sh' and that starts and stops Solr. It seems to be writable for the Web user, because your journal files seem to be properly placed there, but it also has to be writable for the user executing the Solr scripts.

Best,
Bozana Bokan
swing
 
Posts: 144
Joined: Tue Oct 09, 2007 2:59 am

Re: Lucene issue

Postby piotreba » Thu May 08, 2014 7:53 am

Ok, I put the variable into the script-startup and I got the same path in the error message as in the config.inc.php.
The "files" directory has "777" permissions set up.
piotreba
 
Posts: 157
Joined: Mon Dec 13, 2010 3:29 pm

Re: Lucene issue

Postby swing » Thu May 08, 2014 9:15 am

Phew... Seem to be a complicated case/issue/problem... -- I don't understand it... ;-)
Could you please as the same user manually execute exactly the following command:
mkdir $OJS_FILES/lucene
where $OJS_FILES is the path to your "files" directory displayed in the error message, i.e. from your config.inc.php?
This should create a new directory called 'lucene' in your "files" directory.
Does this work?
swing
 
Posts: 144
Joined: Tue Oct 09, 2007 2:59 am

Re: Lucene issue

Postby piotreba » Thu May 08, 2014 9:29 am

yes, I was able to successfully execute this command...
piotreba
 
Posts: 157
Joined: Mon Dec 13, 2010 3:29 pm

Re: Lucene issue

Postby swing » Thu May 08, 2014 9:38 am

Could you please give write permissions to the new directory "lucene" for the user executing the scripts and starting and stopping Solr?
Could you then please comment out the following part of the script "script-startup" that checks if the "files" directory can be found and is writable:
Code: Select all
if [ ! \( -d "$OJS_FILES" -a -w "$OJS_FILES" \) ]; then
  echo "We did not find the location of the OJS files directory or the files directory is not writable."
  exit 1
fi

Could you after that please try to execute the script "chkconfig.sh" again?
swing
 
Posts: 144
Joined: Tue Oct 09, 2007 2:59 am

Re: Lucene issue

Postby piotreba » Thu May 08, 2014 9:58 am

Ok, I did it and ended up with the following message:

mkdir: cannot create directory `"path_to_the_files_dir"/lucene': No such file or directory
Everything ok. You should be able to start Solr now.
piotreba
 
Posts: 157
Joined: Mon Dec 13, 2010 3:29 pm

Re: Lucene issue

Postby piotreba » Thu May 08, 2014 10:10 am

Ok I got it...


The path to the files dir in the config.inc.php was placed between " "...

I have no idea why, supposingly it was taken from the earlier configuration files. I removed quotation marks and now the solr has started...

But my OJS has been working without (visible) issues even though the path was placed between quotation marks.
piotreba
 
Posts: 157
Joined: Mon Dec 13, 2010 3:29 pm

Re: Lucene issue

Postby piotreba » Thu May 08, 2014 10:34 am

Now I have experienced further issues.

I started php rebuildSearchIndex.php

And I got the following messages:

Code: Select all
The Lucene web service returned a status code 404 and the message
<?xml version="1.0"?>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 404 NOT_FOUND</title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/ojs/dih. Reason:
<pre>    NOT_FOUND</pre></p>
<hr/><i><small>Powered by Jetty://</small></i>



Code: Select all
LucenePlugin: Clearing index ... ojs2 - Lucene plugin:
The Lucene web service returned a status code 404 and the message
<?xml version="1.0"?>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 404 NOT_FOUND</title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/ojs/update. Reason:
<pre>    NOT_FOUND</pre></p>
<hr/><i><small>Powered by Jetty://</small></i>
piotreba
 
Posts: 157
Joined: Mon Dec 13, 2010 3:29 pm

Re: Lucene issue

Postby piotreba » Thu May 08, 2014 11:52 am

Ok, I have reinstalled solr and jetty and the lucene search engine seems to work now, well parially...


When I search for something from the PDF (i.e. full text) I have no results at all. So it seems PDFs are not being indexed. Any hints?



I can only see the "LucenePlugin" string in one line when executing rebuildSearchIndex:
LucenePlugin: Clearing index ... done

The string is not present next to the journals name (so I can see Indexing "journal name" ... 1000 articles indexed), as it is stated in the README file of the plugin. But the search works, I have suggestions in the search input and the search results appears.
piotreba
 
Posts: 157
Joined: Mon Dec 13, 2010 3:29 pm

Re: Lucene issue

Postby swing » Thu May 08, 2014 1:11 pm

Ah, glad it worked :) Somehow/Partially :)

That you see "LucenePlugin" only once is correct. -- I'll have to change either the output or the README, sorry!

PDFs should actually also be indexed :(
Could you delete the cache and try again?
Else, you would maybe need to take a look in the index and/or in the log file $OJS_FILES/lucene/solr-java.log...
swing
 
Posts: 144
Joined: Tue Oct 09, 2007 2:59 am

Re: Lucene issue

Postby piotreba » Thu May 08, 2014 2:00 pm

I cleared the cache and rebuilt search index, but no change.


The solr-java.log is preety large. I found example entry, that may indicate there is something wrong with processing galleys:

Code: Select all
The May 8, 2014 10:47:50 PM org.apache.solr.handler.dataimport.EntityProcessorWrapper pullRow
SEVERE: Exception in entity : galley_file
org.apache.solr.handler.dataimport.DataImportHandlerException: Exception in invoking url https://****/article/download/3346/2859 Processing Document # 133
   at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)


I am not sure where the above message ends.

At this point I would like to ask if the base_url should start with https if I force ojs to work entirely through ssl?
piotreba
 
Posts: 157
Joined: Mon Dec 13, 2010 3:29 pm

Re: Lucene issue

Postby swing » Thu May 22, 2014 2:56 am

Hi Peter,

Sorry for not responding for so long -- I've hoped to be able to test it with SSL, but I haven't had any time yet :-(
The SSL could be the problem. I don't think the "https://" is necessary for the "base_url" in "config.inc.php". Could you maybe test it without it? If this doesn't solve the problem, would it be possible for you to test the "rebuildSearchIndex.php" without using the SSL for everything?

Thanks!
Bozana
swing
 
Posts: 144
Joined: Tue Oct 09, 2007 2:59 am

Next

Return to OJS Technical Support

Who is online

Users browsing this forum: Yahoo [Bot] and 3 guests