OJS OCS OMP OHS

You are viewing the PKP Support Forum | PKP Home Wiki



OJS 2.4.4 poor search performance even with Lucene

OJS development discussion, enhancement requests, third-party patches and plug-ins.

Moderators: jmacgreg, btbell, michael, bdgregg, barbarah, asmecher

Forum rules
Developer Resources:

Documentation: The OJS Technical Reference and the OJS API Reference are both available from the OJS Documentation page.

Git: You can access our public Git Repository here. Comprehensive Git usage instructions are available on the wiki.

Bugzilla: You can access our Bugzilla report tracker here.

Search: You can use our Google Custom Search to search across our main website, the support forum, and Bugzilla.

Questions and discussion are welcome, but if you have a workflow or usability question you should probably post to the OJS Editorial Support and Discussion subforum; if you have a technical support question, try the OJS Technical Support subforum.

OJS 2.4.4 poor search performance even with Lucene

Postby rmchugh » Thu Jun 26, 2014 7:10 am

Hi,

I've installed the Lucene plugin on a test version of OJS and indexed about 40,000 docs. Unfortunately, search speed is still not as good as I would expect. I've performed some benchmarks where I've performed the same search in both the OJS interface and directly in Solr and the problem seems to be on the OJS side of thing.

The following table shows a comparison of searches in OJS and in Solr. The first column is the search term, the second the number of results, while the third and fourth are the time in milliseconds taken in Solr and OJS respectively. For OJS I've taken the time from the Network panel of the Firefox Developer tools, while for Solr I've taken it from the response header.

Term | Results | Solr | OJS
------------|---------|------|-----
Kolding | 1514| 372 | 3108
Herning | 1022 | 298 | 2699
gallagher | 16 | 474 | 2605
strik | 66 | 880 | 4038
grønland | 1439 | 797 | 2714
samfund | 5180 | 839 | 2792
rasmussen | 3383 | 190 | 1914



Solr is takes on average 550 ms to carry out a search, while OJS take on average 2840 ms to do the same search, i.e. an additional 2300 ms of processing. So I'm wondering, what is OJS spending all this time on? One answer is that it's doing a hell of a lot of database queries. If I turn debug to On in my config, I can see that on average 2130 SELECT queries are being done for each of the searches. This seems like a huge number! About 1786 of these consist of direct queries to section_settings in the form "SELECT * FROM section_settings WHERE section_id = X". So it seems that these are all being called one after another, after the "SELECT * FROM sections ORDER BY journal_id, seq" query.

Therefore I would like to ask, what is the purpose of these queries? Can they be removed without incident? If not, can they be optimised in the form "SELECT * SETTINGS section_settings WHERE section_id IN (x,y,z)"? Finally, am I missing something else that can be easily optimised to improve OJS search performance?

Many thanks.
rmchugh
 
Posts: 14
Joined: Fri Sep 06, 2013 3:31 am

Re: OJS 2.4.4 poor search performance even with Lucene

Postby rmchugh » Thu Jun 26, 2014 7:54 am

To answer my own question - I think I have found the problem. The issue seems to be with the Custom Ranking setting on the Lucene plugin. If Custom Ranking is enabled (as it seems to be per default) the Lucene plugin will go through each and every section to see if it has a custom ranking and then apply this. (See LucenePlugin.inc.php)

Turning off this setting saves on average 1351 ms per query based on the set of queries I supplied above. The number of SELECTS per page comes down to about 400 per page, still a lot, but not completely insane.

I would recommend that the Lucene Plugin be updated so that custom ranking is not applied per default and that the documentation is updated to strongly advise against using it. If PKP developers agree I would be happy to create a pull request for this.
rmchugh
 
Posts: 14
Joined: Fri Sep 06, 2013 3:31 am

Re: OJS 2.4.4 poor search performance even with Lucene

Postby swing » Thu Jun 26, 2014 12:53 pm

Hi rmchugh,

Yes, please go ahead :-)

Best,
Bozana Bokan
swing
 
Posts: 142
Joined: Tue Oct 09, 2007 2:59 am

Re: OJS 2.4.4 poor search performance even with Lucene

Postby rmchugh » Fri Jun 27, 2014 6:15 am

rmchugh
 
Posts: 14
Joined: Fri Sep 06, 2013 3:31 am


Return to OJS Development

Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 1 guest