[SOLR-2493] SolrQueryParser constantly parse luceneMatchVersion in solrconfig. Large performance hit.

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.1
• Fix Version/s: 3.1.1, 3.2, 4.0-ALPHA
• Component/s: search
• Labels: core parser performance request solr
• Assignee: Uwe Schindler
• Reporter: Stephane Bailliez
• Created: 04/May/11 03:57
• Updated: 03/Jun/11 16:44
• Resolved: 04/May/11 07:18

Description

I' m putting this as blocker as I think this is a serious issue that should be adressed asap with a release. With the current code this is no way near suitable for production use.

For each instance created SolrQueryParser calls

getSchema().getSolrConfig().getLuceneVersion("luceneMatchVersion", Version.LUCENE_24)

instead of using

getSchema().getSolrConfig().luceneMatchVersion

This creates a massive performance hit. For each request, there is generally 3 query parsers created and each of them will parse the xml node in config which involve creating an instance of XPath and behind the scene the usual factory finder pattern quicks in within the xml parser and does a loadClass.

The stack is typically:

at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
at com.sun.org.apache.xml.internal.dtm.ObjectFactory.findProviderClass(ObjectFactory.java:506)
at com.sun.org.apache.xml.internal.dtm.ObjectFactory.lookUpFactoryClass(ObjectFactory.java:217)
at com.sun.org.apache.xml.internal.dtm.ObjectFactory.createObject(ObjectFactory.java:131)
at com.sun.org.apache.xml.internal.dtm.ObjectFactory.createObject(ObjectFactory.java:101)
at com.sun.org.apache.xml.internal.dtm.DTMManager.newInstance(DTMManager.java:135)
at com.sun.org.apache.xpath.internal.XPathContext.<init>(XPathContext.java:100)
at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.eval(XPathImpl.java:201)
at com.sun.org.apache.xpath.internal.jaxp.XPathImpl.evaluate(XPathImpl.java:275)
at org.apache.solr.core.Config.getNode(Config.java:230)
at org.apache.solr.core.Config.getVal(Config.java:256)
at org.apache.solr.core.Config.getLuceneVersion(Config.java:325)
at org.apache.solr.search.SolrQueryParser.<init>(SolrQueryParser.java:76)
at org.apache.solr.schema.IndexSchema.getSolrQueryParser(IndexSchema.java:277)

With the current 3.1 code, I do barely 250 qps with 16 concurrent users with a near empty index.

Switching SolrQueryParser to use getSchema().getSolrConfig().luceneMatchVersion and doing a quick bench test, performance become reasonable beyond 2000 qps.

Issue Links

• is broken by: LUCENE-2458 queryparser makes all CJK queries phrase queries regardless of analyzer