[SOLR-11078] Solr query performance degradation since Solr 6.4.2

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 6.6, 7.1
• Component/s: search, Server

Environment

• CentOS 7.3 (Linux zasolrm03 3.10.0-514.26.2.el7.x86_64 #1 SMP Tue Jul 4 15:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux)
• Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
• 4 CPU, 10GB RAM

Running Solr 6.6.0 with the following JVM settings:
java -server -Xms4G -Xmx4G -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/prodza/solrserver/../logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.log.dir=/home/prodza/solrserver/../logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=SAST -Djetty.home=/home/prodza/solrserver/server -Dsolr.solr.home=/home/prodza/solrserver/../solr -Dsolr.install.dir=/home/prodza/solrserver -Dlog4j.configuration=file:/home/prodza/solrserver/../config/log4j.properties -Xss256k -Xss256k -Dsolr.log.muteconsole -XX:OnOutOfMemoryError=/home/prodza/solrserver/bin/oom_solr.sh 8983 /home/prodza/solrserver/../logs -jar start.jar --module=http

• Assignee: Unassigned
• Reporter: bidorbuy
• Created: 14/Jul/17 06:31
• Updated: 08/Jun/19 15:24
• Resolved: 08/Aug/18 20:24

Description

We are currently running 2 separate Solr servers - refer to screenshots:

• zasolrm02 is running on Solr 6.4.2
• zasolrm03 is running on Solr 6.6.0

Both servers have the same OS / JVM configuration and are using their own indexes. We round-robin load-balance through our Tomcats and notice that Since Solr 6.4.2 performance has dropped. We have two indices per server "searchsuggestions" and "tradesearch". There is a noticeable drop in performance since Solr 6.4.2.

I am not sure if this is perhaps related to metric collation or other underlying changes. I am not sure if other high transaction users have noticed similar issues.

1) zasolrm03 (6.6.0) is almost twice as slow on the tradesearch index:

2) This is also visible in the searchsuggestion index:

3) The Tradesearch index shows the biggest difference:

Issue Links

• is related to: SOLR-12074 Add numeric typed equivalents to StrField