[SOLR-8582] /update/json/docs is 4x slower than /update for indexing a list of json docs

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.3.2, 5.4.1
• Fix Version/s: 5.5, 6.0
• Component/s: update
• Assignee: Noble Paul
• Reporter: Shalin Shekhar Mangar
• Created: 22/Jan/16 11:16
• Updated: 09/May/16 18:51
• Resolved: 23/Jan/16 18:58

Description

Indexing a ~650 MB json file containing a list of 2.2 million json documents, I found that bin/post had become 4x slower after SOLR-7042. Memory consumption has also gone up and I can no longer index this file with a 512mb heap.

The difference is because we now default to /update/json/docs instead of /update. This can be verified on trunk:

time curl 'http://localhost:8983/solr/gettingstarted/update' --data-binary @/hdd/solr-data/imdb.json 
{"responseHeader":{"status":0,"QTime":161869}}
​
real	2m42.044s
user	0m0.292s
sys	0m0.493s
​
time curl 'http://localhost:8983/solr/gettingstarted/update/json/docs' --data-binary @/hdd/solr-data/imdb.json 
{"responseHeader":{"status":0,"QTime":686264}}
​
real	11m26.478s
user	0m0.324s
sys	0m0.552s