[SOLR-7927] Indexing large documents requires larger heap than may be necessary

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.2.1
• Fix Version/s: 5.5, 6.0
• Component/s: update
• Assignee: Unassigned
• Reporter: Shalin Shekhar Mangar
• Created: 14/Aug/15 14:59
• Updated: 09/May/16 18:55

Description

Solr is started with 1280M heap.

./bin/solr start -m 1280m

Indexing a 100MB JSON file (using curl) containing large JSON documents from project Gutenberg fails with OOM but indexing a 549M JSON file containing small documents is indexed just fine.

The same 100MB JSON file with the same heap size can be indexed just fine if I disable the transaction log.

Issue Links

• requires: LUCENE-6779 Reduce memory allocated by CompressingStoredFieldsWriter to write large strings

Sub-Tasks

• Reduce memory allocated by JavaBinCodec to encode large strings: Closed