[SOLR-3944] OOM Causes IllegalStateException on Subsequent Commits Until Restart

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.0
• Component/s: contrib - Solr Cell (Tika extraction)
• Labels: IllegalStateException JavaHeap OutOfMemoryError Tika

Environment

Fedora 16; Tomcat 7.0.27; Solr 4.0

• Assignee: Unassigned
• Reporter: Peter Hancox
• Created: 14/Oct/12 22:56
• Updated: 14/Oct/12 22:56

Description

Using curl to post documents for indexing via Solr Cell. Some large documents (~1GByte) and often database dump files (text SQL) cause "OutOfMemoryError: Java heap space". This would also occur on SOLR 3.5

However, in 4.0 all subsequent commit attempts throw the following exception until Tomcat is restarted.

null:java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot flush

Can live with some documents throwing OOM and not being indexed but cannot live with nothing being committed until Tomcat restart.

Based on my very basic understanding, it would appear that in SOLR 4.0, the initial OOM is putting SOLR into an unrecoverable state with respect to java heap memory for which an application restart is the only possible solution?