[LUCENE-772] Index corruption can cause infinite spin loop when Lucene attempts to incorrectly uncompress fields

• Type: Bug
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.1
• Fix Version/s: 3.6.3
• Component/s: core/search

Environment

Linux (gentoo 2.6.17 at least), jdk 1.5.0_06 and _08 at least, tomcat 5.5. IndexSearcher searching index mounted via NFS; using new lockless commits... We're using the 01-05-07 nightly build of lucene

• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Arthur Smith
• Created: 12/Jan/07 21:30
• Updated: 28/Aug/22 11:33

Description

Unfortunately I don't have a reproducible case of this (yet), but it's happened twice now on our production servers in the past few days, after we switched to the new lockless commits (thanks!). What we're seeing is the search thread running away in the middle of a seemingly ordinary search, after several hundred thousand queries that worked just fine. The search index is NFS mounted, and is updated every minute or so during the day by an indexing process running on a separate server. We do get occasional I/O errors, but we catch and retry and it seems ok after a few seconds.

But twice now we've had run-away threads; the thread dump in both cases was caught in the middle of java.util.zip.Inflater:

"http-8080-3" daemon prio=1 tid=0x08294688 nid=0x1f0d runnable [0x1f17c000..0x1f17e0b0]
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:215)

• locked <0x3d73cba8> (a java.util.zip.Inflater)
  at java.util.zip.Inflater.inflate(Inflater.java:232)
  at org.apache.lucene.index.FieldsReader.uncompress(FieldsReader.java:388)
  at org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:222)
  at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:105)
  at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:324)
• locked <0x3cefbdd8> (a org.apache.lucene.index.SegmentReader) at org.apache.lucene.index.MultiReader.document(MultiReader.java:108)
  at org.apache.lucene.index.IndexReader.document(IndexReader.java:360) at org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:84)
  at org.apache.lucene.search.Hits.doc(Hits.java:104)

[...]

Any ideas what this could be? Thanks!