[LUCENE-2497] Revision 949509 (LUCENE-2480) causes IOE "read past EOF" when processing older format SegmentInfo data when JVM assertion processing is disabled.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0-ALPHA
• Fix Version/s: 4.0-ALPHA
• Component/s: core/index
• Labels: IOE index

Environment

Any JVM running without assertions enabled (such as Tomcat by default)

• Lucene Fields: New
• Assignee: Shai Erera
• Reporter: Kevin Dana
• Created: 12/Jun/10 00:46
• Updated: 28/Aug/22 12:27
• Resolved: 12/Jun/10 10:59

Description

At revision 949509 in org.apache.lucene.index.SegmentInfo at line 155, there is the following code:

 
    if (format > SegmentInfos.FORMAT_4_0) {
      // pre-4.0 indexes write a byte if there is a single norms file
      assert 1 == input.readByte();
    }

Note that the assert statement invokes input.readByte().
If asserts are disabled for the JVM, input.readByte() will not be invoked, causing the following readInt() to return a bogus value, and then causing an IOE during the (mistakenly entered) loop at line 165.
This can occur when processing old format (format "-9") index data under Tomcat (whose startup scripts by default do not turn on asserts).

Full stacktrace:

 
SEVERE: java.lang.RuntimeException: java.io.IOException: read past EOF
	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1066)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:581)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:431)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:286)
	at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:125)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:86)
	at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
	at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:397)
	at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:108)
	at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3800)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4450)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:850)
	at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:724)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:493)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
	at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
	at org.apache.catalina.core.StandardService.start(StandardService.java:516)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.io.IOException: read past EOF
	at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:154)
	at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
	at org.apache.lucene.store.ChecksumIndexInput.readByte(ChecksumIndexInput.java:40)
	at org.apache.lucene.store.DataInput.readInt(DataInput.java:76)
	at org.apache.lucene.store.DataInput.readLong(DataInput.java:99)
	at org.apache.lucene.index.SegmentInfo.<init>(SegmentInfo.java:165)
	at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:230)
	at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:91)
	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:649)
	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:87)
	at org.apache.lucene.index.IndexReader.open(IndexReader.java:415)
	at org.apache.lucene.index.IndexReader.open(IndexReader.java:294)
	at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38)
	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1055)
	... 32 more