[SOLR-13694] IndexSizeEstimator NullPointerException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 8.3
• Assignee: Andrzej Bialecki
• Reporter: Chris M. Hostetter
• Created: 13/Aug/19 18:01
• Updated: 21/Jan/20 16:58
• Resolved: 16/Aug/19 08:52

Description

Jenkins found a reproducible seed for trigging an NPE in IndexSizeEstimatorTest

Based on a little experimental tracing i did, this might be a real bug in IndexSizeEstimator? ... it's calling close on StoredFieldsReader instances it gets from the CodecReader – but AFAICT from the docs/code i'm not certain if it should be doing this. It appears the expectation is that this is direct access to the internal state, that will automatically be closed when the CodecReader is closed.

ie: IndexSizeEstimator is closing StoredFieldsReader pre-maturely, causing it to be unusbale on the next iteration.

(I didn't dig in far enough to guess if there are other places in the IndexSizeEstimator code that are closing CodecReader internals prematurely as well, or just in this situation ... it's also not clear if this only causes failures because this seed uses SimpleTextCodec, and other codecs are more forgiving – or if something else about the index(es) generated for this seed are what cause the problem to manifest)

http://fucit.org/solr-jenkins-reports/job-data/apache/Lucene-Solr-NightlyTests-master/1928

hossman@tray:~/lucene/dev/solr/core [j11] [master] $ git rev-parse HEAD
0291db44bc8e092f7cb2f577f0ac8ab6fa6a5fd7
hossman@tray:~/lucene/dev/solr/core [j11] [master] $ ant test  -Dtestcase=IndexSizeEstimatorTest -Dtests.method=testEstimator -Dtests.seed=23F60434E13D8FD4 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true  -Dtests.locale=eo -Dtests.timezone=Atlantic/Madeira -Dtests.asserts=true -Dtests.file.encoding=UTF-8
...
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=IndexSizeEstimatorTest -Dtests.method=testEstimator -Dtests.seed=23F60434E13D8FD4 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=eo -Dtests.timezone=Atlantic/Madeira -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.88s | IndexSizeEstimatorTest.testEstimator <<<
   [junit4]    > Throwable #1: java.lang.NullPointerException
   [junit4]    >        at __randomizedtesting.SeedInfo.seed([23F60434E13D8FD4:EC2B6B666D451E64]:0)
   [junit4]    >        at org.apache.lucene.codecs.simpletext.SimpleTextStoredFieldsReader.visitDocument(SimpleTextStoredFieldsReader.java:109)
   [junit4]    >        at org.apache.solr.handler.admin.IndexSizeEstimator.estimateStoredFields(IndexSizeEstimator.java:513)
   [junit4]    >        at org.apache.solr.handler.admin.IndexSizeEstimator.estimate(IndexSizeEstimator.java:198)
   [junit4]    >        at org.apache.solr.handler.admin.IndexSizeEstimatorTest.testEstimator(IndexSizeEstimatorTest.java:117)
   [junit4]    >        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    >        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    >        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    >        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   [junit4]    >        at java.base/java.lang.Thread.run(Thread.java:834)

Issue Links

• is caused by: SOLR-13512 Raw index data analysis tool