[LUCENE-7728] TestGrouping.testRandom() failures

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 7.3
• Lucene Fields: New
• Assignee: Alan Woodward
• Reporter: Steven Rowe
• Created: 02/Mar/17 03:30
• Updated: 28/Aug/22 15:11
• Resolved: 15/Mar/18 11:24

Description

My Jenkins found a reproducing branch_6x seed:

   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGrouping -Dtests.method=testRandom -Dtests.seed=6F0B519BBDC786B3 -Dtests.slow=true -Dtests.locale=en-CA -Dtests.timezone=Europe/Zagreb -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 2.50s J0 | TestGrouping.testRandom <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: expected:<10> but was:<29>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([6F0B519BBDC786B3:1D4774940CA730C0]:0)
   [junit4]    > 	at org.apache.lucene.search.grouping.TestGrouping.assertEquals(TestGrouping.java:1288)
   [junit4]    > 	at org.apache.lucene.search.grouping.TestGrouping.testRandom(TestGrouping.java:1130)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {groupend=Lucene50(blocksize=128), sort1=PostingsFormat(name=Memory doPackFST= true), sort2=Lucene50(blocksize=128), content=PostingsFormat(name=Memory doPackFST= false), group=PostingsFormat(name=Memory doPackFST= true)}, docValues:{groupend=DocValuesFormat(name=Direct), author=DocValuesFormat(name=Memory), sort1=DocValuesFormat(name=Memory), id=DocValuesFormat(name=Memory), sort2=DocValuesFormat(name=Direct), content=DocValuesFormat(name=Lucene54), group=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=454, maxMBSortInHeap=6.048552291438714, sim=RandomSimilarity(queryNorm=false,coord=no): {content=DFI(Saturated)}, locale=en-CA, timezone=Europe/Zagreb
   [junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 (64-bit)/cpus=16,threads=1,free=485264792,total=514850816

git bisect says the first bad commit is this one: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/6d952076, but that just removed BooleanSimilarity from the RandomSimilarity's candidates list, which in this case likely has the side effect of picking a different similarity with this seed. So the problem is almost certainly older than this.

Policeman Jenkins reported a master seed four weeks ago https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18890/ for a failure that's still reproducing for me on Linux w/ Java8:

  [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGrouping -Dtests.method=testRandom -Dtests.seed=381A6A2BB3B21736 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=nyn -Dtests.timezone=Antarctica/McMurdo -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
  [junit4] FAILURE 8.72s J0 | TestGrouping.testRandom <<<
  [junit4]    > Throwable #1: java.lang.AssertionError: expected:<2526> but was:<1818>
  [junit4]    > 	at __randomizedtesting.SeedInfo.seed([381A6A2BB3B21736:4A564F2402D2A145]:0)
  [junit4]    > 	at org.apache.lucene.search.grouping.TestGrouping.assertEquals(TestGrouping.java:1299)
  [junit4]    > 	at org.apache.lucene.search.grouping.TestGrouping.testRandom(TestGrouping.java:1141)
  [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:543)
  [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:844)
  [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {groupend=PostingsFormat(name=Direct), sort1=PostingsFormat(name=LuceneVarGapFixedInterval), sort2=PostingsFormat(name=Direct), content=PostingsFormat(name=LuceneVarGapDocFreqInterval), group=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{groupend=DocValuesFormat(name=Lucene70), author=DocValuesFormat(name=Memory), sort1=DocValuesFormat(name=Memory), id=DocValuesFormat(name=Memory), sort2=DocValuesFormat(name=Lucene70), content=DocValuesFormat(name=Direct), group=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1292, maxMBSortInHeap=6.668104559353747, sim=RandomSimilarity(queryNorm=true): {content=DFI(Standardized)}, locale=nyn, timezone=Antarctica/McMurdo
  [junit4]   2> NOTE: Linux 4.4.0-53-generic i386/Oracle Corporation 9-ea (32-bit)/cpus=12,threads=1,free=23290544,total=79691776

git bisect points to the exact same first bad commit for this seed too (see above).