TestMatchAllDocsQuery#testEarlyTermination fails total hits assertion

Very rarely TestMatchAllDocsQuery can fail with an assertion error:

   >     java.lang.AssertionError: expected:<201> but was:<241>
   >         at __randomizedtesting.SeedInfo.seed([B4F2350E137110E3:7B6574512B1D80E0]:0)
   >         at org.junit.Assert.fail(Assert.java:89)
   >         at org.junit.Assert.failNotEquals(Assert.java:835)
   >         at org.junit.Assert.assertEquals(Assert.java:647)
   >         at org.junit.Assert.assertEquals(Assert.java:633)
   >         at org.apache.lucene.search.TestMatchAllDocsQuery.testEarlyTermination(TestMatchAllDocsQuery.java:124)

We are expecting the totalHits.value to be exactly totalHitsThreshold + 1, but it is sometimes more:

assertEquals(totalHitsThreshold + 1, topDocs.totalHits.value);
assertEquals(TotalHits.Relation.GREATER_THAN_OR_EQUAL_TO, topDocs.totalHits.relation);

The failures only seem to happen when we choose to use multiple threads in IndexSearcher. I was able to reproduce this regularly by forcing IndexSearcher to use multiple threads and running the test 100 times. This started failing after https://github.com/apache/lucene/pull/716, where we updated tests to use a collector manager instead of a collector, which introduced the possibility of multiple threads.

I am not sure this actually indicates a bug, or if the test just needs to be tweaked. It seems okay to return more than totalHitsThreshold + 1 sometimes, that doesn't seem to violate any contract?