Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.5, 4.0-ALPHA
    • Fix Version/s: 3.5, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      didn't check 3.x yet, just encountered this one running the tests

      1. LUCENE-3528.patch
        0.9 kB
        Simon Willnauer
      2. LUCENE-3528.patch
        2 kB
        Simon Willnauer
      3. LUCENE-3528.patch
        6 kB
        Simon Willnauer
      4. LUCENE-3528_reproduceHang.patch
        4 kB
        Simon Willnauer

        Activity

        Hide
        Robert Muir added a comment -
            [junit] 2011-10-24 14:28:25
            [junit] Full thread dump Java HotSpot(TM) 64-Bit Server VM (19.1-b02 mixed mode):
            [junit] 
            [junit] "Thread-2" daemon prio=10 tid=0x00007fe66005b800 nid=0x22d6 waiting on condition [0x00007fe66d854000]
            [junit]    java.lang.Thread.State: WAITING (parking)
            [junit] 	at sun.misc.Unsafe.park(Native Method)
            [junit] 	- parking to wait for  <0x00000000e0002120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
            [junit] 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
            [junit] 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
            [junit] 	at org.apache.lucene.index.NRTManager.waitOnGenCondition(NRTManager.java:251)
            [junit] 	at org.apache.lucene.index.NRTManager.waitForGeneration(NRTManager.java:232)
            [junit] 	at org.apache.lucene.index.NRTManager.waitForGeneration(NRTManager.java:196)
            [junit] 	at org.apache.lucene.index.TestNRTManager.addDocuments(TestNRTManager.java:95)
            [junit] 	at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1.run(ThreadedIndexingAndSearchingTestCase.java:223)
            [junit] 
            [junit] "NRT Reopen Thread" daemon prio=10 tid=0x00007fe660024800 nid=0x22d5 in Object.wait() [0x00007fe66d955000]
            [junit]    java.lang.Thread.State: TIMED_WAITING (on object monitor)
            [junit] 	at java.lang.Object.wait(Native Method)
            [junit] 	at java.lang.Object.wait(Object.java:443)
            [junit] 	at org.apache.lucene.index.NRTManagerReopenThread.run(NRTManagerReopenThread.java:162)
            [junit] 	- locked <0x00000000e0006000> (a org.apache.lucene.index.NRTManagerReopenThread)
            [junit] 
            [junit] "Low Memory Detector" daemon prio=10 tid=0x00007fe668001000 nid=0x22ba runnable [0x0000000000000000]
            [junit]    java.lang.Thread.State: RUNNABLE
            [junit] 
            [junit] "CompilerThread1" daemon prio=10 tid=0x0000000040f8b000 nid=0x22b7 waiting on condition [0x0000000000000000]
            [junit]    java.lang.Thread.State: RUNNABLE
            [junit] 
            [junit] "CompilerThread0" daemon prio=10 tid=0x0000000040f88000 nid=0x22b5 waiting on condition [0x0000000000000000]
            [junit]    java.lang.Thread.State: RUNNABLE
            [junit] 
            [junit] "Signal Dispatcher" daemon prio=10 tid=0x0000000040f86000 nid=0x22b2 waiting on condition [0x0000000000000000]
            [junit]    java.lang.Thread.State: RUNNABLE
            [junit] 
            [junit] "Finalizer" daemon prio=10 tid=0x0000000040f69000 nid=0x229e in Object.wait() [0x00007fe66e7d7000]
            [junit]    java.lang.Thread.State: WAITING (on object monitor)
            [junit] 	at java.lang.Object.wait(Native Method)
            [junit] 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
            [junit] 	- locked <0x00000000e0002528> (a java.lang.ref.ReferenceQueue$Lock)
            [junit] 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
            [junit] 	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
            [junit] 
            [junit] "Reference Handler" daemon prio=10 tid=0x0000000040f62000 nid=0x2298 in Object.wait() [0x00007fe66e8d8000]
            [junit]    java.lang.Thread.State: WAITING (on object monitor)
            [junit] 	at java.lang.Object.wait(Native Method)
            [junit] 	at java.lang.Object.wait(Object.java:485)
            [junit] 	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
            [junit] 	- locked <0x00000000e0006090> (a java.lang.ref.Reference$Lock)
            [junit] 
            [junit] "main" prio=10 tid=0x0000000040ef6000 nid=0x2240 in Object.wait() [0x00007fe673e7d000]
            [junit]    java.lang.Thread.State: WAITING (on object monitor)
            [junit] 	at java.lang.Object.wait(Native Method)
            [junit] 	- waiting on <0x00000000e0002090> (a org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1)
            [junit] 	at java.lang.Thread.join(Thread.java:1186)
            [junit] 	- locked <0x00000000e0002090> (a org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1)
            [junit] 	at java.lang.Thread.join(Thread.java:1239)
            [junit] 	at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:524)
            [junit] 	at org.apache.lucene.index.TestNRTManager.testNRTManager(TestNRTManager.java:37)
            [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
            [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
            [junit] 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
            [junit] 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
            [junit] 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
            [junit] 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
            [junit] 	at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
            [junit] 	at org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:610)
            [junit] 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
            [junit] 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
            [junit] 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
            [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149)
            [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51)
            [junit] 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
            [junit] 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
            [junit] 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
            [junit] 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
            [junit] 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
            [junit] 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
            [junit] 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
            [junit] 	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
            [junit] 	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
            [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
            [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
            [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:743)
            [junit] 
            [junit] "VM Thread" prio=10 tid=0x0000000040f5b000 nid=0x228b runnable 
            [junit] 
            [junit] "GC task thread#0 (ParallelGC)" prio=10 tid=0x0000000040f09000 nid=0x2256 runnable 
            [junit] 
            [junit] "GC task thread#1 (ParallelGC)" prio=10 tid=0x0000000040f0a800 nid=0x225a runnable 
            [junit] 
            [junit] "GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000040f0c800 nid=0x2260 runnable 
            [junit] 
            [junit] "GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000040f0e800 nid=0x2266 runnable 
            [junit] 
            [junit] "GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000040f10000 nid=0x226a runnable 
            [junit] 
            [junit] "GC task thread#5 (ParallelGC)" prio=10 tid=0x0000000040f12000 nid=0x226f runnable 
            [junit] 
            [junit] "GC task thread#6 (ParallelGC)" prio=10 tid=0x0000000040f14000 nid=0x2273 runnable 
            [junit] 
            [junit] "GC task thread#7 (ParallelGC)" prio=10 tid=0x0000000040f15800 nid=0x2276 runnable 
            [junit] 
            [junit] "VM Periodic Task Thread" prio=10 tid=0x00007fe668004000 nid=0x22bc waiting on condition 
            [junit] 
            [junit] JNI global references: 1282
            [junit] 
            [junit] Heap
            [junit]  PSYoungGen      total 37888K, used 18301K [0x00000000f5560000, 0x00000000f7d80000, 0x0000000100000000)
            [junit]   eden space 37824K, 48% used [0x00000000f5560000,0x00000000f672f780,0x00000000f7a50000)
            [junit]   from space 64K, 100% used [0x00000000f7b00000,0x00000000f7b10000,0x00000000f7b10000)
            [junit]   to   space 1536K, 0% used [0x00000000f7c00000,0x00000000f7c00000,0x00000000f7d80000)
            [junit]  PSOldGen        total 171392K, used 2193K [0x00000000e0000000, 0x00000000ea760000, 0x00000000f5560000)
            [junit]   object space 171392K, 1% used [0x00000000e0000000,0x00000000e0224610,0x00000000ea760000)
            [junit]  PSPermGen       total 21248K, used 11609K [0x00000000dae00000, 0x00000000dc2c0000, 0x00000000e0000000)
            [junit]   object space 21248K, 54% used [0x00000000dae00000,0x00000000db9565c8,0x00000000dc2c0000)
            [junit] 
        
        Show
        Robert Muir added a comment - [junit] 2011-10-24 14:28:25 [junit] Full thread dump Java HotSpot(TM) 64-Bit Server VM (19.1-b02 mixed mode): [junit] [junit] "Thread-2" daemon prio=10 tid=0x00007fe66005b800 nid=0x22d6 waiting on condition [0x00007fe66d854000] [junit] java.lang.Thread.State: WAITING (parking) [junit] at sun.misc.Unsafe.park(Native Method) [junit] - parking to wait for <0x00000000e0002120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) [junit] at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) [junit] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) [junit] at org.apache.lucene.index.NRTManager.waitOnGenCondition(NRTManager.java:251) [junit] at org.apache.lucene.index.NRTManager.waitForGeneration(NRTManager.java:232) [junit] at org.apache.lucene.index.NRTManager.waitForGeneration(NRTManager.java:196) [junit] at org.apache.lucene.index.TestNRTManager.addDocuments(TestNRTManager.java:95) [junit] at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1.run(ThreadedIndexingAndSearchingTestCase.java:223) [junit] [junit] "NRT Reopen Thread" daemon prio=10 tid=0x00007fe660024800 nid=0x22d5 in Object.wait() [0x00007fe66d955000] [junit] java.lang.Thread.State: TIMED_WAITING (on object monitor) [junit] at java.lang.Object.wait(Native Method) [junit] at java.lang.Object.wait(Object.java:443) [junit] at org.apache.lucene.index.NRTManagerReopenThread.run(NRTManagerReopenThread.java:162) [junit] - locked <0x00000000e0006000> (a org.apache.lucene.index.NRTManagerReopenThread) [junit] [junit] "Low Memory Detector" daemon prio=10 tid=0x00007fe668001000 nid=0x22ba runnable [0x0000000000000000] [junit] java.lang.Thread.State: RUNNABLE [junit] [junit] "CompilerThread1" daemon prio=10 tid=0x0000000040f8b000 nid=0x22b7 waiting on condition [0x0000000000000000] [junit] java.lang.Thread.State: RUNNABLE [junit] [junit] "CompilerThread0" daemon prio=10 tid=0x0000000040f88000 nid=0x22b5 waiting on condition [0x0000000000000000] [junit] java.lang.Thread.State: RUNNABLE [junit] [junit] "Signal Dispatcher" daemon prio=10 tid=0x0000000040f86000 nid=0x22b2 waiting on condition [0x0000000000000000] [junit] java.lang.Thread.State: RUNNABLE [junit] [junit] "Finalizer" daemon prio=10 tid=0x0000000040f69000 nid=0x229e in Object.wait() [0x00007fe66e7d7000] [junit] java.lang.Thread.State: WAITING (on object monitor) [junit] at java.lang.Object.wait(Native Method) [junit] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118) [junit] - locked <0x00000000e0002528> (a java.lang.ref.ReferenceQueue$Lock) [junit] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134) [junit] at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) [junit] [junit] "Reference Handler" daemon prio=10 tid=0x0000000040f62000 nid=0x2298 in Object.wait() [0x00007fe66e8d8000] [junit] java.lang.Thread.State: WAITING (on object monitor) [junit] at java.lang.Object.wait(Native Method) [junit] at java.lang.Object.wait(Object.java:485) [junit] at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) [junit] - locked <0x00000000e0006090> (a java.lang.ref.Reference$Lock) [junit] [junit] "main" prio=10 tid=0x0000000040ef6000 nid=0x2240 in Object.wait() [0x00007fe673e7d000] [junit] java.lang.Thread.State: WAITING (on object monitor) [junit] at java.lang.Object.wait(Native Method) [junit] - waiting on <0x00000000e0002090> (a org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1) [junit] at java.lang.Thread.join(Thread.java:1186) [junit] - locked <0x00000000e0002090> (a org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$1) [junit] at java.lang.Thread.join(Thread.java:1239) [junit] at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:524) [junit] at org.apache.lucene.index.TestNRTManager.testNRTManager(TestNRTManager.java:37) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) [junit] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) [junit] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) [junit] at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) [junit] at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) [junit] at org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:610) [junit] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) [junit] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) [junit] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) [junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149) [junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51) [junit] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) [junit] at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) [junit] at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) [junit] at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) [junit] at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) [junit] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) [junit] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) [junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:236) [junit] at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:743) [junit] [junit] "VM Thread" prio=10 tid=0x0000000040f5b000 nid=0x228b runnable [junit] [junit] "GC task thread#0 (ParallelGC)" prio=10 tid=0x0000000040f09000 nid=0x2256 runnable [junit] [junit] "GC task thread#1 (ParallelGC)" prio=10 tid=0x0000000040f0a800 nid=0x225a runnable [junit] [junit] "GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000040f0c800 nid=0x2260 runnable [junit] [junit] "GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000040f0e800 nid=0x2266 runnable [junit] [junit] "GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000040f10000 nid=0x226a runnable [junit] [junit] "GC task thread#5 (ParallelGC)" prio=10 tid=0x0000000040f12000 nid=0x226f runnable [junit] [junit] "GC task thread#6 (ParallelGC)" prio=10 tid=0x0000000040f14000 nid=0x2273 runnable [junit] [junit] "GC task thread#7 (ParallelGC)" prio=10 tid=0x0000000040f15800 nid=0x2276 runnable [junit] [junit] "VM Periodic Task Thread" prio=10 tid=0x00007fe668004000 nid=0x22bc waiting on condition [junit] [junit] JNI global references: 1282 [junit] [junit] Heap [junit] PSYoungGen total 37888K, used 18301K [0x00000000f5560000, 0x00000000f7d80000, 0x0000000100000000) [junit] eden space 37824K, 48% used [0x00000000f5560000,0x00000000f672f780,0x00000000f7a50000) [junit] from space 64K, 100% used [0x00000000f7b00000,0x00000000f7b10000,0x00000000f7b10000) [junit] to space 1536K, 0% used [0x00000000f7c00000,0x00000000f7c00000,0x00000000f7d80000) [junit] PSOldGen total 171392K, used 2193K [0x00000000e0000000, 0x00000000ea760000, 0x00000000f5560000) [junit] object space 171392K, 1% used [0x00000000e0000000,0x00000000e0224610,0x00000000ea760000) [junit] PSPermGen total 21248K, used 11609K [0x00000000dae00000, 0x00000000dc2c0000, 0x00000000e0000000) [junit] object space 21248K, 54% used [0x00000000dae00000,0x00000000db9565c8,0x00000000dc2c0000) [junit]
        Hide
        Michael McCandless added a comment -

        Hmm I hit a hang too, running with:

        ant test -Dtestcase=TestNRTManager -Dtestmethod=testNRTManager -Dtests.seed=2aa27e1aec20c4a2:-4a5a5ecf46837d0e:-7c4f651f1f0b75d7 -Dtests.multiplier=3 -Dtests.nightly=true
        

        Slightly different trace:

            [junit] Testsuite: org.apache.lucene.index.TestNRTManager
            [junit] 2011-11-05 10:53:55
            [junit] Full thread dump Java HotSpot(TM) 64-Bit Server VM (17.0-b16 mixed mode):
            [junit] 
            [junit] "Low Memory Detector" daemon prio=10 tid=0x00007fd41c0ae000 nid=0x21a5 runnable [0x0000000000000000]
            [junit]    java.lang.Thread.State: RUNNABLE
            [junit] 
            [junit] "CompilerThread1" daemon prio=10 tid=0x00007fd41c0ac000 nid=0x21a4 waiting on condition [0x0000000000000000]
            [junit]    java.lang.Thread.State: RUNNABLE
            [junit] 
            [junit] "CompilerThread0" daemon prio=10 tid=0x00007fd41c0a9000 nid=0x21a3 waiting on condition [0x0000000000000000]
            [junit]    java.lang.Thread.State: RUNNABLE
            [junit] 
            [junit] "Signal Dispatcher" daemon prio=10 tid=0x00007fd41c0a7000 nid=0x21a2 waiting on condition [0x0000000000000000]
            [junit]    java.lang.Thread.State: RUNNABLE
            [junit] 
            [junit] "Finalizer" daemon prio=10 tid=0x00007fd41c088000 nid=0x21a1 in Object.wait() [0x00007fd3f36f9000]
            [junit]    java.lang.Thread.State: WAITING (on object monitor)
            [junit] 	at java.lang.Object.wait(Native Method)
            [junit] 	- waiting on <0x00007fd4114a0448> (a java.lang.ref.ReferenceQueue$Lock)
            [junit] 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
            [junit] 	- locked <0x00007fd4114a0448> (a java.lang.ref.ReferenceQueue$Lock)
            [junit] 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
            [junit] 	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
            [junit] 
            [junit] "Reference Handler" daemon prio=10 tid=0x00007fd41c086000 nid=0x21a0 in Object.wait() [0x00007fd3f37fa000]
            [junit]    java.lang.Thread.State: WAITING (on object monitor)
            [junit] 	at java.lang.Object.wait(Native Method)
            [junit] 	- waiting on <0x00007fd4114a8028> (a java.lang.ref.Reference$Lock)
            [junit] 	at java.lang.Object.wait(Object.java:485)
            [junit] 	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
            [junit] 	- locked <0x00007fd4114a8028> (a java.lang.ref.Reference$Lock)
            [junit] 
            [junit] "main" prio=10 tid=0x00007fd41c009800 nid=0x218c waiting on condition [0x00007fd422ef8000]
            [junit]    java.lang.Thread.State: WAITING (parking)
            [junit] 	at sun.misc.Unsafe.park(Native Method)
            [junit] 	- parking to wait for  <0x00007fd40f557ff8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
            [junit] 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
            [junit] 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
            [junit] 	at org.apache.lucene.index.NRTManager.waitOnGenCondition(NRTManager.java:251)
            [junit] 	at org.apache.lucene.index.NRTManager.waitForGeneration(NRTManager.java:232)
            [junit] 	at org.apache.lucene.index.NRTManager.waitForGeneration(NRTManager.java:196)
            [junit] 	at org.apache.lucene.index.TestNRTManager.getFinalSearcher(TestNRTManager.java:47)
            [junit] 	at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:524)
            [junit] 	at org.apache.lucene.index.TestNRTManager.testNRTManager(TestNRTManager.java:39)
            [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
            [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
            [junit] 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
            [junit] 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
            [junit] 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
            [junit] 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
            [junit] 	at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
            [junit] 	at org.apache.lucene.util.LuceneTestCase$3$1.evaluate(LuceneTestCase.java:508)
            [junit] 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
            [junit] 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
            [junit] 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
            [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149)
            [junit] 	at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51)
            [junit] 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
            [junit] 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
            [junit] 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
            [junit] 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
            [junit] 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
            [junit] 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
            [junit] 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
            [junit] 	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
            [junit] 	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
            [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
            [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
            [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
            [junit] 
            [junit] "VM Thread" prio=10 tid=0x00007fd41c081800 nid=0x219f runnable 
            [junit] 
            [junit] "GC task thread#0 (ParallelGC)" prio=10 tid=0x00007fd41c01c800 nid=0x218d runnable 
            [junit] 
            [junit] "GC task thread#1 (ParallelGC)" prio=10 tid=0x00007fd41c01e000 nid=0x218e runnable 
            [junit] 
            [junit] "GC task thread#2 (ParallelGC)" prio=10 tid=0x00007fd41c020000 nid=0x218f runnable 
            [junit] 
            [junit] "GC task thread#3 (ParallelGC)" prio=10 tid=0x00007fd41c021800 nid=0x2190 runnable 
            [junit] 
            [junit] "GC task thread#4 (ParallelGC)" prio=10 tid=0x00007fd41c023800 nid=0x2191 runnable 
            [junit] 
            [junit] "GC task thread#5 (ParallelGC)" prio=10 tid=0x00007fd41c025800 nid=0x2192 runnable 
            [junit] 
            [junit] "GC task thread#6 (ParallelGC)" prio=10 tid=0x00007fd41c027000 nid=0x2193 runnable 
            [junit] 
            [junit] "GC task thread#7 (ParallelGC)" prio=10 tid=0x00007fd41c029000 nid=0x2194 runnable 
            [junit] 
            [junit] "GC task thread#8 (ParallelGC)" prio=10 tid=0x00007fd41c02b000 nid=0x2195 runnable 
            [junit] 
            [junit] "GC task thread#9 (ParallelGC)" prio=10 tid=0x00007fd41c02c800 nid=0x2196 runnable 
            [junit] 
            [junit] "GC task thread#10 (ParallelGC)" prio=10 tid=0x00007fd41c02e800 nid=0x2197 runnable 
            [junit] 
            [junit] "GC task thread#11 (ParallelGC)" prio=10 tid=0x00007fd41c030800 nid=0x2198 runnable 
            [junit] 
            [junit] "GC task thread#12 (ParallelGC)" prio=10 tid=0x00007fd41c032000 nid=0x2199 runnable 
            [junit] 
            [junit] "GC task thread#13 (ParallelGC)" prio=10 tid=0x00007fd41c034000 nid=0x219a runnable 
            [junit] 
            [junit] "GC task thread#14 (ParallelGC)" prio=10 tid=0x00007fd41c036000 nid=0x219b runnable 
            [junit] 
            [junit] "GC task thread#15 (ParallelGC)" prio=10 tid=0x00007fd41c037800 nid=0x219c runnable 
            [junit] 
            [junit] "GC task thread#16 (ParallelGC)" prio=10 tid=0x00007fd41c039800 nid=0x219d runnable 
            [junit] 
            [junit] "GC task thread#17 (ParallelGC)" prio=10 tid=0x00007fd41c03b000 nid=0x219e runnable 
            [junit] 
            [junit] "VM Periodic Task Thread" prio=10 tid=0x00007fd41c0c0800 nid=0x21a6 waiting on condition 
            [junit] 
            [junit] JNI global references: 1054
            [junit] 
            [junit] Heap
            [junit]  PSYoungGen      total 56448K, used 35319K [0x00007fd40e560000, 0x00007fd4153a0000, 0x00007fd419000000)
            [junit]   eden space 48384K, 56% used [0x00007fd40e560000,0x00007fd4100010d8,0x00007fd4114a0000)
            [junit]   from space 8064K, 99% used [0x00007fd4114a0000,0x00007fd411c7cc68,0x00007fd411c80000)
            [junit]   to   space 8064K, 0% used [0x00007fd414bc0000,0x00007fd414bc0000,0x00007fd4153a0000)
            [junit]  PSOldGen        total 129088K, used 21713K [0x00007fd3f9000000, 0x00007fd400e10000, 0x00007fd40e560000)
            [junit]   object space 129088K, 16% used [0x00007fd3f9000000,0x00007fd3fa534438,0x00007fd400e10000)
            [junit]  PSPermGen       total 21248K, used 10801K [0x00007fd3f3c00000, 0x00007fd3f50c0000, 0x00007fd3f9000000)
            [junit]   object space 21248K, 50% used [0x00007fd3f3c00000,0x00007fd3f468c5c8,0x00007fd3f50c0000)
            [junit] 
        
        Show
        Michael McCandless added a comment - Hmm I hit a hang too, running with: ant test -Dtestcase=TestNRTManager -Dtestmethod=testNRTManager -Dtests.seed=2aa27e1aec20c4a2:-4a5a5ecf46837d0e:-7c4f651f1f0b75d7 -Dtests.multiplier=3 -Dtests.nightly=true Slightly different trace: [junit] Testsuite: org.apache.lucene.index.TestNRTManager [junit] 2011-11-05 10:53:55 [junit] Full thread dump Java HotSpot(TM) 64-Bit Server VM (17.0-b16 mixed mode): [junit] [junit] "Low Memory Detector" daemon prio=10 tid=0x00007fd41c0ae000 nid=0x21a5 runnable [0x0000000000000000] [junit] java.lang.Thread.State: RUNNABLE [junit] [junit] "CompilerThread1" daemon prio=10 tid=0x00007fd41c0ac000 nid=0x21a4 waiting on condition [0x0000000000000000] [junit] java.lang.Thread.State: RUNNABLE [junit] [junit] "CompilerThread0" daemon prio=10 tid=0x00007fd41c0a9000 nid=0x21a3 waiting on condition [0x0000000000000000] [junit] java.lang.Thread.State: RUNNABLE [junit] [junit] "Signal Dispatcher" daemon prio=10 tid=0x00007fd41c0a7000 nid=0x21a2 waiting on condition [0x0000000000000000] [junit] java.lang.Thread.State: RUNNABLE [junit] [junit] "Finalizer" daemon prio=10 tid=0x00007fd41c088000 nid=0x21a1 in Object.wait() [0x00007fd3f36f9000] [junit] java.lang.Thread.State: WAITING (on object monitor) [junit] at java.lang.Object.wait(Native Method) [junit] - waiting on <0x00007fd4114a0448> (a java.lang.ref.ReferenceQueue$Lock) [junit] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118) [junit] - locked <0x00007fd4114a0448> (a java.lang.ref.ReferenceQueue$Lock) [junit] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134) [junit] at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) [junit] [junit] "Reference Handler" daemon prio=10 tid=0x00007fd41c086000 nid=0x21a0 in Object.wait() [0x00007fd3f37fa000] [junit] java.lang.Thread.State: WAITING (on object monitor) [junit] at java.lang.Object.wait(Native Method) [junit] - waiting on <0x00007fd4114a8028> (a java.lang.ref.Reference$Lock) [junit] at java.lang.Object.wait(Object.java:485) [junit] at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) [junit] - locked <0x00007fd4114a8028> (a java.lang.ref.Reference$Lock) [junit] [junit] "main" prio=10 tid=0x00007fd41c009800 nid=0x218c waiting on condition [0x00007fd422ef8000] [junit] java.lang.Thread.State: WAITING (parking) [junit] at sun.misc.Unsafe.park(Native Method) [junit] - parking to wait for <0x00007fd40f557ff8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) [junit] at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) [junit] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) [junit] at org.apache.lucene.index.NRTManager.waitOnGenCondition(NRTManager.java:251) [junit] at org.apache.lucene.index.NRTManager.waitForGeneration(NRTManager.java:232) [junit] at org.apache.lucene.index.NRTManager.waitForGeneration(NRTManager.java:196) [junit] at org.apache.lucene.index.TestNRTManager.getFinalSearcher(TestNRTManager.java:47) [junit] at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:524) [junit] at org.apache.lucene.index.TestNRTManager.testNRTManager(TestNRTManager.java:39) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [junit] at java.lang.reflect.Method.invoke(Method.java:597) [junit] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) [junit] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) [junit] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) [junit] at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) [junit] at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48) [junit] at org.apache.lucene.util.LuceneTestCase$3$1.evaluate(LuceneTestCase.java:508) [junit] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) [junit] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) [junit] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76) [junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149) [junit] at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51) [junit] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) [junit] at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) [junit] at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) [junit] at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) [junit] at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) [junit] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) [junit] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) [junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:236) [junit] at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911) [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768) [junit] [junit] "VM Thread" prio=10 tid=0x00007fd41c081800 nid=0x219f runnable [junit] [junit] "GC task thread#0 (ParallelGC)" prio=10 tid=0x00007fd41c01c800 nid=0x218d runnable [junit] [junit] "GC task thread#1 (ParallelGC)" prio=10 tid=0x00007fd41c01e000 nid=0x218e runnable [junit] [junit] "GC task thread#2 (ParallelGC)" prio=10 tid=0x00007fd41c020000 nid=0x218f runnable [junit] [junit] "GC task thread#3 (ParallelGC)" prio=10 tid=0x00007fd41c021800 nid=0x2190 runnable [junit] [junit] "GC task thread#4 (ParallelGC)" prio=10 tid=0x00007fd41c023800 nid=0x2191 runnable [junit] [junit] "GC task thread#5 (ParallelGC)" prio=10 tid=0x00007fd41c025800 nid=0x2192 runnable [junit] [junit] "GC task thread#6 (ParallelGC)" prio=10 tid=0x00007fd41c027000 nid=0x2193 runnable [junit] [junit] "GC task thread#7 (ParallelGC)" prio=10 tid=0x00007fd41c029000 nid=0x2194 runnable [junit] [junit] "GC task thread#8 (ParallelGC)" prio=10 tid=0x00007fd41c02b000 nid=0x2195 runnable [junit] [junit] "GC task thread#9 (ParallelGC)" prio=10 tid=0x00007fd41c02c800 nid=0x2196 runnable [junit] [junit] "GC task thread#10 (ParallelGC)" prio=10 tid=0x00007fd41c02e800 nid=0x2197 runnable [junit] [junit] "GC task thread#11 (ParallelGC)" prio=10 tid=0x00007fd41c030800 nid=0x2198 runnable [junit] [junit] "GC task thread#12 (ParallelGC)" prio=10 tid=0x00007fd41c032000 nid=0x2199 runnable [junit] [junit] "GC task thread#13 (ParallelGC)" prio=10 tid=0x00007fd41c034000 nid=0x219a runnable [junit] [junit] "GC task thread#14 (ParallelGC)" prio=10 tid=0x00007fd41c036000 nid=0x219b runnable [junit] [junit] "GC task thread#15 (ParallelGC)" prio=10 tid=0x00007fd41c037800 nid=0x219c runnable [junit] [junit] "GC task thread#16 (ParallelGC)" prio=10 tid=0x00007fd41c039800 nid=0x219d runnable [junit] [junit] "GC task thread#17 (ParallelGC)" prio=10 tid=0x00007fd41c03b000 nid=0x219e runnable [junit] [junit] "VM Periodic Task Thread" prio=10 tid=0x00007fd41c0c0800 nid=0x21a6 waiting on condition [junit] [junit] JNI global references: 1054 [junit] [junit] Heap [junit] PSYoungGen total 56448K, used 35319K [0x00007fd40e560000, 0x00007fd4153a0000, 0x00007fd419000000) [junit] eden space 48384K, 56% used [0x00007fd40e560000,0x00007fd4100010d8,0x00007fd4114a0000) [junit] from space 8064K, 99% used [0x00007fd4114a0000,0x00007fd411c7cc68,0x00007fd411c80000) [junit] to space 8064K, 0% used [0x00007fd414bc0000,0x00007fd414bc0000,0x00007fd4153a0000) [junit] PSOldGen total 129088K, used 21713K [0x00007fd3f9000000, 0x00007fd400e10000, 0x00007fd40e560000) [junit] object space 129088K, 16% used [0x00007fd3f9000000,0x00007fd3fa534438,0x00007fd400e10000) [junit] PSPermGen total 21248K, used 10801K [0x00007fd3f3c00000, 0x00007fd3f50c0000, 0x00007fd3f9000000) [junit] object space 21248K, 50% used [0x00007fd3f3c00000,0x00007fd3f468c5c8,0x00007fd3f50c0000) [junit]
        Hide
        Dawid Weiss added a comment -

        Slightly different trace:

        Oh, this reminded me of an interesting feature I meant to implement in the RandomizedRunner here –
        https://github.com/carrotsearch/randomizedtesting/issues/18

        Show
        Dawid Weiss added a comment - Slightly different trace: Oh, this reminded me of an interesting feature I meant to implement in the RandomizedRunner here – https://github.com/carrotsearch/randomizedtesting/issues/18
        Hide
        Michael McCandless added a comment -

        OK so I think my hang occurred because the test had hit an OOME, and then was hung trying to get final searcher. It's still a hang we should figure out, but it's happening under an exceptional situation.

        Dawid: that (getting traces of all threads) is vital, before killing hung tests! When can we cut Lucene/Solr over to RandomizedRunner!!

        Show
        Michael McCandless added a comment - OK so I think my hang occurred because the test had hit an OOME, and then was hung trying to get final searcher. It's still a hang we should figure out, but it's happening under an exceptional situation. Dawid: that (getting traces of all threads) is vital, before killing hung tests! When can we cut Lucene/Solr over to RandomizedRunner!!
        Hide
        Dawid Weiss added a comment -

        I already do get a stack before killing a thread, but it's a single stack. I think probing multiple times will be much more informative (and fun

        Show
        Dawid Weiss added a comment - I already do get a stack before killing a thread, but it's a single stack. I think probing multiple times will be much more informative (and fun
        Hide
        Simon Willnauer added a comment -

        I see one obvious glitch in NRTManager#close(), if the IOUtils.close throws an exception we miss to signal the waiting threads. this should be in a additional try / finally.

        I will look at this again during apache con...

        Show
        Simon Willnauer added a comment - I see one obvious glitch in NRTManager#close(), if the IOUtils.close throws an exception we miss to signal the waiting threads. this should be in a additional try / finally. I will look at this again during apache con...
        Hide
        Simon Willnauer added a comment -

        here is a patch making sure we unlock if SearcherManager#close throws an exception during NRTManager#close().

        Show
        Simon Willnauer added a comment - here is a patch making sure we unlock if SearcherManager#close throws an exception during NRTManager#close().
        Hide
        Robert Muir added a comment -

        +1 for the extra safety regarding close()

        Show
        Robert Muir added a comment - +1 for the extra safety regarding close()
        Hide
        Simon Willnauer added a comment -

        here is a new patch that signals when we try to reopen a searcher that is already closed ie. has a gen == Long.MAX. I am going to commit this one soon.

        This doesn't fix the hanging problem with is actually a bigger problem than I thought in the beginning. I try to isolate a testcase and explain it in a minute.

        Show
        Simon Willnauer added a comment - here is a new patch that signals when we try to reopen a searcher that is already closed ie. has a gen == Long.MAX. I am going to commit this one soon. This doesn't fix the hanging problem with is actually a bigger problem than I thought in the beginning. I try to isolate a testcase and explain it in a minute.
        Hide
        Simon Willnauer added a comment -

        here is an isolated testcase that shows the situation where we hang in NRTManager. This was the good news... the bad news is that this is going to be tricky to fix...

        the problem is that if you add a document and while you are in IW#updateDocument a reopen happens your document could already be in the new opened searcher. This means the gen is incremented internally. So far there is no problem, but if the reopen thread tries to do another reopen it increments the internal genration but doesn't assign the gen to the searcher since no changes occurred. now the thread adding a the document gets the generation back from add/updateDocument and calls IW#waitForGeneration(gen, false) and sleeps forever unless there is yet another update coming in and a reopen returns a new searcher.

        Show
        Simon Willnauer added a comment - here is an isolated testcase that shows the situation where we hang in NRTManager. This was the good news... the bad news is that this is going to be tricky to fix... the problem is that if you add a document and while you are in IW#updateDocument a reopen happens your document could already be in the new opened searcher. This means the gen is incremented internally. So far there is no problem, but if the reopen thread tries to do another reopen it increments the internal genration but doesn't assign the gen to the searcher since no changes occurred. now the thread adding a the document gets the generation back from add/updateDocument and calls IW#waitForGeneration(gen, false) and sleeps forever unless there is yet another update coming in and a reopen returns a new searcher.
        Hide
        Simon Willnauer added a comment -

        I rewrote the testcase to actually deadlock and relaxed the contract of SearcherManager#maybeReopen to also return true if the current reader was not reopened since it already contained all changes. With this fix the test passes. Phew!

        Show
        Simon Willnauer added a comment - I rewrote the testcase to actually deadlock and relaxed the contract of SearcherManager#maybeReopen to also return true if the current reader was not reopened since it already contained all changes. With this fix the test passes. Phew!
        Hide
        Michael McCandless added a comment -

        Egads, this is sneaky! Nice test case.

        Can we move the fix "up"? Ie, leave SearcherManager.maybeReopen as returning boolean indicating if a new reader was really reopened, but then in NRTManager.maybeReopen, always carry the gen forward once we've called maybeReopen?

        Show
        Michael McCandless added a comment - Egads, this is sneaky! Nice test case. Can we move the fix "up"? Ie, leave SearcherManager.maybeReopen as returning boolean indicating if a new reader was really reopened, but then in NRTManager.maybeReopen, always carry the gen forward once we've called maybeReopen?
        Hide
        Dawid Weiss added a comment -

        I've implemented probing. Thread leak/ timeout test stack probing. Works quite nicely, an example output from a test case here:

        https://github.com/carrotsearch/randomizedtesting/issues/18

        @Michael: We will cut to RandomizedRunner in Lucene once the rough edges of the runner itself are polished... I don't want to throw something half-baked on people and need to make sure it's functional and rock-solid. So far I've ported all of Carrot2 tests to run on RandomizedRunner. Lots of good ideas and bug fixes from that so far – see github's issue tracker.

        Dawid

        Show
        Dawid Weiss added a comment - I've implemented probing. Thread leak/ timeout test stack probing. Works quite nicely, an example output from a test case here: https://github.com/carrotsearch/randomizedtesting/issues/18 @Michael: We will cut to RandomizedRunner in Lucene once the rough edges of the runner itself are polished... I don't want to throw something half-baked on people and need to make sure it's functional and rock-solid. So far I've ported all of Carrot2 tests to run on RandomizedRunner. Lots of good ideas and bug fixes from that so far – see github's issue tracker. Dawid
        Hide
        Simon Willnauer added a comment -

        Can we move the fix "up"? Ie, leave SearcherManager.maybeReopen as returning boolean indicating if a new reader was really reopened, but then in NRTManager.maybeReopen, always carry the gen forward once we've called maybeReopen?

        Mike, the problem here is that you need to have some information about if we tried to reopen or not. If you use a NRT reader that doesn't apply deletes you will always get false from reader.isCurrent() provided you have a delete still sitting in memory. So if we move this up we can not tell if we simply didn't get the lock in maybeReopen or if the reopen returned null. We could make the contract of maybeReopen stronger and remove the non-blocking properties from it or add a second method which blocks.
        Simply moving it up is hardly possible since you can not tell what really happened in maybeReopen()

        Show
        Simon Willnauer added a comment - Can we move the fix "up"? Ie, leave SearcherManager.maybeReopen as returning boolean indicating if a new reader was really reopened, but then in NRTManager.maybeReopen, always carry the gen forward once we've called maybeReopen? Mike, the problem here is that you need to have some information about if we tried to reopen or not. If you use a NRT reader that doesn't apply deletes you will always get false from reader.isCurrent() provided you have a delete still sitting in memory. So if we move this up we can not tell if we simply didn't get the lock in maybeReopen or if the reopen returned null. We could make the contract of maybeReopen stronger and remove the non-blocking properties from it or add a second method which blocks. Simply moving it up is hardly possible since you can not tell what really happened in maybeReopen()
        Hide
        Michael McCandless added a comment -

        OK that makes sense Simon... I think your patch is great as is! Thanks.

        Show
        Michael McCandless added a comment - OK that makes sense Simon... I think your patch is great as is! Thanks.
        Hide
        Simon Willnauer added a comment -

        fixed in 3.x & trunk - phew

        Show
        Simon Willnauer added a comment - fixed in 3.x & trunk - phew
        Hide
        Uwe Schindler added a comment -

        Bulk close after release of 3.5

        Show
        Uwe Schindler added a comment - Bulk close after release of 3.5

          People

          • Assignee:
            Simon Willnauer
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development