Lucene - Core
  1. Lucene - Core
  2. LUCENE-4269

TestBalancedSegmentMergePolicy.setUp hang

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      http://85.25.120.39/job/Lucene-trunk-Linux-Java7-64 was hung for a couple days ... stacktrace:

      2012-07-28 13:45:03
      Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.1-b03 mixed mode):
      
      "TEST-TestScope-org.apache.lucene.index.TestBalancedSegmentMergePolicy.testForceMerge-seed#[F42FFCD5FAEE14FF]" prio=10 tid=0x00007f56e8371800 nid=0x106c runnable [0x00007f56b2130000]
         java.lang.Thread.State: RUNNABLE
      	at org.apache.lucene.codecs.BlockTreeTermsWriter$TermsWriter$FindBlocks.freeze(BlockTreeTermsWriter.java:392)
      	at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:212)
      	at org.apache.lucene.util.fst.Builder.add(Builder.java:392)
      	at org.apache.lucene.codecs.BlockTreeTermsWriter$TermsWriter.finishTerm(BlockTreeTermsWriter.java:864)
      	at org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:167)
      	at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:65)
      	at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:323)
      	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:110)
      	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3539)
      	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3154)
      	at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:36)
      	- locked <0x00000000dc0fcfb8> (a org.apache.lucene.index.SerialMergeScheduler)
      	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1727)
      	at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1721)
      	at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2612)
      	- locked <0x00000000dc0fcfc8> (a java.lang.Object)
      	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2699)
      	- locked <0x00000000dc0fcfc8> (a java.lang.Object)
      	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2679)
      	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2663)
      	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:346)
      	at org.apache.lucene.index.TestBalancedSegmentMergePolicy.setUp(TestBalancedSegmentMergePolicy.java:55)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:601)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
      	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
      	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
      	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
      	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
      	at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
      	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
      	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
      	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
      	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
      	at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
      	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
      	at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
      	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
      	at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
      	at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
      	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
      	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
      	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
      	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
      
      "Service Thread" daemon prio=10 tid=0x00007f56e81be800 nid=0x1062 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread1" daemon prio=10 tid=0x00007f56e81bc000 nid=0x1060 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread0" daemon prio=10 tid=0x00007f56e81b9800 nid=0x105f waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Signal Dispatcher" daemon prio=10 tid=0x00007f56e81b7800 nid=0x105e waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007f56e81b5800 nid=0x105c waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Finalizer" daemon prio=10 tid=0x00007f56e8167000 nid=0x1057 in Object.wait() [0x00007f56b3127000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000000dd21e078> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
      	- locked <0x00000000dd21e078> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
      	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
      
      "Reference Handler" daemon prio=10 tid=0x00007f56e8164800 nid=0x1056 in Object.wait() [0x00007f56b3228000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000000dd21e068> (a java.lang.ref.Reference$Lock)
      	at java.lang.Object.wait(Object.java:503)
      	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
      	- locked <0x00000000dd21e068> (a java.lang.ref.Reference$Lock)
      
      "main" prio=10 tid=0x00007f56e800c000 nid=0x1028 in Object.wait() [0x00007f56edd14000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000000dc0f4528> (a com.carrotsearch.randomizedtesting.RandomizedRunner$2)
      	at java.lang.Thread.join(Thread.java:1258)
      	- locked <0x00000000dc0f4528> (a com.carrotsearch.randomizedtesting.RandomizedRunner$2)
      	at java.lang.Thread.join(Thread.java:1332)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:561)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:521)
      	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:153)
      	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:246)
      	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
      
      "VM Thread" prio=10 tid=0x00007f
      

        Activity

        Hide
        Michael McCandless added a comment -

        Events file:

        [
          "BOOTSTRAP",
          {
            "defaultCharset": "US-ASCII",
            "systemProperties": {
              "awt.toolkit": "sun.awt.X11.XToolkit",
              "file.encoding": "US-ASCII",
              "file.encoding.pkg": "sun.io",
              "file.separator": "/",
              "java.awt.graphicsenv": "sun.awt.X11GraphicsEnvironment",
              "java.awt.printerjob": "sun.print.PSPrinterJob",
              "java.class.path": "/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/test-framework/classes/java:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/core/classes/java:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/test-framework/lib/junit-4.10.jar:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/test-framework/lib/randomizedtesting-runner-1.6.0.jar:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/classes/java:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/classes/test:/var/lib/jenkins/tools/ant/supported18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/apache-rat-tasks-0.8.jar:/var/lib/jenkins/.ant/lib/ivy-2.2.0.jar:/var/lib/jenkins/.ant/lib/apache-rat-0.8.jar:/var/lib/jenkins/.ant/lib/apache-rat-plugin-0.8.jar:/var/lib/jenkins/.ant/lib/apache-rat-core-0.8.jar:/var/lib/jenkins/.ant/lib/clover-2.6.3.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ivy-2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-javamail.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-netrexx.jar:/opt/java/64/jdk1.7.0_05/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-1.6.0.jar",
              "java.class.version": "51.0",
              "java.endorsed.dirs": "/opt/java/64/jdk1.7.0_05/jre/lib/endorsed",
              "java.ext.dirs": "/opt/java/64/jdk1.7.0_05/jre/lib/ext:/usr/java/packages/lib/ext",
              "java.home": "/opt/java/64/jdk1.7.0_05/jre",
              "java.io.tmpdir": "/tmp",
              "java.library.path": "/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib",
              "java.runtime.name": "Java(TM) SE Runtime Environment",
              "java.runtime.version": "1.7.0_05-b05",
              "java.specification.name": "Java Platform API Specification",
              "java.specification.vendor": "Oracle Corporation",
              "java.specification.version": "1.7",
              "java.util.logging.config.file": "/dev/null",
              "java.vendor": "Oracle Corporation",
              "java.vendor.url": "http://java.oracle.com/",
              "java.vendor.url.bug": "http://bugreport.sun.com/bugreport/",
              "java.version": "1.7.0_05",
              "java.vm.info": "mixed mode",
              "java.vm.name": "Java HotSpot(TM) 64-Bit Server VM",
              "java.vm.specification.name": "Java Virtual Machine Specification",
              "java.vm.specification.vendor": "Oracle Corporation",
              "java.vm.specification.version": "1.7",
              "java.vm.vendor": "Oracle Corporation",
              "java.vm.version": "23.1-b03",
              "jetty.insecurerandom": "1",
              "jetty.testMode": "1",
              "junit4.memory.total": "131072000",
              "junit4.processors": "8",
              "line.separator": "\n",
              "lucene.version": "5.0-SNAPSHOT",
              "os.arch": "amd64",
              "os.name": "Linux",
              "os.version": "3.2.0-26-generic",
              "path.separator": ":",
              "solr.directoryFactory": "org.apache.solr.core.MockDirectoryFactory",
              "sun.arch.data.model": "64",
              "sun.boot.class.path": "/opt/java/64/jdk1.7.0_05/jre/lib/resources.jar:/opt/java/64/jdk1.7.0_05/jre/lib/rt.jar:/opt/java/64/jdk1.7.0_05/jre/lib/sunrsasign.jar:/opt/java/64/jdk1.7.0_05/jre/lib/jsse.jar:/opt/java/64/jdk1.7.0_05/jre/lib/jce.jar:/opt/java/64/jdk1.7.0_05/jre/lib/charsets.jar:/opt/java/64/jdk1.7.0_05/jre/lib/jfr.jar:/opt/java/64/jdk1.7.0_05/jre/classes",
              "sun.boot.library.path": "/opt/java/64/jdk1.7.0_05/jre/lib/amd64",
              "sun.cpu.endian": "little",
              "sun.cpu.isalist": "",
              "sun.io.unicode.encoding": "UnicodeLittle",
              "sun.java.command": "com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/test/junit4-J0-1556533854f47d3df.events @/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/test/junit4-J0-1556533854f47d3df.suites -stdin",
              "sun.java.launcher": "SUN_STANDARD",
              "sun.jnu.encoding": "UTF-8",
              "sun.management.compiler": "HotSpot 64-Bit Tiered Compilers",
              "sun.os.patch.level": "unknown",
              "tempDir": ".",
              "tests.asserts.gracious": "false",
              "tests.cleanthreads": "perMethod",
              "tests.codec": "random",
              "tests.directory": "random",
              "tests.infostream": "false",
              "tests.iters": "",
              "tests.linedocsfile": "/var/lib/jenkins/lucene-data/enwiki.random.lines.txt",
              "tests.locale": "random",
              "tests.lockdir": "/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build",
              "tests.luceneMatchVersion": "5.0",
              "tests.multiplier": "3",
              "tests.nightly": "true",
              "tests.postingsformat": "random",
              "tests.prefix": "tests",
              "tests.seed": "F42FFCD5FAEE14FF",
              "tests.slow": "true",
              "tests.timezone": "random",
              "tests.verbose": "false",
              "tests.weekly": "false",
              "user.country": "US",
              "user.dir": "/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/test/J0",
              "user.home": "/var/lib/jenkins",
              "user.language": "en",
              "user.name": "jenkins",
              "user.timezone": ""
            }
          }
        ]
        
        [
          "SUITE_STARTED",
          {
            "description": {
              "id": "id#org.apache.lucene.index.TestMultiPassIndexSplitter[3]",
              "displayName": "org.apache.lucene.index.TestMultiPassIndexSplitter",
              "className": "org.apache.lucene.index.TestMultiPassIndexSplitter",
              "annotations": [
                {
                  "com.carrotsearch.randomizedtesting.annotations.ThreadLeaks": {
                    "failTestIfLeaking": false,
                    "stackSamples": 10,
                    "linger": 0,
                    "leakedThreadsBelongToSuite": false
                  }
                },
                {
                  "com.carrotsearch.randomizedtesting.annotations.TestMethodProviders": {
                    "value": [
                      "org.apache.lucene.util.LuceneJUnit3MethodProvider",
                      "com.carrotsearch.randomizedtesting.JUnit4MethodProvider"
                    ]
                  }
                },
                {
                  "org.junit.runner.RunWith": {
                    "value": "com.carrotsearch.randomizedtesting.RandomizedRunner"
                  }
                }
              ],
              "children": [
                {
                  "id": "id#testSplitSeq(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]",
                  "displayName": "testSplitSeq(org.apache.lucene.index.TestMultiPassIndexSplitter)",
                  "methodName": "testSplitSeq",
                  "className": "org.apache.lucene.index.TestMultiPassIndexSplitter",
                  "annotations": [],
                  "children": []
                },
                {
                  "id": "id#testSplitRR(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]",
                  "displayName": "testSplitRR(org.apache.lucene.index.TestMultiPassIndexSplitter)",
                  "methodName": "testSplitRR",
                  "className": "org.apache.lucene.index.TestMultiPassIndexSplitter",
                  "annotations": [],
                  "children": []
                }
              ]
            }
          }
        ]
        
        [
          "TEST_STARTED",
          {
            "description": "id#testSplitSeq(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]"
          }
        ]
        
        [
          "APPEND_STDERR",
          {
            "chunk": "57726974696E6720706172742031202E2E2E0A57726974696E6720706172742032202E2E2E0A57726974696E6720706172742033202E2E2E0A446F6E652E0A"
          }
        ]
        
        [
          "TEST_FINISHED",
          {
            "executionTime": 979,
            "startTimestamp": 1343224614831,
            "description": "id#testSplitSeq(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]"
          }
        ]
        
        [
          "TEST_STARTED",
          {
            "description": "id#testSplitRR(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]"
          }
        ]
        
        [
          "APPEND_STDERR",
          {
            "chunk": "57726974696E6720706172742031202E2E2E0A57726974696E6720706172742032202E2E2E0A57726974696E6720706172742033202E2E2E0A446F6E652E0A"
          }
        ]
        
        [
          "TEST_FINISHED",
          {
            "executionTime": 1350,
            "startTimestamp": 1343224615815,
            "description": "id#testSplitRR(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]"
          }
        ]
        
        [
          "SUITE_COMPLETED",
          {
            "startTimestamp": 1343224614447,
            "executionTime": 2777,
            "description": "id#org.apache.lucene.index.TestMultiPassIndexSplitter[3]"
          }
        ]
        
        [
          "SUITE_STARTED",
          {
            "description": {
              "id": "id#org.apache.lucene.index.TestBalancedSegmentMergePolicy[3]",
              "displayName": "org.apache.lucene.index.TestBalancedSegmentMergePolicy",
              "className": "org.apache.lucene.index.TestBalancedSegmentMergePolicy",
              "annotations": [
                {
                  "com.carrotsearch.randomizedtesting.annotations.ThreadLeaks": {
                    "failTestIfLeaking": false,
                    "stackSamples": 10,
                    "linger": 0,
                    "leakedThreadsBelongToSuite": false
                  }
                },
                {
                  "com.carrotsearch.randomizedtesting.annotations.TestMethodProviders": {
                    "value": [
                      "org.apache.lucene.util.LuceneJUnit3MethodProvider",
                      "com.carrotsearch.randomizedtesting.JUnit4MethodProvider"
                    ]
                  }
                },
                {
                  "org.junit.runner.RunWith": {
                    "value": "com.carrotsearch.randomizedtesting.RandomizedRunner"
                  }
                }
              ],
              "children": [
                {
                  "id": "id#testForceMerge(org.apache.lucene.index.TestBalancedSegmentMergePolicy)[0]",
                  "displayName": "testForceMerge(org.apache.lucene.index.TestBalancedSegmentMergePolicy)",
                  "methodName": "testForceMerge",
                  "className": "org.apache.lucene.index.TestBalancedSegmentMergePolicy",
                  "annotations": [],
                  "children": []
                }
              ]
            }
          }
        ]
        
        [
          "TEST_STARTED",
          {
            "description": "id#testForceMerge(org.apache.lucene.index.TestBalancedSegmentMergePolicy)[0]"
          }
        ]
        
        
        Show
        Michael McCandless added a comment - Events file: [ "BOOTSTRAP", { "defaultCharset": "US-ASCII", "systemProperties": { "awt.toolkit": "sun.awt.X11.XToolkit", "file.encoding": "US-ASCII", "file.encoding.pkg": "sun.io", "file.separator": "/", "java.awt.graphicsenv": "sun.awt.X11GraphicsEnvironment", "java.awt.printerjob": "sun.print.PSPrinterJob", "java.class.path": "/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/test-framework/classes/java:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/core/classes/java:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/test-framework/lib/junit-4.10.jar:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/test-framework/lib/randomizedtesting-runner-1.6.0.jar:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/classes/java:/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/classes/test:/var/lib/jenkins/tools/ant/supported18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/apache-rat-tasks-0.8.jar:/var/lib/jenkins/.ant/lib/ivy-2.2.0.jar:/var/lib/jenkins/.ant/lib/apache-rat-0.8.jar:/var/lib/jenkins/.ant/lib/apache-rat-plugin-0.8.jar:/var/lib/jenkins/.ant/lib/apache-rat-core-0.8.jar:/var/lib/jenkins/.ant/lib/clover-2.6.3.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ivy-2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-javamail.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-netrexx.jar:/opt/java/64/jdk1.7.0_05/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-1.6.0.jar", "java.class.version": "51.0", "java.endorsed.dirs": "/opt/java/64/jdk1.7.0_05/jre/lib/endorsed", "java.ext.dirs": "/opt/java/64/jdk1.7.0_05/jre/lib/ext:/usr/java/packages/lib/ext", "java.home": "/opt/java/64/jdk1.7.0_05/jre", "java.io.tmpdir": "/tmp", "java.library.path": "/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib", "java.runtime.name": "Java(TM) SE Runtime Environment", "java.runtime.version": "1.7.0_05-b05", "java.specification.name": "Java Platform API Specification", "java.specification.vendor": "Oracle Corporation", "java.specification.version": "1.7", "java.util.logging.config.file": "/dev/null", "java.vendor": "Oracle Corporation", "java.vendor.url": "http://java.oracle.com/", "java.vendor.url.bug": "http://bugreport.sun.com/bugreport/", "java.version": "1.7.0_05", "java.vm.info": "mixed mode", "java.vm.name": "Java HotSpot(TM) 64-Bit Server VM", "java.vm.specification.name": "Java Virtual Machine Specification", "java.vm.specification.vendor": "Oracle Corporation", "java.vm.specification.version": "1.7", "java.vm.vendor": "Oracle Corporation", "java.vm.version": "23.1-b03", "jetty.insecurerandom": "1", "jetty.testMode": "1", "junit4.memory.total": "131072000", "junit4.processors": "8", "line.separator": "\n", "lucene.version": "5.0-SNAPSHOT", "os.arch": "amd64", "os.name": "Linux", "os.version": "3.2.0-26-generic", "path.separator": ":", "solr.directoryFactory": "org.apache.solr.core.MockDirectoryFactory", "sun.arch.data.model": "64", "sun.boot.class.path": "/opt/java/64/jdk1.7.0_05/jre/lib/resources.jar:/opt/java/64/jdk1.7.0_05/jre/lib/rt.jar:/opt/java/64/jdk1.7.0_05/jre/lib/sunrsasign.jar:/opt/java/64/jdk1.7.0_05/jre/lib/jsse.jar:/opt/java/64/jdk1.7.0_05/jre/lib/jce.jar:/opt/java/64/jdk1.7.0_05/jre/lib/charsets.jar:/opt/java/64/jdk1.7.0_05/jre/lib/jfr.jar:/opt/java/64/jdk1.7.0_05/jre/classes", "sun.boot.library.path": "/opt/java/64/jdk1.7.0_05/jre/lib/amd64", "sun.cpu.endian": "little", "sun.cpu.isalist": "", "sun.io.unicode.encoding": "UnicodeLittle", "sun.java.command": "com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/test/junit4-J0-1556533854f47d3df.events @/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/test/junit4-J0-1556533854f47d3df.suites -stdin", "sun.java.launcher": "SUN_STANDARD", "sun.jnu.encoding": "UTF-8", "sun.management.compiler": "HotSpot 64-Bit Tiered Compilers", "sun.os.patch.level": "unknown", "tempDir": ".", "tests.asserts.gracious": "false", "tests.cleanthreads": "perMethod", "tests.codec": "random", "tests.directory": "random", "tests.infostream": "false", "tests.iters": "", "tests.linedocsfile": "/var/lib/jenkins/lucene-data/enwiki.random.lines.txt", "tests.locale": "random", "tests.lockdir": "/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build", "tests.luceneMatchVersion": "5.0", "tests.multiplier": "3", "tests.nightly": "true", "tests.postingsformat": "random", "tests.prefix": "tests", "tests.seed": "F42FFCD5FAEE14FF", "tests.slow": "true", "tests.timezone": "random", "tests.verbose": "false", "tests.weekly": "false", "user.country": "US", "user.dir": "/var/lib/jenkins/workspace/Lucene-trunk-Linux-Java7-64/checkout/lucene/build/misc/test/J0", "user.home": "/var/lib/jenkins", "user.language": "en", "user.name": "jenkins", "user.timezone": "" } } ] [ "SUITE_STARTED", { "description": { "id": "id#org.apache.lucene.index.TestMultiPassIndexSplitter[3]", "displayName": "org.apache.lucene.index.TestMultiPassIndexSplitter", "className": "org.apache.lucene.index.TestMultiPassIndexSplitter", "annotations": [ { "com.carrotsearch.randomizedtesting.annotations.ThreadLeaks": { "failTestIfLeaking": false, "stackSamples": 10, "linger": 0, "leakedThreadsBelongToSuite": false } }, { "com.carrotsearch.randomizedtesting.annotations.TestMethodProviders": { "value": [ "org.apache.lucene.util.LuceneJUnit3MethodProvider", "com.carrotsearch.randomizedtesting.JUnit4MethodProvider" ] } }, { "org.junit.runner.RunWith": { "value": "com.carrotsearch.randomizedtesting.RandomizedRunner" } } ], "children": [ { "id": "id#testSplitSeq(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]", "displayName": "testSplitSeq(org.apache.lucene.index.TestMultiPassIndexSplitter)", "methodName": "testSplitSeq", "className": "org.apache.lucene.index.TestMultiPassIndexSplitter", "annotations": [], "children": [] }, { "id": "id#testSplitRR(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]", "displayName": "testSplitRR(org.apache.lucene.index.TestMultiPassIndexSplitter)", "methodName": "testSplitRR", "className": "org.apache.lucene.index.TestMultiPassIndexSplitter", "annotations": [], "children": [] } ] } } ] [ "TEST_STARTED", { "description": "id#testSplitSeq(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]" } ] [ "APPEND_STDERR", { "chunk": "57726974696E6720706172742031202E2E2E0A57726974696E6720706172742032202E2E2E0A57726974696E6720706172742033202E2E2E0A446F6E652E0A" } ] [ "TEST_FINISHED", { "executionTime": 979, "startTimestamp": 1343224614831, "description": "id#testSplitSeq(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]" } ] [ "TEST_STARTED", { "description": "id#testSplitRR(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]" } ] [ "APPEND_STDERR", { "chunk": "57726974696E6720706172742031202E2E2E0A57726974696E6720706172742032202E2E2E0A57726974696E6720706172742033202E2E2E0A446F6E652E0A" } ] [ "TEST_FINISHED", { "executionTime": 1350, "startTimestamp": 1343224615815, "description": "id#testSplitRR(org.apache.lucene.index.TestMultiPassIndexSplitter)[0]" } ] [ "SUITE_COMPLETED", { "startTimestamp": 1343224614447, "executionTime": 2777, "description": "id#org.apache.lucene.index.TestMultiPassIndexSplitter[3]" } ] [ "SUITE_STARTED", { "description": { "id": "id#org.apache.lucene.index.TestBalancedSegmentMergePolicy[3]", "displayName": "org.apache.lucene.index.TestBalancedSegmentMergePolicy", "className": "org.apache.lucene.index.TestBalancedSegmentMergePolicy", "annotations": [ { "com.carrotsearch.randomizedtesting.annotations.ThreadLeaks": { "failTestIfLeaking": false, "stackSamples": 10, "linger": 0, "leakedThreadsBelongToSuite": false } }, { "com.carrotsearch.randomizedtesting.annotations.TestMethodProviders": { "value": [ "org.apache.lucene.util.LuceneJUnit3MethodProvider", "com.carrotsearch.randomizedtesting.JUnit4MethodProvider" ] } }, { "org.junit.runner.RunWith": { "value": "com.carrotsearch.randomizedtesting.RandomizedRunner" } } ], "children": [ { "id": "id#testForceMerge(org.apache.lucene.index.TestBalancedSegmentMergePolicy)[0]", "displayName": "testForceMerge(org.apache.lucene.index.TestBalancedSegmentMergePolicy)", "methodName": "testForceMerge", "className": "org.apache.lucene.index.TestBalancedSegmentMergePolicy", "annotations": [], "children": [] } ] } } ] [ "TEST_STARTED", { "description": "id#testForceMerge(org.apache.lucene.index.TestBalancedSegmentMergePolicy)[0]" } ]
        Hide
        Robert Muir added a comment -

        Note also users have reported actual hangs, such as http://www.lucidimagination.com/search/document/79dbebe26ad5c804

        Show
        Robert Muir added a comment - Note also users have reported actual hangs, such as http://www.lucidimagination.com/search/document/79dbebe26ad5c804
        Hide
        Michael McCandless added a comment -

        This does reproduce ... I dug a bit: the test got SerialMergeScheduler, and then BSMP just kept merging the same single segment over and over.

        Rather than debug BSMP I'm inclined to deprecate/remove it ... TieredMergePolicy should accomplish the same thing as BSMP in an NRT setting.

        Show
        Michael McCandless added a comment - This does reproduce ... I dug a bit: the test got SerialMergeScheduler, and then BSMP just kept merging the same single segment over and over. Rather than debug BSMP I'm inclined to deprecate/remove it ... TieredMergePolicy should accomplish the same thing as BSMP in an NRT setting.
        Hide
        Uwe Schindler added a comment -

        If we deprecate it we have to fix this? And then it makes no sense to deprecate it

        We should remove it in 4.0 if it is as good as TieredMergePolicy! In 3.6.2 we should maybe really deprecate it and add a warning that it does not work with SerialMergeSchedulerr (or it should throw ex when started with Serial MergeScheduler)

        Show
        Uwe Schindler added a comment - If we deprecate it we have to fix this? And then it makes no sense to deprecate it We should remove it in 4.0 if it is as good as TieredMergePolicy! In 3.6.2 we should maybe really deprecate it and add a warning that it does not work with SerialMergeSchedulerr (or it should throw ex when started with Serial MergeScheduler)
        Hide
        Michael McCandless added a comment -

        The bug is really in BalancedSegmentMergePolicy, it's just that SMS exacerbates this because it forever merges (ie an addDocument or commit will just run forever). If you use CMS then this busy loop runs in the background...

        I'll remove it in 4.x/5.x and deprecate in 3.6.x.

        Show
        Michael McCandless added a comment - The bug is really in BalancedSegmentMergePolicy, it's just that SMS exacerbates this because it forever merges (ie an addDocument or commit will just run forever). If you use CMS then this busy loop runs in the background... I'll remove it in 4.x/5.x and deprecate in 3.6.x.

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development