Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.2, 7.0
    • Component/s: None
    • Labels:
      None
    1. SOLR-9076.patch
      56 kB
      Gregory Chanan
    2. SOLR-9076.patch
      67 kB
      Gregory Chanan
    3. SOLR-9076.patch
      72 kB
      Mark Miller
    4. SOLR-9076.patch
      73 kB
      Mark Miller
    5. SOLR-9076.patch
      69 kB
      Mark Miller
    6. SOLR-9076.patch
      1.0 kB
      Mark Miller
    7. SOLR-9076-Fix-dependencies.patch
      18 kB
      Mark Miller
    8. SOLR-9076-fixnetty.patch
      17 kB
      Gregory Chanan
    9. SOLR-9076-Hack.patch
      259 kB
      Mark Miller

      Issue Links

        Activity

        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        I hit SOLR-7115 for some reason trying to get this to work. A workaround is currently included in this patch.

        Show
        markrmiller@gmail.com Mark Miller added a comment - I hit SOLR-7115 for some reason trying to get this to work. A workaround is currently included in this patch.
        Hide
        mdrob Mike Drob added a comment -

        +/org.apache.htrace/htrace-core = 3.2.0-incubating
        +/org.apache.htrace/htrace-core4 = 4.0.1-incubating

        We don't need both of these, do we? Just the 4.0.1 version, I'd expect.

        Show
        mdrob Mike Drob added a comment - +/org.apache.htrace/htrace-core = 3.2.0-incubating +/org.apache.htrace/htrace-core4 = 4.0.1-incubating We don't need both of these, do we? Just the 4.0.1 version, I'd expect.
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        Seem to need both to avoid class not found issues.

        Show
        markrmiller@gmail.com Mark Miller added a comment - Seem to need both to avoid class not found issues.
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        Okay, that seems to be because there is a little more to sort out. Another patch coming.

        I hit SOLR-7115 for some reason trying to get this to work.

        Oddly, with no changes to what I was doing, even without the workaround, I'm not seeing this today. Need to investigate more.

        Show
        markrmiller@gmail.com Mark Miller added a comment - Okay, that seems to be because there is a little more to sort out. Another patch coming. I hit SOLR-7115 for some reason trying to get this to work. Oddly, with no changes to what I was doing, even without the workaround, I'm not seeing this today. Need to investigate more.
        Hide
        gchanan Gregory Chanan added a comment -

        rebased patch – looks like the TestRecoveryHdfs.java change already went in.

        Show
        gchanan Gregory Chanan added a comment - rebased patch – looks like the TestRecoveryHdfs.java change already went in.
        Hide
        gchanan Gregory Chanan added a comment -

        Here's a patch that passed the test and precommits. Only change from previous patch is it removes the org.htrace versions (which error'ed out in the precommit because they aren't used anymore) and removes the org.htrace license/notice/sha1.

        I will commit this shortly.

        Show
        gchanan Gregory Chanan added a comment - Here's a patch that passed the test and precommits. Only change from previous patch is it removes the org.htrace versions (which error'ed out in the precommit because they aren't used anymore) and removes the org.htrace license/notice/sha1. I will commit this shortly.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit f273cb1b3ae722ee58b289653ad8a3bc5066838f in lucene-solr's branch refs/heads/master from Gregory Chanan
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f273cb1 ]

        SOLR-9076: Update to Hadoop 2.7.2

        Show
        jira-bot ASF subversion and git services added a comment - Commit f273cb1b3ae722ee58b289653ad8a3bc5066838f in lucene-solr's branch refs/heads/master from Gregory Chanan [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f273cb1 ] SOLR-9076 : Update to Hadoop 2.7.2
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit b76f64fdc0559a7b94feb2b97c78c9c151f8f477 in lucene-solr's branch refs/heads/branch_6x from Gregory Chanan
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b76f64f ]

        SOLR-9076: Update to Hadoop 2.7.2

        Show
        jira-bot ASF subversion and git services added a comment - Commit b76f64fdc0559a7b94feb2b97c78c9c151f8f477 in lucene-solr's branch refs/heads/branch_6x from Gregory Chanan [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b76f64f ] SOLR-9076 : Update to Hadoop 2.7.2
        Hide
        gchanan Gregory Chanan added a comment -

        Committed to 7.0 and 6.2. Thanks Mark!

        Show
        gchanan Gregory Chanan added a comment - Committed to 7.0 and 6.2. Thanks Mark!
        Hide
        steve_rowe Steve Rowe added a comment -

        My Jenkins found reproducing nightly failures in the map-reduce contrib of MorphlineBasicMiniMRTest on master and branch_6x - any seed triggers the same error. On master when I rollback to before the commit on this issue, the failure stops.

        Here's the master failure:

        Checking out Revision d730f4a1caddac1f28b60a118904f0c1d5290fa0 (refs/remotes/origin/master)
        [...]
           [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MorphlineBasicMiniMRTest -Dtests.seed=939E67829C51091F -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=uk-UA -Dtests.timezone=Europe/Riga -Dtests.asserts=true -Dtests.file.encoding=UTF-8
           [junit4] ERROR   0.00s J1 | MorphlineBasicMiniMRTest (suite) <<<
           [junit4]    > Throwable #1: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory
           [junit4]    >        at __randomizedtesting.SeedInfo.seed([939E67829C51091F]:0)
           [junit4]    >        at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:334)
           [junit4]    >        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
           [junit4]    >        at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
           [junit4]    >        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
           [junit4]    >        at org.apache.solr.hadoop.hack.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:83)
           [junit4]    >        at org.apache.solr.hadoop.hack.MiniMRCluster.<init>(MiniMRCluster.java:181)
           [junit4]    >        at org.apache.solr.hadoop.hack.MiniMRCluster.<init>(MiniMRCluster.java:169)
           [junit4]    >        at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.setupClass(MorphlineBasicMiniMRTest.java:180)
           [junit4]    >        at java.lang.Thread.run(Thread.java:745)
           [junit4]    > Caused by: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory
           [junit4]    >        at java.lang.Class.forName0(Native Method)
           [junit4]    >        at java.lang.Class.forName(Class.java:348)
           [junit4]    >        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134)
           [junit4]    >        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
           [junit4]    >        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
           [junit4]    >        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
           [junit4]    >        at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:121)
           [junit4]    >        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
           [junit4]    >        at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
           [junit4]    >        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:245)
           [junit4]    >        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
           [junit4]    >        at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
           [junit4]    >        at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:260)
           [junit4]    >        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
           [junit4]    >        at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:316)
           [junit4]    >        ... 31 more
           [junit4]    > Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelPipelineFactory
           [junit4]    >        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
           [junit4]    >        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
           [junit4]    >        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
           [junit4]    >        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
           [junit4]    >        ... 46 more
        
        Show
        steve_rowe Steve Rowe added a comment - My Jenkins found reproducing nightly failures in the map-reduce contrib of MorphlineBasicMiniMRTest on master and branch_6x - any seed triggers the same error. On master when I rollback to before the commit on this issue, the failure stops. Here's the master failure: Checking out Revision d730f4a1caddac1f28b60a118904f0c1d5290fa0 (refs/remotes/origin/master) [...] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MorphlineBasicMiniMRTest -Dtests.seed=939E67829C51091F -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=uk-UA -Dtests.timezone=Europe/Riga -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [junit4] ERROR 0.00s J1 | MorphlineBasicMiniMRTest (suite) <<< [junit4] > Throwable #1: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory [junit4] > at __randomizedtesting.SeedInfo.seed([939E67829C51091F]:0) [junit4] > at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:334) [junit4] > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) [junit4] > at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) [junit4] > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) [junit4] > at org.apache.solr.hadoop.hack.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:83) [junit4] > at org.apache.solr.hadoop.hack.MiniMRCluster.<init>(MiniMRCluster.java:181) [junit4] > at org.apache.solr.hadoop.hack.MiniMRCluster.<init>(MiniMRCluster.java:169) [junit4] > at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.setupClass(MorphlineBasicMiniMRTest.java:180) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > Caused by: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory [junit4] > at java.lang.Class.forName0(Native Method) [junit4] > at java.lang.Class.forName(Class.java:348) [junit4] > at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2134) [junit4] > at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099) [junit4] > at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193) [junit4] > at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219) [junit4] > at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:121) [junit4] > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) [junit4] > at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) [junit4] > at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:245) [junit4] > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) [junit4] > at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) [junit4] > at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:260) [junit4] > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) [junit4] > at org.apache.solr.hadoop.hack.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:316) [junit4] > ... 31 more [junit4] > Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelPipelineFactory [junit4] > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) [junit4] > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) [junit4] > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) [junit4] > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) [junit4] > ... 46 more
        Hide
        gchanan Gregory Chanan added a comment -

        Strange, I wonder why that didn't show up when I ran the tests? Maybe I need a different profile.

        I'll take a look.

        Show
        gchanan Gregory Chanan added a comment - Strange, I wonder why that didn't show up when I ran the tests? Maybe I need a different profile. I'll take a look.
        Hide
        gchanan Gregory Chanan added a comment -

        Strange, I wonder why that didn't show up when I ran the tests? Maybe I need a different profile.

        It's a nightly test. I was able to reproduce it.

        Show
        gchanan Gregory Chanan added a comment - Strange, I wonder why that didn't show up when I ran the tests? Maybe I need a different profile. It's a nightly test. I was able to reproduce it.
        Hide
        gchanan Gregory Chanan added a comment -

        I added org.jboss.netty.netty version 3.2.4.Final and now I get this:

         2> java.lang.reflect.InvocationTargetException
           [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
           [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
           [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
           [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
           [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:401)
           [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
           [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194)
           [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
           [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
           [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
           [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
           [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
           [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
           [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
           [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
           [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
           [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
           [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
           [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
           [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
           [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
           [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
           [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
           [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
           [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
           [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
           [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
           [junit4]   2> 	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.teardownClass(MorphlineBasicMiniMRTest.java:196)
           [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
           [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
           [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
           [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
           [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
           [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
           [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
           [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
           [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
           [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
           [junit4]   2> Caused by: java.lang.NullPointerException
           [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
           [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
           [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
           [junit4]   2> 	... 53 more
        
        Show
        gchanan Gregory Chanan added a comment - I added org.jboss.netty.netty version 3.2.4.Final and now I get this: 2> java.lang.reflect.InvocationTargetException [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498) [junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111) [junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144) [junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:401) [junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430) [junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415) [junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) [junit4] 2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213) [junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594) [junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72) [junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68) [junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171) [junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872) [junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726) [junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705) [junit4] 2> at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.teardownClass(MorphlineBasicMiniMRTest.java:196) [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498) [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) [junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) [junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) [junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) [junit4] 2> at java.lang. Thread .run( Thread .java:745) [junit4] 2> Caused by: java.lang.NullPointerException [junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203) [junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370) [junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729) [junit4] 2> ... 53 more
        Hide
        gchanan Gregory Chanan added a comment -

        Here's a patch that adds the netty dependency. I'm still seeing test failures locally, not sure if they are a product of my environment or not yet.

        Show
        gchanan Gregory Chanan added a comment - Here's a patch that adds the netty dependency. I'm still seeing test failures locally, not sure if they are a product of my environment or not yet.
        Hide
        gchanan Gregory Chanan added a comment -

        Saw this on another machine:

           [junit4]    > Throwable #1: java.io.IOException: Failed on local exception: java.io.IOException: Broken pipe; Host Details : local host is: "ubuntu14-ec2-beefy-slave-03a7.vpc.cloudera.com/172.26.18.223"; destination host is: "ubuntu14-ec2-beefy-slave-03a7.vpc.cloudera.com":53094; 
           [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F57BFC0B596FCED0:FB29480558F9FCDF]:0)
           [junit4]    > 	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:776)
           [junit4]    > 	at org.apache.hadoop.ipc.Client.call(Client.java:1479)
           [junit4]    > 	at org.apache.hadoop.ipc.Client.call(Client.java:1412)
           [junit4]    > 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
           [junit4]    > 	at com.sun.proxy.$Proxy112.getClusterMetrics(Unknown Source)
           [junit4]    > 	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:206)
           [junit4]    > 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
           [junit4]    > 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
           [junit4]    > 	at com.sun.proxy.$Proxy113.getClusterMetrics(Unknown Source)
           [junit4]    > 	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:487)
           [junit4]    > 	at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151)
           [junit4]    > 	at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179)
           [junit4]    > 	at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:247)
           [junit4]    > 	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:748)
           [junit4]    > 	at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:746)
           [junit4]    > 	at java.security.AccessController.doPrivileged(Native Method)
           [junit4]    > 	at javax.security.auth.Subject.doAs(Subject.java:422)
           [junit4]    > 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
           [junit4]    > 	at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:746)
           [junit4]    > 	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:642)
           [junit4]    > 	at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:605)
           [junit4]    > 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
           [junit4]    > 	at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:364)
           [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
           [junit4]    > Caused by: java.io.IOException: Broken pipe
           [junit4]    > 	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
           [junit4]    > 	at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
           [junit4]    > 	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
           [junit4]    > 	at sun.nio.ch.IOUtil.write(IOUtil.java:65)
           [junit4]    > 	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
           [junit4]    > 	at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63)
           [junit4]    > 	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
           [junit4]    > 	at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159)
           [junit4]    > 	at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117)
           [junit4]    > 	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
           [junit4]    > 	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
           [junit4]    > 	at java.io.DataOutputStream.flush(DataOutputStream.java:123)
           [junit4]    > 	at org.apache.hadoop.ipc.Client$Connection$3.run(Client.java:1043)
           [junit4]    > 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
           [junit4]    > 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
           [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           [junit4]    > 	... 1 more
        
        Show
        gchanan Gregory Chanan added a comment - Saw this on another machine: [junit4] > Throwable #1: java.io.IOException: Failed on local exception: java.io.IOException: Broken pipe; Host Details : local host is: "ubuntu14-ec2-beefy-slave-03a7.vpc.cloudera.com/172.26.18.223" ; destination host is: "ubuntu14-ec2-beefy-slave-03a7.vpc.cloudera.com" :53094; [junit4] > at __randomizedtesting.SeedInfo.seed([F57BFC0B596FCED0:FB29480558F9FCDF]:0) [junit4] > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:776) [junit4] > at org.apache.hadoop.ipc.Client.call(Client.java:1479) [junit4] > at org.apache.hadoop.ipc.Client.call(Client.java:1412) [junit4] > at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) [junit4] > at com.sun.proxy.$Proxy112.getClusterMetrics(Unknown Source) [junit4] > at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getClusterMetrics(ApplicationClientProtocolPBClientImpl.java:206) [junit4] > at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) [junit4] > at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) [junit4] > at com.sun.proxy.$Proxy113.getClusterMetrics(Unknown Source) [junit4] > at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:487) [junit4] > at org.apache.hadoop.mapred.ResourceMgrDelegate.getClusterMetrics(ResourceMgrDelegate.java:151) [junit4] > at org.apache.hadoop.mapred.YARNRunner.getClusterMetrics(YARNRunner.java:179) [junit4] > at org.apache.hadoop.mapreduce.Cluster.getClusterStatus(Cluster.java:247) [junit4] > at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:748) [junit4] > at org.apache.hadoop.mapred.JobClient$3.run(JobClient.java:746) [junit4] > at java.security.AccessController.doPrivileged(Native Method) [junit4] > at javax.security.auth.Subject.doAs(Subject.java:422) [junit4] > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) [junit4] > at org.apache.hadoop.mapred.JobClient.getClusterStatus(JobClient.java:746) [junit4] > at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:642) [junit4] > at org.apache.solr.hadoop.MapReduceIndexerTool.run(MapReduceIndexerTool.java:605) [junit4] > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) [junit4] > at org.apache.solr.hadoop.MorphlineBasicMiniMRTest.mrRun(MorphlineBasicMiniMRTest.java:364) [junit4] > at java.lang. Thread .run( Thread .java:745) [junit4] > Caused by: java.io.IOException: Broken pipe [junit4] > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) [junit4] > at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) [junit4] > at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) [junit4] > at sun.nio.ch.IOUtil.write(IOUtil.java:65) [junit4] > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) [junit4] > at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63) [junit4] > at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) [junit4] > at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159) [junit4] > at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117) [junit4] > at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) [junit4] > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) [junit4] > at java.io.DataOutputStream.flush(DataOutputStream.java:123) [junit4] > at org.apache.hadoop.ipc.Client$Connection$3.run(Client.java:1043) [junit4] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] > ... 1 more
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        I took the .hack test classes out to rule those out as a source. I still see the same strange issues around missing netty classes. Which is odd, because this drove moving to Netty 4 as well, so why does it want Netty 3 classes - does it have conflicting Netty reqs? We really want to avoid bringing in more than one version if we can help it, even for tests. But then it seems you got past that, so I'm not sure why I still saw class not found problems with Netty even with the patch.

        Show
        markrmiller@gmail.com Mark Miller added a comment - I took the .hack test classes out to rule those out as a source. I still see the same strange issues around missing netty classes. Which is odd, because this drove moving to Netty 4 as well, so why does it want Netty 3 classes - does it have conflicting Netty reqs? We really want to avoid bringing in more than one version if we can help it, even for tests. But then it seems you got past that, so I'm not sure why I still saw class not found problems with Netty even with the patch.
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        Okay, I tried again with a clean checkout and your patch. Things seem to work on this attempt, except the mapreduce job fails, but I think that is just SOLR-9073.

        I still think it's really strange we need two versions of Netty though...

        Show
        markrmiller@gmail.com Mark Miller added a comment - Okay, I tried again with a clean checkout and your patch. Things seem to work on this attempt, except the mapreduce job fails, but I think that is just SOLR-9073 . I still think it's really strange we need two versions of Netty though...
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        Actually, the first thing I see failing in the MR job is:

        Caused by: java.lang.NoClassDefFoundError: org/bouncycastle/operator/OperatorCreationException
        	at java.lang.Class.forName0(Native Method)
        	at java.lang.Class.forName(Class.java:264)
        	at org.apache.solr.morphlines.cell.SolrCellBuilder$SolrCell.<init>(SolrCellBuilder.java:175)
        	... 21 more
        Caused by: java.lang.ClassNotFoundException: org.bouncycastle.operator.OperatorCreationException
        	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        	... 24 more
        
        Show
        markrmiller@gmail.com Mark Miller added a comment - Actually, the first thing I see failing in the MR job is: Caused by: java.lang.NoClassDefFoundError: org/bouncycastle/operator/OperatorCreationException at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.solr.morphlines.cell.SolrCellBuilder$SolrCell.<init>(SolrCellBuilder.java:175) ... 21 more Caused by: java.lang.ClassNotFoundException: org.bouncycastle.operator.OperatorCreationException at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 24 more
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        It requires a couple dependencies still:
        <dependency org="org.bouncycastle" name="bcpkix-jdk15on" rev="1.47" conf="test"/>
        <dependency org="com.rometools" name="rome" rev="1.6.1" conf="test"/>

        I'm looking at SOLR-9073 - we have to start using a core name for embedded and that causes some random grief.

        Show
        markrmiller@gmail.com Mark Miller added a comment - It requires a couple dependencies still: <dependency org="org.bouncycastle" name="bcpkix-jdk15on" rev="1.47" conf="test"/> <dependency org="com.rometools" name="rome" rev="1.6.1" conf="test"/> I'm looking at SOLR-9073 - we have to start using a core name for embedded and that causes some random grief.
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        Here is a nasty hack patch that gets the test passing (including hacks for SOLR-9073).

        It hacks in a couple missing dependencies, moves around some config files, and allows creating a core when a core.properties file already exists.

        Need to make it all work without those hacks.

        Show
        markrmiller@gmail.com Mark Miller added a comment - Here is a nasty hack patch that gets the test passing (including hacks for SOLR-9073 ). It hacks in a couple missing dependencies, moves around some config files, and allows creating a core when a core.properties file already exists. Need to make it all work without those hacks.
        Hide
        gchanan Gregory Chanan added a comment -

        Which is odd, because this drove moving to Netty 4 as well, so why does it want Netty 3 classes - does it have conflicting Netty reqs?

        Good question. Looks like the dependency is coming from the bkjournal contrib – I wonder if there's some configuration we can use to disable that in the tests.

        Here's mvn dependency:tree output:

        [INFO] org.apache.hadoop.contrib:hadoop-hdfs-bkjournal:jar:2.7.2
        ...
        [INFO] +- org.apache.bookkeeper:bookkeeper-server:jar:4.2.3:compile
        [INFO] |  +- org.slf4j:slf4j-api:jar:1.7.10:compile
        [INFO] |  +- org.slf4j:slf4j-log4j12:jar:1.7.10:compile
        [INFO] |  +- org.jboss.netty:netty:jar:3.2.4.Final:compile
        ...
        
        Show
        gchanan Gregory Chanan added a comment - Which is odd, because this drove moving to Netty 4 as well, so why does it want Netty 3 classes - does it have conflicting Netty reqs? Good question. Looks like the dependency is coming from the bkjournal contrib – I wonder if there's some configuration we can use to disable that in the tests. Here's mvn dependency:tree output: [INFO] org.apache.hadoop.contrib:hadoop-hdfs-bkjournal:jar:2.7.2 ... [INFO] +- org.apache.bookkeeper:bookkeeper-server:jar:4.2.3:compile [INFO] | +- org.slf4j:slf4j-api:jar:1.7.10:compile [INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.10:compile [INFO] | +- org.jboss.netty:netty:jar:3.2.4.Final:compile ...
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        I think this is the patch we need here. It's slightly different versions then I commented above though, so I won't have full confidence again until SOLR-9073 is resolved, and that has turned out to be quite annoying to solve nicely rather than just via hack.

        Show
        markrmiller@gmail.com Mark Miller added a comment - I think this is the patch we need here. It's slightly different versions then I commented above though, so I won't have full confidence again until SOLR-9073 is resolved, and that has turned out to be quite annoying to solve nicely rather than just via hack.
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        Bah, no such luck with using existing Rome and Bouncy Castle dependencies - it requires the versions I have above rather than the ones we have.

        Show
        markrmiller@gmail.com Mark Miller added a comment - Bah, no such luck with using existing Rome and Bouncy Castle dependencies - it requires the versions I have above rather than the ones we have.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 2c96c91dd82ed692a97697ac6de26463ce26ea55 in lucene-solr's branch refs/heads/master from markrmiller
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2c96c91 ]

        SOLR-9076: Add some missing dependencies.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 2c96c91dd82ed692a97697ac6de26463ce26ea55 in lucene-solr's branch refs/heads/master from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2c96c91 ] SOLR-9076 : Add some missing dependencies.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 40d35045eacb42bef3b251f9b737e3975b463b2b in lucene-solr's branch refs/heads/branch_6x from markrmiller
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=40d3504 ]

        SOLR-9076: Add some missing dependencies.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 40d35045eacb42bef3b251f9b737e3975b463b2b in lucene-solr's branch refs/heads/branch_6x from markrmiller [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=40d3504 ] SOLR-9076 : Add some missing dependencies.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 997489f78abe992677692e684458b3d9ac7115bd in lucene-solr's branch refs/heads/branch_6x from Alan Woodward
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=997489f ]

        SOLR-9076: Fix ivy config to pass precommit

        Show
        jira-bot ASF subversion and git services added a comment - Commit 997489f78abe992677692e684458b3d9ac7115bd in lucene-solr's branch refs/heads/branch_6x from Alan Woodward [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=997489f ] SOLR-9076 : Fix ivy config to pass precommit
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit c38cdedbf2100189c068ec5d3f2ff061fd0696ac in lucene-solr's branch refs/heads/master from Alan Woodward
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c38cded ]

        SOLR-9076: Fix ivy config to pass precommit

        Show
        jira-bot ASF subversion and git services added a comment - Commit c38cdedbf2100189c068ec5d3f2ff061fd0696ac in lucene-solr's branch refs/heads/master from Alan Woodward [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c38cded ] SOLR-9076 : Fix ivy config to pass precommit
        Hide
        steve_rowe Steve Rowe added a comment -

        Looks like dependencies are still an issue? From my Jenkins's nightly 6.x job yesterday - these reproduce for me:

        [...]
           [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MorphlineBasicMiniMRTest -Dtests.seed=96C0B1B6D2779A2B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ms-MY -Dtests.timezone=AGT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
           [junit4] ERROR   0.00s J1 | MorphlineBasicMiniMRTest (suite) <<<
           [junit4]    > Throwable #1: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory
        
           [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MorphlineGoLiveMiniMRTest -Dtests.seed=96C0B1B6D2779A2B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=pt -Dtests.timezone=America/Aruba -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
           [junit4] ERROR   0.00s J0 | MorphlineGoLiveMiniMRTest (suite) <<<
           [junit4]    > Throwable #1: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
        

        Getting regular fails on my master nightly runs for the same reason.

        I don't understand why these failures don't show up on ASF or Policeman Jenkins?

        Show
        steve_rowe Steve Rowe added a comment - Looks like dependencies are still an issue? From my Jenkins's nightly 6.x job yesterday - these reproduce for me: [...] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MorphlineBasicMiniMRTest -Dtests.seed=96C0B1B6D2779A2B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ms-MY -Dtests.timezone=AGT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [junit4] ERROR 0.00s J1 | MorphlineBasicMiniMRTest (suite) <<< [junit4] > Throwable #1: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MorphlineGoLiveMiniMRTest -Dtests.seed=96C0B1B6D2779A2B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=pt -Dtests.timezone=America/Aruba -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [junit4] ERROR 0.00s J0 | MorphlineGoLiveMiniMRTest (suite) <<< [junit4] > Throwable #1: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory Getting regular fails on my master nightly runs for the same reason. I don't understand why these failures don't show up on ASF or Policeman Jenkins?
        Hide
        steve_rowe Steve Rowe added a comment -

        I don't understand why these failures don't show up on ASF or Policeman Jenkins?

        I understand now: ASF and my Jenkins don't set tests.haltonfailure to false (except on ASF Clover jobs), so once a test fails in a module, further modules' tests never get run, and since Nightly tests fail with extreme regularity in Solr core, the contrib tests are never being run. (I can't see Policeman Jenkins Groovy script that sets sysprops scripts/linux-random-java8.groovy - Uwe Schindler do you know about Policeman Jenkins's use of tests.haltonfailure?)

        The difference for my Jenkins is how tests are run, via a script that runs 4 Ant processes in parallel, and Solrj and contrib tests are run in a separate Ant process. So Solr core failures don't stop contrib tests from running (though Solrj failures do).

        I've started a nightly manual run with tests.haltonfailure=false, and I also added that to my Jenkins scripts. I'm not really sure why it ever makes for Jenkins to run with it set to true? If things go well (i.e. tests.haltonfailure does what I think it does), I'll add it to all the ASF and Policeman Jenkins jobs too.

        Show
        steve_rowe Steve Rowe added a comment - I don't understand why these failures don't show up on ASF or Policeman Jenkins? I understand now: ASF and my Jenkins don't set tests.haltonfailure to false (except on ASF Clover jobs), so once a test fails in a module, further modules' tests never get run, and since Nightly tests fail with extreme regularity in Solr core, the contrib tests are never being run . (I can't see Policeman Jenkins Groovy script that sets sysprops scripts/linux-random-java8.groovy - Uwe Schindler do you know about Policeman Jenkins's use of tests.haltonfailure ?) The difference for my Jenkins is how tests are run, via a script that runs 4 Ant processes in parallel, and Solrj and contrib tests are run in a separate Ant process. So Solr core failures don't stop contrib tests from running (though Solrj failures do). I've started a nightly manual run with tests.haltonfailure=false , and I also added that to my Jenkins scripts. I'm not really sure why it ever makes for Jenkins to run with it set to true ? If things go well (i.e. tests.haltonfailure does what I think it does), I'll add it to all the ASF and Policeman Jenkins jobs too.
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        Ill take a look.

        Nightly tests fail with extreme regularity in Solr core

        I think i have a jira for this somewhere - I've tried to start tackling it before, but it's a big job.

        Show
        markrmiller@gmail.com Mark Miller added a comment - Ill take a look. Nightly tests fail with extreme regularity in Solr core I think i have a jira for this somewhere - I've tried to start tackling it before, but it's a big job.
        Hide
        thetaphi Uwe Schindler added a comment -

        Policeman Jenkins dies mit der haltonfailure to false. I wanted to try that, too. Especially to also run all precommit and javadocs tasks.

        To do this just add a line in the ant config of Jenkins jobs (don't add to cmd line, there is a separate input box for properties).

        I just ranted to make sure that the junit plugin on Jenkins fails build at very end after parsing the XML test reports. I think it should do this.

        I will try this later.

        Its a good idea.

        Show
        thetaphi Uwe Schindler added a comment - Policeman Jenkins dies mit der haltonfailure to false. I wanted to try that, too. Especially to also run all precommit and javadocs tasks. To do this just add a line in the ant config of Jenkins jobs (don't add to cmd line, there is a separate input box for properties). I just ranted to make sure that the junit plugin on Jenkins fails build at very end after parsing the XML test reports. I think it should do this. I will try this later. Its a good idea.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit a6655a9d39cbfd0f8c85eceee00eab1f64d24023 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a6655a9 ]

        SOLR-9076: disable broken nightly tests MorphlineBasicMiniMRTest and MorphlineGoLiveMiniMRTest via @AwaitsFix

        Show
        jira-bot ASF subversion and git services added a comment - Commit a6655a9d39cbfd0f8c85eceee00eab1f64d24023 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a6655a9 ] SOLR-9076 : disable broken nightly tests MorphlineBasicMiniMRTest and MorphlineGoLiveMiniMRTest via @AwaitsFix
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 85a585c51698edd823769a159856524407cf6456 in lucene-solr's branch refs/heads/master from Steve Rowe
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=85a585c ]

        SOLR-9076: disable broken nightly tests MorphlineBasicMiniMRTest and MorphlineGoLiveMiniMRTest via @AwaitsFix

        Show
        jira-bot ASF subversion and git services added a comment - Commit 85a585c51698edd823769a159856524407cf6456 in lucene-solr's branch refs/heads/master from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=85a585c ] SOLR-9076 : disable broken nightly tests MorphlineBasicMiniMRTest and MorphlineGoLiveMiniMRTest via @AwaitsFix
        Hide
        steve_rowe Steve Rowe added a comment -

        Closing now that the kite/morphlines/map-reduce contribs have been removed, so the test failures are no longer an issue: SOLR-9221.

        Show
        steve_rowe Steve Rowe added a comment - Closing now that the kite/morphlines/map-reduce contribs have been removed, so the test failures are no longer an issue: SOLR-9221 .

          People

          • Assignee:
            markrmiller@gmail.com Mark Miller
            Reporter:
            markrmiller@gmail.com Mark Miller
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development