Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1795

Upgrade bigtop_toolchain to Maven 3.2.5

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: toolchain
    • Labels:
      None

      Description

      Zeppelin (see BIGTOP-1769 for main Zeppelin Integration JIRA) requires Maven 3.1+ as of https://github.com/apache/incubator-zeppelin/commit/2498e5d

      1. BIGTOP-1795.patch
        3 kB
        Jonathan Kelly
      2. BIGTOP-1795.patch
        3 kB
        Jonathan Kelly

        Issue Links

          Activity

          Hide
          oflebbe Olaf Flebbe added a comment -

          New patch: LGTM, +1 will commit it in a minute.

          Show
          oflebbe Olaf Flebbe added a comment - New patch: LGTM, +1 will commit it in a minute.
          Hide
          jonathak Jonathan Kelly added a comment -

          Added new rebased patch with upgrade to 3.2.5 instead of 3.1.1.

          Show
          jonathak Jonathan Kelly added a comment - Added new rebased patch with upgrade to 3.2.5 instead of 3.1.1.
          Hide
          jonathak Jonathan Kelly added a comment -

          Right, for BIGTOP-1769, I wanted to make the minimal upgrade necessary. Maven 3.2.5 would be fine though. I'll update and rebase my patch.

          Show
          jonathak Jonathan Kelly added a comment - Right, for BIGTOP-1769 , I wanted to make the minimal upgrade necessary. Maven 3.2.5 would be fine though. I'll update and rebase my patch.
          Hide
          warwithin YoungWoo Kim added a comment -

          AFAIK, BIGTOP-1769 requires Maven 3.1+ but if Zeppelin works with 3.2.5, I think 3.2.5 is a good choice for Bigtop. What do you think Jonathan Kelly?

          Show
          warwithin YoungWoo Kim added a comment - AFAIK, BIGTOP-1769 requires Maven 3.1+ but if Zeppelin works with 3.2.5, I think 3.2.5 is a good choice for Bigtop. What do you think Jonathan Kelly ?
          Hide
          oflebbe Olaf Flebbe added a comment -

          A full build with maven 3.2.5 worked for me. Are there any specific reasons why you preferred maven 3.1.x over 3.2.x ?

          Show
          oflebbe Olaf Flebbe added a comment - A full build with maven 3.2.5 worked for me. Are there any specific reasons why you preferred maven 3.1.x over 3.2.x ?
          Hide
          oflebbe Olaf Flebbe added a comment -

          I would like to have 3.2.5 if possible, because of MNG-5672. I am trying a full build with 3.2.5 right now.

          BTW, the patch has to be rebased (trivial change) as well.

          Show
          oflebbe Olaf Flebbe added a comment - I would like to have 3.2.5 if possible, because of MNG-5672 . I am trying a full build with 3.2.5 right now. BTW, the patch has to be rebased (trivial change) as well.
          Hide
          oflebbe Olaf Flebbe added a comment -

          Yep I did try 3.3.3 myself: compiling tez with 3.3.3 this is blocked by MNG-5787 aka MNG-5835.

          Show
          oflebbe Olaf Flebbe added a comment - Yep I did try 3.3.3 myself: compiling tez with 3.3.3 this is blocked by MNG-5787 aka MNG-5835 .
          Hide
          warwithin YoungWoo Kim added a comment -

          Yes, 3.0 for now. To upgrade 3.1+ we need BIGTOP-1877

          Show
          warwithin YoungWoo Kim added a comment - Yes, 3.0 for now. To upgrade 3.1+ we need BIGTOP-1877
          Hide
          jayunit100 jay vyas added a comment -

          Ok, so are we keeping maven 3.0 for now ? Or do we want to keep this guy open?

          Show
          jayunit100 jay vyas added a comment - Ok, so are we keeping maven 3.0 for now ? Or do we want to keep this guy open?
          Hide
          warwithin YoungWoo Kim added a comment -

          FYI,
          Just evaluated the compatibility between Bigtop stack and each stable version of Maven 3.1+. That said, I tested with Maven 3.1.1, 3.2.5 and 3.3.3.
          Building pkgs with Maven 3.1.1 works but required BIGTOP-1877 and also I got same result with 3.2.5. But, Tez build failed with Maven 3.3.3

          Show
          warwithin YoungWoo Kim added a comment - FYI, Just evaluated the compatibility between Bigtop stack and each stable version of Maven 3.1+. That said, I tested with Maven 3.1.1, 3.2.5 and 3.3.3. Building pkgs with Maven 3.1.1 works but required BIGTOP-1877 and also I got same result with 3.2.5. But, Tez build failed with Maven 3.3.3
          Hide
          warwithin YoungWoo Kim added a comment -

          Forgot to mention about the details. With Maven 3.1.1, Crunch build failed with an error:

          java.lang.NoClassDefFoundError: org/sonatype/aether/graph/DependencyFilter
          	at java.lang.Class.getDeclaredConstructors0(Native Method)
          	at java.lang.Class.privateGetDeclaredConstructors(Class.java:2585)
          	at java.lang.Class.getDeclaredConstructors(Class.java:1906)
          	at com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245)
          	at com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99)
          	at com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:653)
          	at com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:863)
          	at com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:790)
          	at com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:278)
          	at com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:210)
          	at com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:986)
          	at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1019)
          	at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:982)
          	at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1032)
          	at org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48)
          	at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:86)
          	at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:55)
          	at com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:70)
          	at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:100)
          	at org.eclipse.sisu.plexus.PlexusLifecycleManager.onProvision(PlexusLifecycleManager.java:133)
          	at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:109)
          	at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:55)
          	at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:68)
          	at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:47)
          	at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
          	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1054)
          	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
          	at com.google.inject.Scopes$1$1.get(Scopes.java:59)
          	at com.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
          	at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:997)
          	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1047)
          	at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:993)
          	at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:82)
          	at org.eclipse.sisu.plexus.LazyPlexusBean.getValue(LazyPlexusBean.java:51)
          	at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:260)
          	at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:240)
          	at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:234)
          	at org.apache.maven.plugins.site.AbstractSiteRenderingMojo.getReports(AbstractSiteRenderingMojo.java:241)
          	at org.apache.maven.plugins.site.SiteMojo.execute(SiteMojo.java:121)
          	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
          	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
          	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
          	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
          	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
          	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
          	at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
          	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
          	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
          	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
          	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
          	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
          	at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Nethod)M
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:606)
          	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
          	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
          	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
          	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
          Caused by: java.lang.ClassNotFoundException: org.sonatype.aether.graph.DependencyFilter
          	at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
          	at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:259)
          	at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:235)
          	at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:227)
          	... 60 more
          

          You can refer to MNG-5482 and https://cwiki.apache.org/confluence/display/MAVEN/AetherClassNotFound for details.

          Show
          warwithin YoungWoo Kim added a comment - Forgot to mention about the details. With Maven 3.1.1, Crunch build failed with an error: java.lang.NoClassDefFoundError: org/sonatype/aether/graph/DependencyFilter at java.lang.Class.getDeclaredConstructors0(Native Method) at java.lang.Class.privateGetDeclaredConstructors(Class.java:2585) at java.lang.Class.getDeclaredConstructors(Class.java:1906) at com.google.inject.spi.InjectionPoint.forConstructorOf(InjectionPoint.java:245) at com.google.inject.internal.ConstructorBindingImpl.create(ConstructorBindingImpl.java:99) at com.google.inject.internal.InjectorImpl.createUninitializedBinding(InjectorImpl.java:653) at com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java:863) at com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java:790) at com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java:278) at com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java:210) at com.google.inject.internal.InjectorImpl.getProviderOrThrow(InjectorImpl.java:986) at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1019) at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:982) at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1032) at org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48) at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:86) at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:55) at com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:70) at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:100) at org.eclipse.sisu.plexus.PlexusLifecycleManager.onProvision(PlexusLifecycleManager.java:133) at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:109) at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:55) at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:68) at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:47) at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1054) at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) at com.google.inject.Scopes$1$1.get(Scopes.java:59) at com.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:997) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1047) at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:993) at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:82) at org.eclipse.sisu.plexus.LazyPlexusBean.getValue(LazyPlexusBean.java:51) at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:260) at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:240) at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:234) at org.apache.maven.plugins.site.AbstractSiteRenderingMojo.getReports(AbstractSiteRenderingMojo.java:241) at org.apache.maven.plugins.site.SiteMojo.execute(SiteMojo.java:121) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59) at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214) at org.apache.maven.cli.MavenCli.main(MavenCli.java:158) at sun.reflect.NativeMethodAccessorImpl.invoke0(Nethod)M at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: java.lang.ClassNotFoundException: org.sonatype.aether.graph.DependencyFilter at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50) at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:259) at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:235) at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:227) ... 60 more You can refer to MNG-5482 and https://cwiki.apache.org/confluence/display/MAVEN/AetherClassNotFound for details.
          Hide
          warwithin YoungWoo Kim added a comment -

          Jonathan Kelly, Unfortunately Apache Crunch 0.10 build does not work with Maven 3.1.1 but the rest of the stack are fine! I filed BIGTOP-1877.

          Let me address upgrading Crunch and then I'll commit your patch so please rebase and re-attach your patch. Thanks!

          Show
          warwithin YoungWoo Kim added a comment - Jonathan Kelly , Unfortunately Apache Crunch 0.10 build does not work with Maven 3.1.1 but the rest of the stack are fine! I filed BIGTOP-1877 . Let me address upgrading Crunch and then I'll commit your patch so please rebase and re-attach your patch. Thanks!
          Hide
          warwithin YoungWoo Kim added a comment - - edited

          Jonathan Kelly, It would be nice if you are reporting back with a result, building the entire stack with Maven 3.1+.

          And please rebase the patch. Thanks!

          Show
          warwithin YoungWoo Kim added a comment - - edited Jonathan Kelly , It would be nice if you are reporting back with a result, building the entire stack with Maven 3.1+. And please rebase the patch. Thanks!
          Hide
          warwithin YoungWoo Kim added a comment -

          From the release history [1], It seems like official release for Java 7+ is version 3.3.x and users usually download the latest stable version from website. what about Maven 3.3.x?

          1. https://maven.apache.org/docs/history.html

          Show
          warwithin YoungWoo Kim added a comment - From the release history [1] , It seems like official release for Java 7+ is version 3.3.x and users usually download the latest stable version from website. what about Maven 3.3.x? 1. https://maven.apache.org/docs/history.html
          Hide
          rnowling RJ Nowling added a comment -

          The version of maven in CentOS EPEL is quite old. We would have to build a separate package for Maven or get it from somewhere else. Unfortunately, it will conflict with the EPEL version which will cause headaches for users. I suggest we talk to upstream Zeppelin.

          Show
          rnowling RJ Nowling added a comment - The version of maven in CentOS EPEL is quite old. We would have to build a separate package for Maven or get it from somewhere else. Unfortunately, it will conflict with the EPEL version which will cause headaches for users. I suggest we talk to upstream Zeppelin.
          Hide
          jonathak Jonathan Kelly added a comment -

          OK, I have not yet tried a full build.

          Show
          jonathak Jonathan Kelly added a comment - OK, I have not yet tried a full build.
          Hide
          cos Konstantin Boudnik added a comment -

          Let's make sure this upgrade doesn't break the rest of the stack. Likely it shouldn't but a full build won't hurt, I suppose.

          Show
          cos Konstantin Boudnik added a comment - Let's make sure this upgrade doesn't break the rest of the stack. Likely it shouldn't but a full build won't hurt, I suppose.

            People

            • Assignee:
              jonathak Jonathan Kelly
              Reporter:
              jonathak Jonathan Kelly
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development