Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1953

Upgrade bigtop_toolchain to Maven 3.3.3

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: toolchain
    • Labels:
      None

      Description

      In order to upgrade to Spark 1.5.0 once it is released (~Sep 2015), we will need to upgrade the Bigtop toolchain to Maven 3.3.3 due to https://issues.apache.org/jira/browse/SPARK-9521.

        Issue Links

          Activity

          Hide
          oflebbe Olaf Flebbe added a comment -

          While working on BIGTOP-1795 I tried maven 3.3.3.

          Compiling tez with 3.3.3 this is blocked by MNG-5787 aka MNG-5835.

          We cannot switch all components until both problems are resolved

          Show
          oflebbe Olaf Flebbe added a comment - While working on BIGTOP-1795 I tried maven 3.3.3. Compiling tez with 3.3.3 this is blocked by MNG-5787 aka MNG-5835 . We cannot switch all components until both problems are resolved
          Hide
          jonathak Jonathan Kelly added a comment -

          Ah, yes, I'd forgotten about that. What exactly is the error you get when building Tez with Maven 3.3.3? Is there any way that the problem can be solved within the Tez project, or does it actually need to be solved in Maven with these two MNG JIRAs?

          Show
          jonathak Jonathan Kelly added a comment - Ah, yes, I'd forgotten about that. What exactly is the error you get when building Tez with Maven 3.3.3? Is there any way that the problem can be solved within the Tez project, or does it actually need to be solved in Maven with these two MNG JIRAs?
          Hide
          warwithin YoungWoo Kim added a comment -

          TEZ-2560 blocked upgrading Maven 3.3+

          But Tez 6.2 will include TEZ-2560 and the Tez community will release 6.2 soon. I'll file a JIRA for upgrading Tez. Thanks!

          Show
          warwithin YoungWoo Kim added a comment - TEZ-2560 blocked upgrading Maven 3.3+ But Tez 6.2 will include TEZ-2560 and the Tez community will release 6.2 soon. I'll file a JIRA for upgrading Tez. Thanks!
          Hide
          warwithin YoungWoo Kim added a comment - - edited

          Tez 0.6.2 is out now! I filed BIGTOP-1964 and attached a patch.

          Show
          warwithin YoungWoo Kim added a comment - - edited Tez 0.6.2 is out now! I filed BIGTOP-1964 and attached a patch.
          Hide
          oflebbe Olaf Flebbe added a comment -

          Patch (we really should reengineer bigtop_toolchain, its a mess)

          Show
          oflebbe Olaf Flebbe added a comment - Patch (we really should reengineer bigtop_toolchain, its a mess)
          Hide
          evans_ye Evans Ye added a comment -

          +1. Tested by running a full build against new maven version. The 3.3 works fine with tez 0.6.2. Please commit or I'll do it later. Thanks.

          Show
          evans_ye Evans Ye added a comment - +1. Tested by running a full build against new maven version. The 3.3 works fine with tez 0.6.2. Please commit or I'll do it later. Thanks.
          Hide
          oflebbe Olaf Flebbe added a comment -

          applied and resolved. Thanks Evans Ye

          Can you create new "official" docker containers with current toolchain already applied ?

          Show
          oflebbe Olaf Flebbe added a comment - applied and resolved. Thanks Evans Ye Can you create new "official" docker containers with current toolchain already applied ?
          Hide
          evans_ye Evans Ye added a comment -

          Sure. Actually I've created new images: http://bigtop01.cloudera.org:8080/view/Docker/job/Docker-Toolchain-Trunk/
          But a thing came up into my mind is that how do we named these images?
          Right now we don't tag images with the release number, which I admit it isn't a good practice:
          https://hub.docker.com/r/bigtop/slaves/tags/

          I think we can name them as the following scheme:

          • bigtop/slaves:trunk-centos-6
          • bigtop/slaves:1.0.0-centos-6

          Or probably you have a better design in BIGTOP-1996, then we can discuss there.

          Show
          evans_ye Evans Ye added a comment - Sure. Actually I've created new images: http://bigtop01.cloudera.org:8080/view/Docker/job/Docker-Toolchain-Trunk/ But a thing came up into my mind is that how do we named these images? Right now we don't tag images with the release number, which I admit it isn't a good practice: https://hub.docker.com/r/bigtop/slaves/tags/ I think we can name them as the following scheme: bigtop/slaves:trunk-centos-6 bigtop/slaves:1.0.0-centos-6 Or probably you have a better design in BIGTOP-1996 , then we can discuss there.

            People

            • Assignee:
              oflebbe Olaf Flebbe
              Reporter:
              jonathak Jonathan Kelly
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development