Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 2.7.3
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: build, fs/s3
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Different builds of Hadoop are pulling in wildly different versions of Joda time, depending on what other transitive dependencies are involved. Example: 2.7.3 is somehow picking up Joda time 2.9.4; branch-2.8 is actually behind on 2.8.1. That's going to cause confusion when people upgrade from 2.7.x to 2.8 and find a dependency has got older

      I propose explicitly declaring a dependency on joda-time in s3a, then set the version to 2.9.4; upgrades are things we can manage

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10404 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10404/)
          HADOOP-13541 explicitly declare the Joda time version S3A depends on. (stevel: rev 7fdfcd8a6c9e2dd9b0fb6d4196bc371f6f9a676c)

          • (edit) hadoop-project/pom.xml
          • (edit) hadoop-tools/hadoop-aws/pom.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10404 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10404/ ) HADOOP-13541 explicitly declare the Joda time version S3A depends on. (stevel: rev 7fdfcd8a6c9e2dd9b0fb6d4196bc371f6f9a676c) (edit) hadoop-project/pom.xml (edit) hadoop-tools/hadoop-aws/pom.xml
          Hide
          stevel@apache.org Steve Loughran added a comment -

          thanks, fixed in 2.8+

          Show
          stevel@apache.org Steve Loughran added a comment - thanks, fixed in 2.8+
          Hide
          cnauroth Chris Nauroth added a comment -

          +1. Steve, thank you for the patch. Aaron, thank you for reviewing.

          Show
          cnauroth Chris Nauroth added a comment - +1. Steve, thank you for the patch. Aaron, thank you for reviewing.
          Hide
          andrew.wang Andrew Wang added a comment -

          I'll +1 on faith, I assume you ran mvn dependency plugin to verify.

          Any thoughts about how we can prevent/verify this kind of thing for the future? I don't think it's picked up by our API checking tool.

          Show
          andrew.wang Andrew Wang added a comment - I'll +1 on faith, I assume you ran mvn dependency plugin to verify. Any thoughts about how we can prevent/verify this kind of thing for the future? I don't think it's picked up by our API checking tool.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          -1 on tests are false alarm: this is a build time fix

          Show
          stevel@apache.org Steve Loughran added a comment - -1 on tests are false alarm: this is a build time fix
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 8m 52s Maven dependency ordering for branch
          +1 mvninstall 6m 57s branch-2.8 passed
          +1 compile 5m 47s branch-2.8 passed with JDK v1.8.0_101
          +1 compile 6m 49s branch-2.8 passed with JDK v1.7.0_101
          +1 mvnsite 0m 36s branch-2.8 passed
          +1 mvneclipse 3m 25s branch-2.8 passed
          +1 javadoc 0m 23s branch-2.8 passed with JDK v1.8.0_101
          +1 javadoc 0m 25s branch-2.8 passed with JDK v1.7.0_101
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 0m 22s the patch passed
          +1 compile 5m 42s the patch passed with JDK v1.8.0_101
          +1 javac 5m 42s the patch passed
          +1 compile 6m 44s the patch passed with JDK v1.7.0_101
          +1 javac 6m 44s the patch passed
          +1 mvnsite 0m 34s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 0m 22s the patch passed with JDK v1.8.0_101
          +1 javadoc 0m 26s the patch passed with JDK v1.7.0_101
          +1 unit 0m 10s hadoop-project in the patch passed with JDK v1.7.0_101.
          +1 unit 0m 16s hadoop-aws in the patch passed with JDK v1.7.0_101.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          50m 50s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5af2af1
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825273/HADOOP-13541-branch-2.8-001.patch
          JIRA Issue HADOOP-13541
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 53b90cf99385 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.8 / f871aed
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10359/testReport/
          modules C: hadoop-project hadoop-tools/hadoop-aws U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10359/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 8m 52s Maven dependency ordering for branch +1 mvninstall 6m 57s branch-2.8 passed +1 compile 5m 47s branch-2.8 passed with JDK v1.8.0_101 +1 compile 6m 49s branch-2.8 passed with JDK v1.7.0_101 +1 mvnsite 0m 36s branch-2.8 passed +1 mvneclipse 3m 25s branch-2.8 passed +1 javadoc 0m 23s branch-2.8 passed with JDK v1.8.0_101 +1 javadoc 0m 25s branch-2.8 passed with JDK v1.7.0_101 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 0m 22s the patch passed +1 compile 5m 42s the patch passed with JDK v1.8.0_101 +1 javac 5m 42s the patch passed +1 compile 6m 44s the patch passed with JDK v1.7.0_101 +1 javac 6m 44s the patch passed +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 0m 22s the patch passed with JDK v1.8.0_101 +1 javadoc 0m 26s the patch passed with JDK v1.7.0_101 +1 unit 0m 10s hadoop-project in the patch passed with JDK v1.7.0_101. +1 unit 0m 16s hadoop-aws in the patch passed with JDK v1.7.0_101. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 50m 50s Subsystem Report/Notes Docker Image:yetus/hadoop:5af2af1 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825273/HADOOP-13541-branch-2.8-001.patch JIRA Issue HADOOP-13541 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 53b90cf99385 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.8 / f871aed Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10359/testReport/ modules C: hadoop-project hadoop-tools/hadoop-aws U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10359/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          tested with branch-2.8 against s3a ireland

          Show
          stevel@apache.org Steve Loughran added a comment - tested with branch-2.8 against s3a ireland
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Patch against branch-2.8; it's the first one lagging the 2.7.3 version.

          we should also consider applying this to branch-2.7, to retain version control there in the face of any upgrade of dependencies

          Show
          stevel@apache.org Steve Loughran added a comment - Patch against branch-2.8; it's the first one lagging the 2.7.3 version. we should also consider applying this to branch-2.7, to retain version control there in the face of any upgrade of dependencies

            People

            • Assignee:
              stevel@apache.org Steve Loughran
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development