Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13149

Windows distro build fails on dist-copynativelibs.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: build
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-12892 pulled the dist-copynativelibs script into an external file. The call to this script is failing when running a distro build on Windows.

      1. HADOOP-13149.001.patch
        3 kB
        Chris Nauroth
      2. HADOOP-13149.branch-2.01.patch
        0.9 kB
        Akira Ajisaka

        Issue Links

          Activity

          Hide
          cnauroth Chris Nauroth added a comment -

          I'm attaching a patch. This is similar to the fix I did for HADOOP-12899.

          Show
          cnauroth Chris Nauroth added a comment - I'm attaching a patch. This is similar to the fix I did for HADOOP-12899 .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 27s trunk passed
          +1 compile 0m 7s trunk passed with JDK v1.8.0_91
          +1 compile 0m 8s trunk passed with JDK v1.7.0_95
          +1 mvnsite 0m 11s trunk passed
          +1 mvneclipse 0m 9s trunk passed
          +1 javadoc 0m 8s trunk passed with JDK v1.8.0_91
          +1 javadoc 0m 8s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 8s the patch passed
          +1 compile 0m 6s the patch passed with JDK v1.8.0_91
          +1 javac 0m 6s the patch passed
          +1 compile 0m 6s the patch passed with JDK v1.7.0_95
          +1 javac 0m 6s the patch passed
          +1 mvnsite 0m 8s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 javadoc 0m 6s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 7s the patch passed with JDK v1.7.0_95
          +1 unit 0m 5s hadoop-project-dist in the patch passed with JDK v1.8.0_91.
          +1 unit 0m 6s hadoop-project-dist in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          10m 38s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804046/HADOOP-13149.001.patch
          JIRA Issue HADOOP-13149
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 2f4e24fb91c8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 7be53b6
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9432/testReport/
          modules C: hadoop-project-dist U: hadoop-project-dist
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9432/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 27s trunk passed +1 compile 0m 7s trunk passed with JDK v1.8.0_91 +1 compile 0m 8s trunk passed with JDK v1.7.0_95 +1 mvnsite 0m 11s trunk passed +1 mvneclipse 0m 9s trunk passed +1 javadoc 0m 8s trunk passed with JDK v1.8.0_91 +1 javadoc 0m 8s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 8s the patch passed +1 compile 0m 6s the patch passed with JDK v1.8.0_91 +1 javac 0m 6s the patch passed +1 compile 0m 6s the patch passed with JDK v1.7.0_95 +1 javac 0m 6s the patch passed +1 mvnsite 0m 8s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 0m 6s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 7s the patch passed with JDK v1.7.0_95 +1 unit 0m 5s hadoop-project-dist in the patch passed with JDK v1.8.0_91. +1 unit 0m 6s hadoop-project-dist in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 10m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804046/HADOOP-13149.001.patch JIRA Issue HADOOP-13149 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 2f4e24fb91c8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7be53b6 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9432/testReport/ modules C: hadoop-project-dist U: hadoop-project-dist Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9432/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          The -1 is for lack of tests. The patch changes pom.xml only. I verified manually by running the distro build successfully.

          Show
          cnauroth Chris Nauroth added a comment - The -1 is for lack of tests. The patch changes pom.xml only. I verified manually by running the distro build successfully.
          Hide
          aw Allen Wittenauer added a comment -

          +1

          Show
          aw Allen Wittenauer added a comment - +1
          Hide
          cnauroth Chris Nauroth added a comment -

          Allen, thank you for the review. I committed this to trunk.

          Show
          cnauroth Chris Nauroth added a comment - Allen, thank you for the review. I committed this to trunk.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9767 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9767/)
          HADOOP-13149. Windows distro build fails on dist-copynativelibs. (cnauroth: rev 576e2d1278a584ef8e804832c852c3f8323bb64a)

          • hadoop-project-dist/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9767 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9767/ ) HADOOP-13149 . Windows distro build fails on dist-copynativelibs. (cnauroth: rev 576e2d1278a584ef8e804832c852c3f8323bb64a) hadoop-project-dist/pom.xml
          Hide
          ajisakaa Akira Ajisaka added a comment -

          HADOOP-12892 was backported to branch-2 and branch-2.8, so this commit should be backported as well. What do you think, Chris Nauroth?

          Show
          ajisakaa Akira Ajisaka added a comment - HADOOP-12892 was backported to branch-2 and branch-2.8, so this commit should be backported as well. What do you think, Chris Nauroth ?
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Attaching a patch for the backport.

          Show
          ajisakaa Akira Ajisaka added a comment - Attaching a patch for the backport.
          Hide
          cnauroth Chris Nauroth added a comment -

          Akira Ajisaka, were you able to validate building the distro on Windows for branch-2 and branch-2.8 with this patch? If so, then +1 pending pre-commit. I have reopened the issue and clicked submit patch to get a pre-commit run on the branch-2 patch.

          Show
          cnauroth Chris Nauroth added a comment - Akira Ajisaka , were you able to validate building the distro on Windows for branch-2 and branch-2.8 with this patch? If so, then +1 pending pre-commit. I have reopened the issue and clicked submit patch to get a pre-commit run on the branch-2 patch.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Chris Nauroth for updating the jira. Now I don't have Windows environment, so it takes some time to prepare the environment using MSDN license.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Chris Nauroth for updating the jira. Now I don't have Windows environment, so it takes some time to prepare the environment using MSDN license.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 31s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 2s branch-2 passed
          +1 compile 0m 13s branch-2 passed with JDK v1.8.0_91
          +1 compile 0m 12s branch-2 passed with JDK v1.7.0_101
          +1 mvnsite 0m 12s branch-2 passed
          +1 mvneclipse 0m 10s branch-2 passed
          +1 javadoc 0m 8s branch-2 passed with JDK v1.8.0_91
          +1 javadoc 0m 9s branch-2 passed with JDK v1.7.0_101
          +1 mvninstall 0m 8s the patch passed
          +1 compile 0m 6s the patch passed with JDK v1.8.0_91
          +1 javac 0m 6s the patch passed
          +1 compile 0m 7s the patch passed with JDK v1.7.0_101
          +1 javac 0m 7s the patch passed
          +1 mvnsite 0m 9s the patch passed
          +1 mvneclipse 0m 7s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 0m 6s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 6s the patch passed with JDK v1.7.0_101
          +1 unit 0m 6s hadoop-project-dist in the patch passed with JDK v1.7.0_101.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          10m 56s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:d1c475d
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810990/HADOOP-13149.branch-2.01.patch
          JIRA Issue HADOOP-13149
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux ea7c9ce85832 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / d1c475d
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9791/testReport/
          modules C: hadoop-project-dist U: hadoop-project-dist
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9791/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 31s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 2s branch-2 passed +1 compile 0m 13s branch-2 passed with JDK v1.8.0_91 +1 compile 0m 12s branch-2 passed with JDK v1.7.0_101 +1 mvnsite 0m 12s branch-2 passed +1 mvneclipse 0m 10s branch-2 passed +1 javadoc 0m 8s branch-2 passed with JDK v1.8.0_91 +1 javadoc 0m 9s branch-2 passed with JDK v1.7.0_101 +1 mvninstall 0m 8s the patch passed +1 compile 0m 6s the patch passed with JDK v1.8.0_91 +1 javac 0m 6s the patch passed +1 compile 0m 7s the patch passed with JDK v1.7.0_101 +1 javac 0m 7s the patch passed +1 mvnsite 0m 9s the patch passed +1 mvneclipse 0m 7s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 0m 6s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 6s the patch passed with JDK v1.7.0_101 +1 unit 0m 6s hadoop-project-dist in the patch passed with JDK v1.7.0_101. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 10m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:d1c475d JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810990/HADOOP-13149.branch-2.01.patch JIRA Issue HADOOP-13149 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux ea7c9ce85832 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / d1c475d Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9791/testReport/ modules C: hadoop-project-dist U: hadoop-project-dist Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9791/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Akira Ajisaka, that's no problem. I tested it myself, and it appears to partially fix the build. There still seems to be another problem, probably unrelated, in the call to ctest from hadoop-hdfs-native-client. I'll dig into that and try to get a full end-to-end distro build working.

          I'd expect that we'd need HADOOP-12899 backported too.

          Show
          cnauroth Chris Nauroth added a comment - Akira Ajisaka , that's no problem. I tested it myself, and it appears to partially fix the build. There still seems to be another problem, probably unrelated, in the call to ctest from hadoop-hdfs-native-client. I'll dig into that and try to get a full end-to-end distro build working. I'd expect that we'd need HADOOP-12899 backported too.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Chris for testing the patch. Could you give +1 to this? I agreed with you that HADOOP-12899 should be backported as well.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Chris for testing the patch. Could you give +1 to this? I agreed with you that HADOOP-12899 should be backported as well.
          Hide
          cnauroth Chris Nauroth added a comment -

          +1. I committed this to branch-2 and branch-2.8. Akira Ajisaka, thank you.

          It turns out that we didn't need to bring in HADOOP-12899, because the patch that caused that problem is still only in trunk. We did need to bring in HDFS-10353 though. I'll comment over there.

          Show
          cnauroth Chris Nauroth added a comment - +1. I committed this to branch-2 and branch-2.8. Akira Ajisaka , thank you. It turns out that we didn't need to bring in HADOOP-12899 , because the patch that caused that problem is still only in trunk. We did need to bring in HDFS-10353 though. I'll comment over there.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank you, Chris!

          Show
          ajisakaa Akira Ajisaka added a comment - Thank you, Chris!

            People

            • Assignee:
              cnauroth Chris Nauroth
              Reporter:
              cnauroth Chris Nauroth
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development