Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13298

Fix the leftover L&N files in hadoop-build-tools/src/main/resources/META-INF/

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      After HADOOP-12893, an extra copy of LICENSE.txt and NOTICE.txt exists in hadoop-build-tools/src/main/resources/META-INF/ after build. We should remove it and do it the maven way.

      Details in https://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201606.mbox/%3CCAFS=Wjwx8nMqj6FZXUzZBWRAEoGgfr+_YWL_mkFp4LNuxpggMA@mail.gmail.com%3E

      Thanks Steve Loughran for raising the issue and Sean Busbey for offering the help!

      1. HADOOP-13298.001.patch
        0.4 kB
        Tsuyoshi Ozawa
      2. HADOOP-13298.002.patch
        1 kB
        Tsuyoshi Ozawa
      3. HADOOP-13298.003.patch
        1 kB
        Tsuyoshi Ozawa
      4. HADOOP-13298.004.patch
        3 kB
        Tsuyoshi Ozawa
      5. HADOOP-13298.005.patch
        4 kB
        Tsuyoshi Ozawa

        Issue Links

          Activity

          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Hey Xiao Chen and Sean Busbey, my suggestion is just adding the META-INF dir to .gitignore. What do you think?

          Show
          ozawa Tsuyoshi Ozawa added a comment - Hey Xiao Chen and Sean Busbey , my suggestion is just adding the META-INF dir to .gitignore. What do you think?
          Hide
          busbey Sean Busbey added a comment -

          that only masks the problem in git. it doesn't impact that maven tooling will presume something is amiss, since the build is not supposed to alter the source directories.

          Show
          busbey Sean Busbey added a comment - that only masks the problem in git. it doesn't impact that maven tooling will presume something is amiss, since the build is not supposed to alter the source directories.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Got the point, thanks for your explanation.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Got the point, thanks for your explanation.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Attaching a patch based on the discussion Sean and Xiao had on the mailing list.

          • Moving destination of copy to hadoop-build-tools/target/generated-sources/META-INF/
          • Changing the directory to be included by maven-remote-resources-plugin to hadoop-build-tools/target/generated-sources/META-INF/.
          Show
          ozawa Tsuyoshi Ozawa added a comment - Attaching a patch based on the discussion Sean and Xiao had on the mailing list. Moving destination of copy to hadoop-build-tools/target/generated-sources/META-INF/ Changing the directory to be included by maven-remote-resources-plugin to hadoop-build-tools/target/generated-sources/META-INF/.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Sean Busbey Akira Ajisaka Xiao Chen I tested following points with the latest patch: 1. hadoop-build-tools/src/main/resources/META-INF is not created, 2. tar.gz includes LICENSE.txt and NOTICE.txt. Could you check it?

          Show
          ozawa Tsuyoshi Ozawa added a comment - Sean Busbey Akira Ajisaka Xiao Chen I tested following points with the latest patch: 1. hadoop-build-tools/src/main/resources/META-INF is not created, 2. tar.gz includes LICENSE.txt and NOTICE.txt. Could you check it?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 17s Maven dependency ordering for branch
          +1 mvninstall 8m 1s trunk passed
          +1 compile 8m 7s trunk passed
          +1 mvnsite 10m 54s trunk passed
          +1 mvneclipse 7m 31s trunk passed
          +1 javadoc 5m 1s trunk passed
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 7m 13s the patch passed
          +1 compile 7m 16s the patch passed
          +1 javac 7m 16s the patch passed
          +1 mvnsite 9m 45s the patch passed
          +1 mvneclipse 1m 6s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 javadoc 4m 58s the patch passed
          -1 unit 96m 7s root in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          168m 5s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestRenameWhileOpen



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817361/HADOOP-13298.002.patch
          JIRA Issue HADOOP-13298
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 4f4e4c0d4ecb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 819224d
          Default Java 1.8.0_91
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9966/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9966/testReport/
          modules C: hadoop-build-tools . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9966/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 8m 1s trunk passed +1 compile 8m 7s trunk passed +1 mvnsite 10m 54s trunk passed +1 mvneclipse 7m 31s trunk passed +1 javadoc 5m 1s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 7m 13s the patch passed +1 compile 7m 16s the patch passed +1 javac 7m 16s the patch passed +1 mvnsite 9m 45s the patch passed +1 mvneclipse 1m 6s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 javadoc 4m 58s the patch passed -1 unit 96m 7s root in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 168m 5s Reason Tests Failed junit tests hadoop.hdfs.TestRenameWhileOpen Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817361/HADOOP-13298.002.patch JIRA Issue HADOOP-13298 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 4f4e4c0d4ecb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 819224d Default Java 1.8.0_91 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9966/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9966/testReport/ modules C: hadoop-build-tools . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9966/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Hi Tsuyoshi Ozawa,
          Thanks for working on this!
          Unfortunately, this is what I did when hearing from Sean in the email discussion, and it didn't work.

          2. tar.gz includes LICENSE.txt and NOTICE.txt.

          What we need is to include the L&N in the jars (META-INF sections in particular). This can be verified by running the following script in hadoop-dist dir after mvn package.

          #!/bin/sh 
          
          for f in $(find ./target -name "hadoop*SNAPSHOT.jar"); do
              jar -tf $f | grep "LICENSE" > /dev/null
              RET1=$?
              jar -tf $f | grep "NOTICE" > /dev/null
              RET2=$?
          
              if [ $RET1 -ne 0 -a $RET2 -ne 0 ]; then
                  echo $f "missing LICENSE and NOTICE!";
              elif [ $RET1 -ne 0 ]; then
                  echo $f "missing LICENSE!";
              elif [ $RET2 -ne 0 ]; then
                  echo $f "missing NOTICE!";
              else
                  echo $f "is ok";
              fi
          done
          

          The L&Ns are no longer copied into the jar with this change.

          Show
          xiaochen Xiao Chen added a comment - Hi Tsuyoshi Ozawa , Thanks for working on this! Unfortunately, this is what I did when hearing from Sean in the email discussion, and it didn't work. 2. tar.gz includes LICENSE.txt and NOTICE.txt. What we need is to include the L&N in the jars (META-INF sections in particular). This can be verified by running the following script in hadoop-dist dir after mvn package. #!/bin/sh for f in $(find ./target -name "hadoop*SNAPSHOT.jar" ); do jar -tf $f | grep "LICENSE" > /dev/ null RET1=$? jar -tf $f | grep "NOTICE" > /dev/ null RET2=$? if [ $RET1 -ne 0 -a $RET2 -ne 0 ]; then echo $f "missing LICENSE and NOTICE!" ; elif [ $RET1 -ne 0 ]; then echo $f "missing LICENSE!" ; elif [ $RET2 -ne 0 ]; then echo $f "missing NOTICE!" ; else echo $f "is ok" ; fi done The L&Ns are no longer copied into the jar with this change.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Xiao Chen Thanks for your review and clarification! Attaching a updated patch, v3, to include L&N files in the jar file.

          I confirmed hadoop-dist-3.0.0-alpha1-SNAPSHOT.jar includes L&N files with v3 patch locally.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Xiao Chen Thanks for your review and clarification! Attaching a updated patch, v3, to include L&N files in the jar file. I confirmed hadoop-dist-3.0.0-alpha1-SNAPSHOT.jar includes L&N files with v3 patch locally.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          I also used your script to confirm It's very useful. IIUC, it should be included in dev-support dir. Do you mind filing it as issue and creating a patch?

          Show
          ozawa Tsuyoshi Ozawa added a comment - I also used your script to confirm It's very useful. IIUC, it should be included in dev-support dir. Do you mind filing it as issue and creating a patch?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 26s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 19s Maven dependency ordering for branch
          +1 mvninstall 9m 24s trunk passed
          +1 compile 9m 35s trunk passed
          +1 mvnsite 10m 6s trunk passed
          +1 mvneclipse 1m 57s trunk passed
          +1 javadoc 5m 26s trunk passed
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 7m 7s the patch passed
          +1 compile 6m 54s the patch passed
          +1 javac 6m 54s the patch passed
          +1 mvnsite 9m 20s the patch passed
          +1 mvneclipse 1m 4s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          +1 javadoc 4m 46s the patch passed
          -1 unit 10m 58s root in the patch failed.
          +1 asflicense 0m 27s The patch does not generate ASF License warnings.
          78m 56s



          Reason Tests
          Failed junit tests hadoop.net.TestDNS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817608/HADOOP-13298.003.patch
          JIRA Issue HADOOP-13298
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux be997f05f90f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 438b7c5
          Default Java 1.8.0_91
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9978/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9978/testReport/
          modules C: hadoop-build-tools . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9978/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 19s Maven dependency ordering for branch +1 mvninstall 9m 24s trunk passed +1 compile 9m 35s trunk passed +1 mvnsite 10m 6s trunk passed +1 mvneclipse 1m 57s trunk passed +1 javadoc 5m 26s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 7m 7s the patch passed +1 compile 6m 54s the patch passed +1 javac 6m 54s the patch passed +1 mvnsite 9m 20s the patch passed +1 mvneclipse 1m 4s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 javadoc 4m 46s the patch passed -1 unit 10m 58s root in the patch failed. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 78m 56s Reason Tests Failed junit tests hadoop.net.TestDNS Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817608/HADOOP-13298.003.patch JIRA Issue HADOOP-13298 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux be997f05f90f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 438b7c5 Default Java 1.8.0_91 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9978/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9978/testReport/ modules C: hadoop-build-tools . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9978/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Tsuyoshi Ozawa, Nice catch on the resourcesDirectory!

          I created HADOOP-13374 for the script inclusion, and used that to verify patch 3 here works. +1 (non-binding).

          Show
          xiaochen Xiao Chen added a comment - Thanks Tsuyoshi Ozawa , Nice catch on the resourcesDirectory ! I created HADOOP-13374 for the script inclusion, and used that to verify patch 3 here works. +1 (non-binding).
          Hide
          ajisakaa Akira Ajisaka added a comment -

          The latest patch does not work for me. I ran the followings:

          $ git clean -d -f
          $ dev-support/bin/smart-apply-patch HADOOP-13298
          $ mvn clean
          $ mvn install -DskipTests
          

          That way I cannot see any LICENSE and NOTICE in all the jars. Maybe are the LICENSE and NOTICE still in your main/resources/META-INF/?

          Show
          ajisakaa Akira Ajisaka added a comment - The latest patch does not work for me. I ran the followings: $ git clean -d -f $ dev-support/bin/smart-apply-patch HADOOP-13298 $ mvn clean $ mvn install -DskipTests That way I cannot see any LICENSE and NOTICE in all the jars. Maybe are the LICENSE and NOTICE still in your main/resources/META-INF/?
          Hide
          ozawa Tsuyoshi Ozawa added a comment - - edited

          Akira Ajisaka I hit same issue yesterday, so could you check following things?

          • Please confirm that you're online: plugin must be downloaded.
          • Please check that your local code base is latest one: I didn't take a look deeper, but it doesn't work on old code base.
          Show
          ozawa Tsuyoshi Ozawa added a comment - - edited Akira Ajisaka I hit same issue yesterday, so could you check following things? Please confirm that you're online: plugin must be downloaded. Please check that your local code base is latest one: I didn't take a look deeper, but it doesn't work on old code base.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          $ mvn install -DskipTests

          How to confirm is as follows:

          • mvn package -Pdist -DskipTests

          I think mvn install is not enough.

          Show
          ozawa Tsuyoshi Ozawa added a comment - $ mvn install -DskipTests How to confirm is as follows: mvn package -Pdist -DskipTests I think mvn install is not enough.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          I think mvn install is not enough.

          hmm, I'm wrong. Let me check...

          Show
          ozawa Tsuyoshi Ozawa added a comment - I think mvn install is not enough. hmm, I'm wrong. Let me check...
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Updating a patch to fix.

          • Use maven-resources-plugin instead of copying L&N files by maven-antrun-plugin: it adds directory for resources.
          Show
          ozawa Tsuyoshi Ozawa added a comment - Updating a patch to fix. Use maven-resources-plugin instead of copying L&N files by maven-antrun-plugin: it adds directory for resources.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Sean Busbey Akira Ajisaka Xiao Chen could you try the v4 patch?

          Show
          ozawa Tsuyoshi Ozawa added a comment - Sean Busbey Akira Ajisaka Xiao Chen could you try the v4 patch?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 2m 8s Maven dependency ordering for branch
          +1 mvninstall 7m 15s trunk passed
          +1 compile 7m 17s trunk passed
          +1 mvnsite 9m 47s trunk passed
          +1 mvneclipse 1m 2s trunk passed
          +1 javadoc 4m 53s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 7m 10s the patch passed
          +1 compile 7m 11s the patch passed
          +1 javac 7m 11s the patch passed
          +1 mvnsite 9m 42s the patch passed
          +1 mvneclipse 1m 1s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          +1 javadoc 4m 51s the patch passed
          -1 unit 11m 40s root in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          75m 45s



          Reason Tests
          Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817608/HADOOP-13298.003.patch
          JIRA Issue HADOOP-13298
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux dcf9dad0344a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / dbe97aa
          Default Java 1.8.0_91
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9999/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9999/testReport/
          modules C: hadoop-build-tools . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9999/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 2m 8s Maven dependency ordering for branch +1 mvninstall 7m 15s trunk passed +1 compile 7m 17s trunk passed +1 mvnsite 9m 47s trunk passed +1 mvneclipse 1m 2s trunk passed +1 javadoc 4m 53s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 7m 10s the patch passed +1 compile 7m 11s the patch passed +1 javac 7m 11s the patch passed +1 mvnsite 9m 42s the patch passed +1 mvneclipse 1m 1s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 javadoc 4m 51s the patch passed -1 unit 11m 40s root in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 75m 45s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817608/HADOOP-13298.003.patch JIRA Issue HADOOP-13298 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux dcf9dad0344a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dbe97aa Default Java 1.8.0_91 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9999/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9999/testReport/ modules C: hadoop-build-tools . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9999/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank you for the reply. I was online and the code base was latest.

          Show
          ajisakaa Akira Ajisaka added a comment - Thank you for the reply. I was online and the code base was latest.
          Hide
          xiaochen Xiao Chen added a comment - - edited

          Thanks for the new rev, Tsuyoshi Ozawa.
          My apologies to let patch 3 slip, my guess is I somehow verified in a wrong dir..

          I built using patch 4 (by mvn package), but the jar still seem to have no L&N. I used the exact scripts in HADOOP-13374, and also manually spot checked a jar. (Just vim'ed on the jar file, no L&N under META-INF)

          BTW, I made this change to pass compilation locally. Not sure why jenkins didn't complain about it.

          <directory>../</directory>
          <includes>LICENSE.txt,NOTICE.txt</includes>
          

          to

          <directory>../</directory>
          <includes>
            <include>NOTICE.txt</include>
            <include>LICENSE.txt</include>
          </includes>
          
          Show
          xiaochen Xiao Chen added a comment - - edited Thanks for the new rev, Tsuyoshi Ozawa . My apologies to let patch 3 slip, my guess is I somehow verified in a wrong dir.. I built using patch 4 (by mvn package ), but the jar still seem to have no L&N. I used the exact scripts in HADOOP-13374 , and also manually spot checked a jar. (Just vim'ed on the jar file, no L&N under META-INF) BTW, I made this change to pass compilation locally. Not sure why jenkins didn't complain about it. <directory> ../ </directory> <includes> LICENSE.txt,NOTICE.txt </includes> to <directory> ../ </directory> <includes> <include> NOTICE.txt </include> <include> LICENSE.txt </include> </includes>
          Hide
          ozawa Tsuyoshi Ozawa added a comment - - edited

          Thanks for the feedback, Xiao Chen and Akira Ajisaka.

          I found an interesting result based on your feedbacks:

          • v3 patch seems to only work on Ubuntu.
          • v4 patch seems to only work on Mac.

          Anyway, I will try to create a patch work on both of them.

          Show
          ozawa Tsuyoshi Ozawa added a comment - - edited Thanks for the feedback, Xiao Chen and Akira Ajisaka . I found an interesting result based on your feedbacks: v3 patch seems to only work on Ubuntu. v4 patch seems to only work on Mac. Anyway, I will try to create a patch work on both of them.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Or, my cleaning before trying is not enough. Let me check again.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Or, my cleaning before trying is not enough. Let me check again.
          Hide
          xiaochen Xiao Chen added a comment -

          I tested v4 on Mac, but did mvn clean first and made sure no modified files except the patch before compiling.

          Show
          xiaochen Xiao Chen added a comment - I tested v4 on Mac, but did mvn clean first and made sure no modified files except the patch before compiling.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Xiao Chen Akira Ajisaka thanks for the feedback. I found that that was my mistake - there were old main/src/resources/META-INF, and maven-remote-resource plugin picked up it. I'm sorry that I confuse you.

          Updating a patch to fix:

          • Adding resource section for remote-resources plugin's being able to understand the location of L&N files.
          • Adding resourcesDirectory to add entries for L&N files to remote-resources.xml in jar file.
          Show
          ozawa Tsuyoshi Ozawa added a comment - Xiao Chen Akira Ajisaka thanks for the feedback. I found that that was my mistake - there were old main/src/resources/META-INF , and maven-remote-resource plugin picked up it. I'm sorry that I confuse you. Updating a patch to fix: Adding resource section for remote-resources plugin's being able to understand the location of L&N files. Adding resourcesDirectory to add entries for L&N files to remote-resources.xml in jar file.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 1m 48s Maven dependency ordering for branch
          +1 mvninstall 6m 58s trunk passed
          +1 compile 7m 43s trunk passed
          +1 mvnsite 10m 18s trunk passed
          +1 mvneclipse 1m 35s trunk passed
          +1 javadoc 5m 6s trunk passed
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 7m 31s the patch passed
          +1 compile 7m 58s the patch passed
          +1 javac 7m 58s the patch passed
          +1 mvnsite 10m 31s the patch passed
          +1 mvneclipse 1m 0s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          +1 javadoc 4m 38s the patch passed
          -1 unit 144m 21s root in the patch failed.
          -1 asflicense 0m 24s The patch generated 2 ASF License warnings.
          211m 15s



          Reason Tests
          Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization
            hadoop.yarn.server.TestContainerManagerSecurity



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818133/HADOOP-13298.005.patch
          JIRA Issue HADOOP-13298
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux b4b7d49b4426 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b5ee7db
          Default Java 1.8.0_91
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10008/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10008/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/10008/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-build-tools . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10008/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 1m 48s Maven dependency ordering for branch +1 mvninstall 6m 58s trunk passed +1 compile 7m 43s trunk passed +1 mvnsite 10m 18s trunk passed +1 mvneclipse 1m 35s trunk passed +1 javadoc 5m 6s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 7m 31s the patch passed +1 compile 7m 58s the patch passed +1 javac 7m 58s the patch passed +1 mvnsite 10m 31s the patch passed +1 mvneclipse 1m 0s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 javadoc 4m 38s the patch passed -1 unit 144m 21s root in the patch failed. -1 asflicense 0m 24s The patch generated 2 ASF License warnings. 211m 15s Reason Tests Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization   hadoop.yarn.server.TestContainerManagerSecurity Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818133/HADOOP-13298.005.patch JIRA Issue HADOOP-13298 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux b4b7d49b4426 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b5ee7db Default Java 1.8.0_91 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10008/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10008/testReport/ asflicense https://builds.apache.org/job/PreCommit-HADOOP-Build/10008/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-build-tools . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10008/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Tsuyoshi Ozawa for the new rev! I've verified it to work nicely. This is definitely a better way to do it than the initial HADOOP-12893.

          Failures seem unrelated to this patch, +1 (non-binding).

          Show
          xiaochen Xiao Chen added a comment - Thanks Tsuyoshi Ozawa for the new rev! I've verified it to work nicely. This is definitely a better way to do it than the initial HADOOP-12893 . Failures seem unrelated to this patch, +1 (non-binding).
          Hide
          andrew.wang Andrew Wang added a comment -

          I did a build and ran my earlier script, it just shows the hadoop-project-dist JARs like before.

          +1, thanks for working on this everyone!

          Show
          andrew.wang Andrew Wang added a comment - I did a build and ran my earlier script, it just shows the hadoop-project-dist JARs like before. +1, thanks for working on this everyone!
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank Tsuyoshi Ozawa for updating the patch. +1, the patch works well except for hadoop-project-dist module.

          Show
          ajisakaa Akira Ajisaka added a comment - Thank Tsuyoshi Ozawa for updating the patch. +1, the patch works well except for hadoop-project-dist module.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Thank you for the reviews! Checking this in.

          hadoop-project-dist module doesn't include L&N files before the patch, so it's OK.

          Show
          ozawa Tsuyoshi Ozawa added a comment - Thank you for the reviews! Checking this in. hadoop-project-dist module doesn't include L&N files before the patch, so it's OK.
          Hide
          ozawa Tsuyoshi Ozawa added a comment -

          Committed this to trunk, branch-2, branch-2.8, branch-2,7, branch-2.7.3, and branch-2.6.

          Thanks all for working this!

          Show
          ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk, branch-2, branch-2.8, branch-2,7, branch-2.7.3, and branch-2.6. Thanks all for working this!
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

            People

            • Assignee:
              ozawa Tsuyoshi Ozawa
              Reporter:
              xiaochen Xiao Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development