Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: build
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      A new test resource dfs.hosts.json was added to HDFS-9005 for better readability. Given json file doesn't allow comments, it violates ASF license. To address this, we can add the file to rat exclusions list in the pom.xml.

      1. HDFS-10221.patch
        0.7 kB
        Ming Ma
      2. HDFS-10221-2.patch
        0.7 kB
        Ming Ma

        Issue Links

          Activity

          Hide
          mingma Ming Ma added a comment -

          Thanks all!

          Show
          mingma Ming Ma added a comment - Thanks all!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9530 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9530/)
          HDFS-10221. Add .json to the rat exclusions. Contributed by Ming Ma. (aajisaka: rev 32c0c3ecdf72e89a63f4aee5e75d1c5a12714b89)

          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9530 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9530/ ) HDFS-10221 . Add .json to the rat exclusions. Contributed by Ming Ma. (aajisaka: rev 32c0c3ecdf72e89a63f4aee5e75d1c5a12714b89) hadoop-hdfs-project/hadoop-hdfs/pom.xml
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed v2 patch to trunk, branch-2, and branch-2.8. Thanks all for contributing to this issue.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed v2 patch to trunk, branch-2, and branch-2.8. Thanks all for contributing to this issue.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1

          Show
          ajisakaa Akira Ajisaka added a comment - +1
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Licenses are happy

          +1

          Show
          stevel@apache.org Steve Loughran added a comment - Licenses are happy +1
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 48s trunk passed
          +1 compile 0m 56s trunk passed with JDK v1.8.0_74
          +1 compile 0m 48s trunk passed with JDK v1.7.0_95
          +1 mvnsite 0m 56s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 javadoc 1m 20s trunk passed with JDK v1.8.0_74
          +1 javadoc 2m 1s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 54s the patch passed
          +1 compile 0m 54s the patch passed with JDK v1.8.0_74
          +1 javac 0m 54s the patch passed
          +1 compile 0m 46s the patch passed with JDK v1.7.0_95
          +1 javac 0m 46s the patch passed
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 javadoc 1m 18s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 59s the patch passed with JDK v1.7.0_95
          -1 unit 86m 17s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
          -1 unit 76m 51s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 35s Patch does not generate ASF License warnings.
          187m 27s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.server.blockmanagement.TestBlockManagerSafeMode
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush
            hadoop.hdfs.TestRollingUpgrade



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795886/HDFS-10221-2.patch
          JIRA Issue HDFS-10221
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 428cdf11ba54 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8286270
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14993/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14993/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14993/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14993/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14993/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14993/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 48s trunk passed +1 compile 0m 56s trunk passed with JDK v1.8.0_74 +1 compile 0m 48s trunk passed with JDK v1.7.0_95 +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 14s trunk passed +1 javadoc 1m 20s trunk passed with JDK v1.8.0_74 +1 javadoc 2m 1s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 54s the patch passed +1 compile 0m 54s the patch passed with JDK v1.8.0_74 +1 javac 0m 54s the patch passed +1 compile 0m 46s the patch passed with JDK v1.7.0_95 +1 javac 0m 46s the patch passed +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 1m 18s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 59s the patch passed with JDK v1.7.0_95 -1 unit 86m 17s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 76m 51s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 35s Patch does not generate ASF License warnings. 187m 27s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.blockmanagement.TestBlockManagerSafeMode   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.hdfs.TestRollingUpgrade Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795886/HDFS-10221-2.patch JIRA Issue HDFS-10221 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 428cdf11ba54 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8286270 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14993/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14993/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14993/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14993/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14993/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14993/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          +1, LGTM
          Pending Jenkins for commit. Triggered one now.

          Show
          vinayrpet Vinayakumar B added a comment - +1, LGTM Pending Jenkins for commit. Triggered one now.
          Hide
          mingma Ming Ma added a comment -

          Thanks all. Here is the new patch that uses wildcard.

          Show
          mingma Ming Ma added a comment - Thanks all. Here is the new patch that uses wildcard.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Interesting comments hack, but that would mean that every use of a mapper would need to be tuned; adding the pattern avoids that problem. It's not legal JSON any more.

          I'd like to see that wildcard in instead

          Show
          stevel@apache.org Steve Loughran added a comment - Interesting comments hack, but that would mean that every use of a mapper would need to be tuned; adding the pattern avoids that problem. It's not legal JSON any more. I'd like to see that wildcard in instead
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 28s trunk passed
          +1 compile 0m 37s trunk passed with JDK v1.8.0_74
          +1 compile 0m 40s trunk passed with JDK v1.7.0_95
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 javadoc 1m 4s trunk passed with JDK v1.8.0_74
          +1 javadoc 1m 44s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 35s the patch passed with JDK v1.8.0_74
          +1 javac 0m 35s the patch passed
          +1 compile 0m 38s the patch passed with JDK v1.7.0_95
          +1 javac 0m 38s the patch passed
          +1 mvnsite 0m 49s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 javadoc 0m 59s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 43s the patch passed with JDK v1.7.0_95
          -1 unit 57m 9s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
          -1 unit 53m 4s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          129m 52s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.hdfs.TestRollingUpgrade
            hadoop.cli.TestHDFSCLI
          JDK v1.7.0_95 Failed junit tests hadoop.cli.TestHDFSCLI



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795647/HDFS-10221.patch
          JIRA Issue HDFS-10221
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 14173cef7340 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0050fa5
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14982/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14982/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14982/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14982/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14982/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14982/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 28s trunk passed +1 compile 0m 37s trunk passed with JDK v1.8.0_74 +1 compile 0m 40s trunk passed with JDK v1.7.0_95 +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 javadoc 1m 4s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 44s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 45s the patch passed +1 compile 0m 35s the patch passed with JDK v1.8.0_74 +1 javac 0m 35s the patch passed +1 compile 0m 38s the patch passed with JDK v1.7.0_95 +1 javac 0m 38s the patch passed +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 0m 59s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 43s the patch passed with JDK v1.7.0_95 -1 unit 57m 9s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 53m 4s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 129m 52s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.TestRollingUpgrade   hadoop.cli.TestHDFSCLI JDK v1.7.0_95 Failed junit tests hadoop.cli.TestHDFSCLI Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795647/HDFS-10221.patch JIRA Issue HDFS-10221 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 14173cef7340 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0050fa5 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14982/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14982/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14982/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14982/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14982/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14982/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          qwertymaniac Harsh J added a comment -

          It appears we can also request the parser to allow comments as a feature, depending on the parser, for ex. this change and JIRA: LENS-729 / https://issues.apache.org/jira/secure/attachment/12750264/LENS-729.patch

          This would help preserve licenses vs. excluding it, just in case.

          Show
          qwertymaniac Harsh J added a comment - It appears we can also request the parser to allow comments as a feature, depending on the parser, for ex. this change and JIRA: LENS-729 / https://issues.apache.org/jira/secure/attachment/12750264/LENS-729.patch This would help preserve licenses vs. excluding it, just in case.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Adding <exclude>**/*.json</exclude> is fine.

          Show
          ajisakaa Akira Ajisaka added a comment - Adding <exclude>**/*.json</exclude> is fine.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          ...Does anyone know a way to tell rat to ignore *.json? That would stop the problem coming back

          Show
          stevel@apache.org Steve Loughran added a comment - ...Does anyone know a way to tell rat to ignore *.json? That would stop the problem coming back
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1 pending Jenkins.

          Show
          ajisakaa Akira Ajisaka added a comment - +1 pending Jenkins.
          Hide
          mingma Ming Ma added a comment -

          Here is the patch.

          Show
          mingma Ming Ma added a comment - Here is the patch.

            People

            • Assignee:
              mingma Ming Ma
              Reporter:
              mingma Ming Ma
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development