Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Existing node label file system store implementation uses append to write edit logs. However, some file system doesn't support append, we need add an implementation to support such non-appendable file systems as well.

      1. YARN-4405.1.patch
        19 kB
        Wangda Tan
      2. YARN-4405.2.patch
        19 kB
        Wangda Tan
      3. YARN-4405.3.patch
        20 kB
        Wangda Tan
      4. YARN-4405.4.patch
        23 kB
        Wangda Tan

        Issue Links

          Activity

          Hide
          leftnoteasy Wangda Tan added a comment -

          Attached initial patch for review.

          Show
          leftnoteasy Wangda Tan added a comment - Attached initial patch for review.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 8m 48s trunk passed
          +1 compile 2m 27s trunk passed with JDK v1.8.0_66
          +1 compile 2m 31s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 1m 51s trunk passed
          +1 mvneclipse 0m 45s trunk passed
          +1 findbugs 4m 26s trunk passed
          +1 javadoc 1m 57s trunk passed with JDK v1.8.0_66
          +1 javadoc 4m 36s trunk passed with JDK v1.7.0_85
          +1 mvninstall 2m 1s the patch passed
          +1 compile 2m 40s the patch passed with JDK v1.8.0_66
          +1 javac 2m 40s the patch passed
          +1 compile 2m 44s the patch passed with JDK v1.7.0_85
          +1 javac 2m 44s the patch passed
          -1 checkstyle 0m 35s Patch generated 7 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 264, now 267).
          +1 mvnsite 2m 7s the patch passed
          +1 mvneclipse 0m 52s the patch passed
          -1 whitespace 0m 0s The patch has 14 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 findbugs 1m 51s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common introduced 1 new FindBugs issues.
          +1 javadoc 2m 4s the patch passed with JDK v1.8.0_66
          +1 javadoc 4m 25s the patch passed with JDK v1.7.0_85
          -1 unit 0m 33s hadoop-yarn-api in the patch failed with JDK v1.8.0_66.
          +1 unit 2m 23s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
          -1 unit 68m 52s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 0m 32s hadoop-yarn-api in the patch failed with JDK v1.7.0_85.
          -1 unit 0m 27s hadoop-yarn-common in the patch failed with JDK v1.7.0_85.
          -1 unit 0m 40s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85.
          +1 asflicense 0m 24s Patch does not generate ASF License warnings.
          126m 48s



          Reason Tests
          FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
            org.apache.hadoop.yarn.nodelabels.NonAppendableFSNodeLabelStore.writeNewMirror() does not release lock on all paths At NonAppendableFSNodeLabelStore.java:on all paths At NonAppendableFSNodeLabelStore.java:[line 85]
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_85 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775135/YARN-4405.1.patch
          JIRA Issue YARN-4405
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 373cee38c397 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 58f6f54
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_85.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9830/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Max memory used 76MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9830/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 8m 48s trunk passed +1 compile 2m 27s trunk passed with JDK v1.8.0_66 +1 compile 2m 31s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 51s trunk passed +1 mvneclipse 0m 45s trunk passed +1 findbugs 4m 26s trunk passed +1 javadoc 1m 57s trunk passed with JDK v1.8.0_66 +1 javadoc 4m 36s trunk passed with JDK v1.7.0_85 +1 mvninstall 2m 1s the patch passed +1 compile 2m 40s the patch passed with JDK v1.8.0_66 +1 javac 2m 40s the patch passed +1 compile 2m 44s the patch passed with JDK v1.7.0_85 +1 javac 2m 44s the patch passed -1 checkstyle 0m 35s Patch generated 7 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 264, now 267). +1 mvnsite 2m 7s the patch passed +1 mvneclipse 0m 52s the patch passed -1 whitespace 0m 0s The patch has 14 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 1m 51s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common introduced 1 new FindBugs issues. +1 javadoc 2m 4s the patch passed with JDK v1.8.0_66 +1 javadoc 4m 25s the patch passed with JDK v1.7.0_85 -1 unit 0m 33s hadoop-yarn-api in the patch failed with JDK v1.8.0_66. +1 unit 2m 23s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 68m 52s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 0m 32s hadoop-yarn-api in the patch failed with JDK v1.7.0_85. -1 unit 0m 27s hadoop-yarn-common in the patch failed with JDK v1.7.0_85. -1 unit 0m 40s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 126m 48s Reason Tests FindBugs module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common   org.apache.hadoop.yarn.nodelabels.NonAppendableFSNodeLabelStore.writeNewMirror() does not release lock on all paths At NonAppendableFSNodeLabelStore.java:on all paths At NonAppendableFSNodeLabelStore.java: [line 85] JDK v1.8.0_66 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields   hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_85 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775135/YARN-4405.1.patch JIRA Issue YARN-4405 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 373cee38c397 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 58f6f54 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/new-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_85.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9830/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9830/console This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Attached ver.2 patch fixed test failures and findbug warnings.

          Show
          leftnoteasy Wangda Tan added a comment - Attached ver.2 patch fixed test failures and findbug warnings.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 8m 57s trunk passed
          +1 compile 2m 33s trunk passed with JDK v1.8.0_66
          +1 compile 2m 29s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 1m 57s trunk passed
          +1 mvneclipse 0m 45s trunk passed
          +1 findbugs 4m 44s trunk passed
          +1 javadoc 1m 49s trunk passed with JDK v1.8.0_66
          +1 javadoc 4m 24s trunk passed with JDK v1.7.0_85
          +1 mvninstall 1m 45s the patch passed
          +1 compile 2m 38s the patch passed with JDK v1.8.0_66
          +1 javac 2m 38s the patch passed
          +1 compile 2m 37s the patch passed with JDK v1.7.0_85
          +1 javac 2m 37s the patch passed
          -1 checkstyle 0m 34s Patch generated 7 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 264, now 267).
          +1 mvnsite 2m 1s the patch passed
          +1 mvneclipse 0m 42s the patch passed
          -1 whitespace 0m 0s The patch has 14 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 findbugs 4m 57s the patch passed
          +1 javadoc 1m 48s the patch passed with JDK v1.8.0_66
          +1 javadoc 4m 2s the patch passed with JDK v1.7.0_85
          -1 unit 0m 27s hadoop-yarn-api in the patch failed with JDK v1.8.0_66.
          +1 unit 2m 6s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
          -1 unit 61m 3s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 0m 29s hadoop-yarn-api in the patch failed with JDK v1.7.0_85.
          +1 unit 2m 21s hadoop-yarn-common in the patch passed with JDK v1.7.0_85.
          -1 unit 61m 4s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85.
          +1 asflicense 0m 25s Patch does not generate ASF License warnings.
          178m 48s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_85 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775408/YARN-4405.2.patch
          JIRA Issue YARN-4405
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2bcff9ba3564 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6b9a5be
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9845/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Max memory used 76MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9845/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 8m 57s trunk passed +1 compile 2m 33s trunk passed with JDK v1.8.0_66 +1 compile 2m 29s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 57s trunk passed +1 mvneclipse 0m 45s trunk passed +1 findbugs 4m 44s trunk passed +1 javadoc 1m 49s trunk passed with JDK v1.8.0_66 +1 javadoc 4m 24s trunk passed with JDK v1.7.0_85 +1 mvninstall 1m 45s the patch passed +1 compile 2m 38s the patch passed with JDK v1.8.0_66 +1 javac 2m 38s the patch passed +1 compile 2m 37s the patch passed with JDK v1.7.0_85 +1 javac 2m 37s the patch passed -1 checkstyle 0m 34s Patch generated 7 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 264, now 267). +1 mvnsite 2m 1s the patch passed +1 mvneclipse 0m 42s the patch passed -1 whitespace 0m 0s The patch has 14 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 4m 57s the patch passed +1 javadoc 1m 48s the patch passed with JDK v1.8.0_66 +1 javadoc 4m 2s the patch passed with JDK v1.7.0_85 -1 unit 0m 27s hadoop-yarn-api in the patch failed with JDK v1.8.0_66. +1 unit 2m 6s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 61m 3s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 0m 29s hadoop-yarn-api in the patch failed with JDK v1.7.0_85. +1 unit 2m 21s hadoop-yarn-common in the patch passed with JDK v1.7.0_85. -1 unit 61m 4s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 178m 48s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_85 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775408/YARN-4405.2.patch JIRA Issue YARN-4405 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2bcff9ba3564 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6b9a5be findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-YARN-Build/9845/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9845/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9845/console This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Attached ver.3 addressed test failures.

          Show
          leftnoteasy Wangda Tan added a comment - Attached ver.3 addressed test failures.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 8m 1s trunk passed
          +1 compile 2m 1s trunk passed with JDK v1.8.0_66
          +1 compile 2m 15s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 29s trunk passed
          +1 mvnsite 1m 40s trunk passed
          +1 mvneclipse 0m 40s trunk passed
          +1 findbugs 4m 2s trunk passed
          +1 javadoc 1m 29s trunk passed with JDK v1.8.0_66
          +1 javadoc 3m 53s trunk passed with JDK v1.7.0_85
          +1 mvninstall 1m 32s the patch passed
          +1 compile 1m 58s the patch passed with JDK v1.8.0_66
          +1 javac 1m 58s the patch passed
          +1 compile 2m 14s the patch passed with JDK v1.7.0_85
          +1 javac 2m 14s the patch passed
          -1 checkstyle 0m 29s Patch generated 7 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 264, now 267).
          +1 mvnsite 1m 41s the patch passed
          +1 mvneclipse 0m 41s the patch passed
          -1 whitespace 0m 0s The patch has 14 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 4m 29s the patch passed
          +1 javadoc 1m 30s the patch passed with JDK v1.8.0_66
          +1 javadoc 3m 55s the patch passed with JDK v1.7.0_85
          -1 unit 0m 24s hadoop-yarn-api in the patch failed with JDK v1.8.0_66.
          +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
          -1 unit 59m 59s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 0m 26s hadoop-yarn-api in the patch failed with JDK v1.7.0_85.
          +1 unit 2m 14s hadoop-yarn-common in the patch passed with JDK v1.7.0_85.
          -1 unit 60m 50s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          170m 44s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_85 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775452/YARN-4405.3.patch
          JIRA Issue YARN-4405
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux fa44533f50fb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e8bd1ba
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9846/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Max memory used 76MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9846/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 8m 1s trunk passed +1 compile 2m 1s trunk passed with JDK v1.8.0_66 +1 compile 2m 15s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 40s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 4m 2s trunk passed +1 javadoc 1m 29s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 53s trunk passed with JDK v1.7.0_85 +1 mvninstall 1m 32s the patch passed +1 compile 1m 58s the patch passed with JDK v1.8.0_66 +1 javac 1m 58s the patch passed +1 compile 2m 14s the patch passed with JDK v1.7.0_85 +1 javac 2m 14s the patch passed -1 checkstyle 0m 29s Patch generated 7 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 264, now 267). +1 mvnsite 1m 41s the patch passed +1 mvneclipse 0m 41s the patch passed -1 whitespace 0m 0s The patch has 14 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 4m 29s the patch passed +1 javadoc 1m 30s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 55s the patch passed with JDK v1.7.0_85 -1 unit 0m 24s hadoop-yarn-api in the patch failed with JDK v1.8.0_66. +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 59m 59s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 0m 26s hadoop-yarn-api in the patch failed with JDK v1.7.0_85. +1 unit 2m 14s hadoop-yarn-common in the patch passed with JDK v1.7.0_85. -1 unit 60m 50s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 170m 44s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_85 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775452/YARN-4405.3.patch JIRA Issue YARN-4405 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux fa44533f50fb 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e8bd1ba findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-YARN-Build/9846/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9846/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9846/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Hi Wangda Tan
          Thank you for starting this ticket.

          Couple of minor points:
          1. Can mirror.new.tmp also pre-exists is the system (may be RM crashed earlier just after creating this file)
          2. I feel may be close() need to be overridden. Otherwise for NonAppendableFSNodeLabelStore, we will still have base implementation to close editlogOs which is not needed.

              IOUtils.cleanup(LOG, fs, editlogOs);
          

          3. May be some logs at the end of NonAppendableFSNodeLabelStore#recover can help to provide some info that file creation/deletion/modification is done successfully.

          Show
          sunilg Sunil G added a comment - Hi Wangda Tan Thank you for starting this ticket. Couple of minor points: 1. Can mirror.new.tmp also pre-exists is the system (may be RM crashed earlier just after creating this file) 2. I feel may be close() need to be overridden. Otherwise for NonAppendableFSNodeLabelStore, we will still have base implementation to close editlogOs which is not needed. IOUtils.cleanup(LOG, fs, editlogOs); 3. May be some logs at the end of NonAppendableFSNodeLabelStore#recover can help to provide some info that file creation/deletion/modification is done successfully.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks Sunil G.

          Addressed #2/#3. For #1, I was using

          fs.create(newTmpPath, true);
          

          So old tmp file will be always overwritten by the new one.

          Attaching ver.4 patch.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks Sunil G . Addressed #2/#3. For #1, I was using fs.create(newTmpPath, true ); So old tmp file will be always overwritten by the new one. Attaching ver.4 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 9m 1s trunk passed
          +1 compile 11m 24s trunk passed with JDK v1.8.0_66
          +1 compile 9m 22s trunk passed with JDK v1.7.0_85
          +1 checkstyle 1m 4s trunk passed
          +1 mvnsite 2m 48s trunk passed
          +1 mvneclipse 0m 57s trunk passed
          +1 findbugs 6m 1s trunk passed
          +1 javadoc 2m 32s trunk passed with JDK v1.8.0_66
          +1 javadoc 5m 12s trunk passed with JDK v1.7.0_85
          +1 mvninstall 3m 15s the patch passed
          +1 compile 9m 26s the patch passed with JDK v1.8.0_66
          +1 javac 9m 26s the patch passed
          +1 compile 9m 26s the patch passed with JDK v1.7.0_85
          +1 javac 9m 26s the patch passed
          -1 checkstyle 1m 3s Patch generated 7 new checkstyle issues in root (total was 264, now 267).
          +1 mvnsite 2m 47s the patch passed
          +1 mvneclipse 0m 57s the patch passed
          -1 whitespace 0m 0s The patch has 13 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 7m 6s the patch passed
          +1 javadoc 2m 49s the patch passed with JDK v1.8.0_66
          +1 javadoc 5m 22s the patch passed with JDK v1.7.0_85
          -1 unit 8m 45s hadoop-common in the patch failed with JDK v1.8.0_66.
          +1 unit 0m 26s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
          +1 unit 2m 2s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
          -1 unit 61m 1s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 7m 49s hadoop-common in the patch failed with JDK v1.7.0_85.
          +1 unit 0m 27s hadoop-yarn-api in the patch passed with JDK v1.7.0_85.
          +1 unit 2m 14s hadoop-yarn-common in the patch passed with JDK v1.7.0_85.
          -1 unit 60m 53s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85.
          +1 asflicense 0m 24s Patch does not generate ASF License warnings.
          236m 25s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.ha.TestZKFailoverController
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens
          JDK v1.7.0_85 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.TestClientRMTokens



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775611/YARN-4405.4.patch
          JIRA Issue YARN-4405
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux a4b7902ca6dc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 62e9348
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9849/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: .
          Max memory used 75MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9849/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 9m 1s trunk passed +1 compile 11m 24s trunk passed with JDK v1.8.0_66 +1 compile 9m 22s trunk passed with JDK v1.7.0_85 +1 checkstyle 1m 4s trunk passed +1 mvnsite 2m 48s trunk passed +1 mvneclipse 0m 57s trunk passed +1 findbugs 6m 1s trunk passed +1 javadoc 2m 32s trunk passed with JDK v1.8.0_66 +1 javadoc 5m 12s trunk passed with JDK v1.7.0_85 +1 mvninstall 3m 15s the patch passed +1 compile 9m 26s the patch passed with JDK v1.8.0_66 +1 javac 9m 26s the patch passed +1 compile 9m 26s the patch passed with JDK v1.7.0_85 +1 javac 9m 26s the patch passed -1 checkstyle 1m 3s Patch generated 7 new checkstyle issues in root (total was 264, now 267). +1 mvnsite 2m 47s the patch passed +1 mvneclipse 0m 57s the patch passed -1 whitespace 0m 0s The patch has 13 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 7m 6s the patch passed +1 javadoc 2m 49s the patch passed with JDK v1.8.0_66 +1 javadoc 5m 22s the patch passed with JDK v1.7.0_85 -1 unit 8m 45s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 0m 26s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 2m 2s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 61m 1s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 7m 49s hadoop-common in the patch failed with JDK v1.7.0_85. +1 unit 0m 27s hadoop-yarn-api in the patch passed with JDK v1.7.0_85. +1 unit 2m 14s hadoop-yarn-common in the patch passed with JDK v1.7.0_85. -1 unit 60m 53s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 236m 25s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_85 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775611/YARN-4405.4.patch JIRA Issue YARN-4405 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux a4b7902ca6dc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 62e9348 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-YARN-Build/9849/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9849/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: . Max memory used 75MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9849/console This message was automatically generated.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Sunil G, please let me know if you have any other comments on latest patch.

          Thanks,

          Show
          leftnoteasy Wangda Tan added a comment - Sunil G , please let me know if you have any other comments on latest patch. Thanks,
          Hide
          sunilg Sunil G added a comment -

          Thanks Wangda Tan. Latest patch looks good.
          +1.

          Show
          sunilg Sunil G added a comment - Thanks Wangda Tan . Latest patch looks good. +1.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8917 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8917/)
          YARN-4405. Support node label store in non-appendable file system. (jianhe: rev 755dda8dd8bb23864abc752bad506f223fcac010)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestFileSystemNodeLabelsStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/DummyCommonNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/test/java/org/apache/hadoop/yarn/conf/TestYarnConfigurationFields.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/NullRMNodeLabelsManager.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/FileSystemNodeLabelsStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/NodeLabelsStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8917 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8917/ ) YARN-4405 . Support node label store in non-appendable file system. (jianhe: rev 755dda8dd8bb23864abc752bad506f223fcac010) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestFileSystemNodeLabelsStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/DummyCommonNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/test/java/org/apache/hadoop/yarn/conf/TestYarnConfigurationFields.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/NullRMNodeLabelsManager.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/FileSystemNodeLabelsStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/NodeLabelsStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          Hide
          drankye Kai Zheng added a comment -

          YARN-4419 was just opened. Please help check and comment it. Thanks.

          Show
          drankye Kai Zheng added a comment - YARN-4419 was just opened. Please help check and comment it. Thanks.
          Hide
          sunilg Sunil G added a comment -

          Yes, its a compile pblm

          sb_work2/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestFileSystemNodeLabelsStore.java:[75,15] cannot find symbol
          [ERROR] symbol:   class NonAppendableFSNodeLabelStore
          [ERROR] location: class org.apache.hadoop.yarn.nodelabels.TestFileSystemNodeLabelsStore
          [ERROR] -> [Help 1]
          

          Usually such issues will be handled with an addenum patch in the same ticket. So it could be handled here, Wangda Tan, could you please help for the same.

          Show
          sunilg Sunil G added a comment - Yes, its a compile pblm sb_work2/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestFileSystemNodeLabelsStore.java:[75,15] cannot find symbol [ERROR] symbol: class NonAppendableFSNodeLabelStore [ERROR] location: class org.apache.hadoop.yarn.nodelabels.TestFileSystemNodeLabelsStore [ERROR] -> [Help 1] Usually such issues will be handled with an addenum patch in the same ticket. So it could be handled here, Wangda Tan , could you please help for the same.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks for reporting, Kai Zheng, Sunil G.

          Fixing this issue now.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks for reporting, Kai Zheng , Sunil G . Fixing this issue now.
          Hide
          jianhe Jian He added a comment -

          sorry, missed a file while committing, fixed now.

          Show
          jianhe Jian He added a comment - sorry, missed a file while committing, fixed now.
          Hide
          jianhe Jian He added a comment -

          Committed to trunk and branch-2, thanks Wangda Tan and Sunil G for reviewing the patch.

          Show
          jianhe Jian He added a comment - Committed to trunk and branch-2, thanks Wangda Tan and Sunil G for reviewing the patch.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks Jian He, I was committing just now and found it is already fixed.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks Jian He , I was committing just now and found it is already fixed.
          Hide
          sunilg Sunil G added a comment -

          Thanks Jian He and Wangda Tan. Its fine now.

          Show
          sunilg Sunil G added a comment - Thanks Jian He and Wangda Tan . Its fine now.
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #663 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/663/)
          YARN-4405. Support node label store in non-appendable file system. (jianhe: rev 755dda8dd8bb23864abc752bad506f223fcac010)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/NodeLabelsStore.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestFileSystemNodeLabelsStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/test/java/org/apache/hadoop/yarn/conf/TestYarnConfigurationFields.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/NullRMNodeLabelsManager.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/DummyCommonNodeLabelsManager.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/FileSystemNodeLabelsStore.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #663 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/663/ ) YARN-4405 . Support node label store in non-appendable file system. (jianhe: rev 755dda8dd8bb23864abc752bad506f223fcac010) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/NodeLabelsStore.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/TestFileSystemNodeLabelsStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/test/java/org/apache/hadoop/yarn/conf/TestYarnConfigurationFields.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/nodelabels/NullRMNodeLabelsManager.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/nodelabels/DummyCommonNodeLabelsManager.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/FileSystemNodeLabelsStore.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/nodelabels/CommonNodeLabelsManager.java
          Hide
          leftnoteasy Wangda Tan added a comment -

          Committed to branch-2.8.

          Show
          leftnoteasy Wangda Tan added a comment - Committed to branch-2.8.

            People

            • Assignee:
              leftnoteasy Wangda Tan
              Reporter:
              leftnoteasy Wangda Tan
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development