Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11163

Mover should move the file blocks to default storage once policy is unset

    Details

    • Hadoop Flags:
      Reviewed

      Description

      HDFS-9534 added new API in FileSystem to unset the storage policy. Once policy is unset blocks should move back to the default storage policy.

      Currently mover is not moving file blocks which have zero storage ID

            // currently we ignore files with unspecified storage policy
            if (policyId == HdfsConstants.BLOCK_STORAGE_POLICY_ID_UNSPECIFIED) {
              return;
            }
      
      1. HDFS-11163-001.patch
        4 kB
        Surendra Singh Lilhore
      2. HDFS-11163-002.patch
        4 kB
        Surendra Singh Lilhore
      3. HDFS-11163-003.patch
        14 kB
        Surendra Singh Lilhore
      4. HDFS-11163-004.patch
        13 kB
        Surendra Singh Lilhore
      5. HDFS-11163-005.patch
        13 kB
        Surendra Singh Lilhore
      6. HDFS-11163-006.patch
        10 kB
        Surendra Singh Lilhore
      7. HDFS-11163-007.patch
        11 kB
        Surendra Singh Lilhore
      8. HDFS-11163-branch-2.001.patch
        11 kB
        Surendra Singh Lilhore
      9. HDFS-11163-branch-2.002.patch
        12 kB
        Surendra Singh Lilhore
      10. HDFS-11163-branch-2.003.patch
        11 kB
        Surendra Singh Lilhore
      11. temp-YARN-6278.HDFS-11163.patch
        19 kB
        Wangda Tan

        Issue Links

          Activity

          Hide
          arpitagarwal Arpit Agarwal added a comment -
          Show
          arpitagarwal Arpit Agarwal added a comment - Nice find Surendra Singh Lilhore .
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Thanks Arpit Agarwal

          Attached initial patch
          Please review..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Thanks Arpit Agarwal Attached initial patch Please review..
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 30s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 50s trunk passed
          +1 javadoc 0m 43s trunk passed
          +1 mvninstall 0m 52s the patch passed
          +1 compile 0m 51s the patch passed
          +1 javac 0m 51s the patch passed
          -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 49 unchanged - 0 fixed = 53 total (was 49)
          +1 mvnsite 1m 0s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 4s the patch passed
          +1 javadoc 0m 39s the patch passed
          -1 unit 72m 47s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          93m 7s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840302/HDFS-11163-001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c718782a8bf7 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 005850b
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17650/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17650/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17650/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17650/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 30s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 50s trunk passed +1 javadoc 0m 43s trunk passed +1 mvninstall 0m 52s the patch passed +1 compile 0m 51s the patch passed +1 javac 0m 51s the patch passed -0 checkstyle 0m 29s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 49 unchanged - 0 fixed = 53 total (was 49) +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 4s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 72m 47s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 93m 7s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840302/HDFS-11163-001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c718782a8bf7 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 005850b Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17650/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17650/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17650/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17650/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Fixed checkstyle warnings
          Please review...

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Fixed checkstyle warnings Please review...
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 16s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 52s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 40s trunk passed
          +1 javadoc 0m 39s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          +1 checkstyle 0m 22s the patch passed
          +1 mvnsite 0m 48s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 45s the patch passed
          +1 javadoc 0m 37s the patch passed
          +1 unit 62m 25s hadoop-hdfs in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          81m 8s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840556/HDFS-11163-002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7b230e0f6305 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 07825f2
          Default Java 1.8.0_111
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17664/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17664/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 16s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 40s trunk passed +1 javadoc 0m 39s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 37s the patch passed +1 unit 62m 25s hadoop-hdfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 81m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840556/HDFS-11163-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7b230e0f6305 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 07825f2 Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17664/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17664/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Tsz Wo Nicholas Sze, can you please review the patch ?

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Tsz Wo Nicholas Sze , can you please review the patch ?
          Hide
          cnauroth Chris Nauroth added a comment -

          Surendra Singh Lilhore, thank you for the patch. This looks correct to me. One thing I'm unsure about is the potential impact on performance of Mover. It will require an additional getStoragePolicy RPC per file with the default storage policy, whereas previously there was no RPC for those files. Unfortunately, I don't see a way to avoid that, at least not with the current APIs, because that's how we resolve inheritance of storage policies from parent paths. I would prefer to get an opinion from Tsz Wo Nicholas Sze.

          Show
          cnauroth Chris Nauroth added a comment - Surendra Singh Lilhore , thank you for the patch. This looks correct to me. One thing I'm unsure about is the potential impact on performance of Mover. It will require an additional getStoragePolicy RPC per file with the default storage policy, whereas previously there was no RPC for those files. Unfortunately, I don't see a way to avoid that, at least not with the current APIs, because that's how we resolve inheritance of storage policies from parent paths. I would prefer to get an opinion from Tsz Wo Nicholas Sze .
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          > ... It will require an additional getStoragePolicy RPC per file with the default storage policy, whereas previously there was no RPC for those files. ...

          Indeed, the additional RPC is not needed since HdfsLocatedFileStatus already has the resolved storage policy. We don't need to call getStoragePolicy again.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - > ... It will require an additional getStoragePolicy RPC per file with the default storage policy, whereas previously there was no RPC for those files. ... Indeed, the additional RPC is not needed since HdfsLocatedFileStatus already has the resolved storage policy. We don't need to call getStoragePolicy again.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Thanks Chris Nauroth and Tsz Wo Nicholas Sze for comments..

          Indeed, the additional RPC is not needed since HdfsLocatedFileStatus already has the resolved storage policy. We don't need to call getStoragePolicy again.

          Yes, HdfsLocatedFileStatus has resolved storage policy. I am calling getStoragePolicy because it will give default policy in case of BLOCK_STORAGE_POLICY_ID_UNSPECIFIED.

            public BlockStoragePolicy getPolicy(byte id) {
              // id == 0 means policy not specified.
              return id == 0? getDefaultPolicy(): policies[id];
            }
          

          we can add one API to get default policy from namenode, so we can avoid getStoragePolicy RPC per file.

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Thanks Chris Nauroth and Tsz Wo Nicholas Sze for comments.. Indeed, the additional RPC is not needed since HdfsLocatedFileStatus already has the resolved storage policy. We don't need to call getStoragePolicy again. Yes, HdfsLocatedFileStatus has resolved storage policy. I am calling getStoragePolicy because it will give default policy in case of BLOCK_STORAGE_POLICY_ID_UNSPECIFIED . public BlockStoragePolicy getPolicy( byte id) { // id == 0 means policy not specified. return id == 0? getDefaultPolicy(): policies[id]; } we can add one API to get default policy from namenode, so we can avoid getStoragePolicy RPC per file.
          Hide
          cnauroth Chris Nauroth added a comment -

          we can add one API to get default policy from namenode, so we can avoid getStoragePolicy RPC per file.

          If we went in this direction, then maybe it could fit into getServerDefaults.

          However, I think the challenge is that it really needs to be sensitive to path. If the storage policy is unspecified at an inode, then the real effective storage policy might be resolved via inheritance from the inode's ancestry. I can't think of a clever way to completely avoid additional RPCs, though perhaps a new API could help reduce it by memoizing results from the ancestry on the client side.

          This is tricky. Sorry I missed it in the review of HDFS-9534.

          Show
          cnauroth Chris Nauroth added a comment - we can add one API to get default policy from namenode, so we can avoid getStoragePolicy RPC per file. If we went in this direction, then maybe it could fit into getServerDefaults . However, I think the challenge is that it really needs to be sensitive to path. If the storage policy is unspecified at an inode, then the real effective storage policy might be resolved via inheritance from the inode's ancestry. I can't think of a clever way to completely avoid additional RPCs, though perhaps a new API could help reduce it by memoizing results from the ancestry on the client side. This is tricky. Sorry I missed it in the review of HDFS-9534 .
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          If the storage policy is unspecified at an inode, then the real effective storage policy might be resolved via inheritance from the inode's ancestry.

          As Tsz Wo Nicholas Sze mentioned, HdfsLocatedFileStatus already has the resolved storage policy. (If inode policy unspecified, it will inherit from ancestors).

          This issue is only applicable if ancestors policy also unspecified, in this case status.getStoragePolicy() will give BLOCK_STORAGE_POLICY_ID_UNSPECIFIED.

          We need namenode default storage policy which we can use in case of {{ BLOCK_STORAGE_POLICY_ID_UNSPECIFIED}}. We can add new field in FsServerDefaults for default policy.

          So we can do something like this...

                final byte policyId = status.getStoragePolicy();
                if (policyId == HdfsConstants.BLOCK_STORAGE_POLICY_ID_UNSPECIFIED) {
                  policyId = dfs.getServerDefaults().getDefaultStoragePolicy();
                }
          
          Show
          surendrasingh Surendra Singh Lilhore added a comment - If the storage policy is unspecified at an inode, then the real effective storage policy might be resolved via inheritance from the inode's ancestry. As Tsz Wo Nicholas Sze mentioned, HdfsLocatedFileStatus already has the resolved storage policy. (If inode policy unspecified, it will inherit from ancestors). This issue is only applicable if ancestors policy also unspecified, in this case status.getStoragePolicy() will give BLOCK_STORAGE_POLICY_ID_UNSPECIFIED. We need namenode default storage policy which we can use in case of {{ BLOCK_STORAGE_POLICY_ID_UNSPECIFIED}}. We can add new field in FsServerDefaults for default policy. So we can do something like this... final byte policyId = status.getStoragePolicy(); if (policyId == HdfsConstants.BLOCK_STORAGE_POLICY_ID_UNSPECIFIED) { policyId = dfs.getServerDefaults().getDefaultStoragePolicy(); }
          Hide
          cnauroth Chris Nauroth added a comment -

          ...HdfsLocatedFileStatus already has the resolved storage policy.

          Thank you for clarifying. I missed this. In that case, I think it makes sense to put it in FsServerDefaults.

          Show
          cnauroth Chris Nauroth added a comment - ...HdfsLocatedFileStatus already has the resolved storage policy. Thank you for clarifying. I missed this. In that case, I think it makes sense to put it in FsServerDefaults .
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Thanks Chris Nauroth.
          Attached updated patch.
          Please review..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Thanks Chris Nauroth . Attached updated patch. Please review..
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 18s Maven dependency ordering for branch
          +1 mvninstall 14m 39s trunk passed
          +1 compile 9m 49s trunk passed
          +1 checkstyle 1m 48s trunk passed
          +1 mvnsite 3m 4s trunk passed
          +1 mvneclipse 0m 51s trunk passed
          +1 findbugs 5m 18s trunk passed
          +1 javadoc 1m 58s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 2m 14s the patch passed
          +1 compile 11m 3s the patch passed
          +1 cc 11m 3s the patch passed
          +1 javac 11m 3s the patch passed
          -0 checkstyle 1m 36s root: The patch generated 7 new + 472 unchanged - 2 fixed = 479 total (was 474)
          +1 mvnsite 2m 43s the patch passed
          +1 mvneclipse 0m 58s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 5m 51s the patch passed
          +1 javadoc 2m 8s the patch passed
          +1 unit 9m 43s hadoop-common in the patch passed.
          +1 unit 1m 12s hadoop-hdfs-client in the patch passed.
          -1 unit 89m 40s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 50s The patch does not generate ASF License warnings.
          167m 27s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker
          Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
            org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12847399/HDFS-11163-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 4b483ec3da3b 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d3170f9
          Default Java 1.8.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18169/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18169/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18169/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18169/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 18s Maven dependency ordering for branch +1 mvninstall 14m 39s trunk passed +1 compile 9m 49s trunk passed +1 checkstyle 1m 48s trunk passed +1 mvnsite 3m 4s trunk passed +1 mvneclipse 0m 51s trunk passed +1 findbugs 5m 18s trunk passed +1 javadoc 1m 58s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 14s the patch passed +1 compile 11m 3s the patch passed +1 cc 11m 3s the patch passed +1 javac 11m 3s the patch passed -0 checkstyle 1m 36s root: The patch generated 7 new + 472 unchanged - 2 fixed = 479 total (was 474) +1 mvnsite 2m 43s the patch passed +1 mvneclipse 0m 58s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 51s the patch passed +1 javadoc 2m 8s the patch passed +1 unit 9m 43s hadoop-common in the patch passed. +1 unit 1m 12s hadoop-hdfs-client in the patch passed. -1 unit 89m 40s hadoop-hdfs in the patch failed. +1 asflicense 0m 50s The patch does not generate ASF License warnings. 167m 27s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean   org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12847399/HDFS-11163-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 4b483ec3da3b 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d3170f9 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18169/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18169/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18169/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18169/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Chris Nauroth would you review this ?

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Chris Nauroth would you review this ?
          Hide
          vinayrpet Vinayakumar B added a comment -

          Updated looks good.

          Have few comments.
          1.

          +  public static final byte DEFAULT_STORAGE_POLICY_ID = 7;

          For Non-HDFS FileSystems let the default value be 0 itself. Current default value of 7 is defined in HDFS' BlockStoragePolicySuite, this is anyway taken for FsServerDefaults in FSNameSystem.

          So, I think no need of Constant DEFAULT_STORAGE_POLICY_ID = 7 in CommonConfigurationsKeys. Default instance of FsServerDefaults can have 0 as the value. Let 7 be taken from FSNameSystem for HDFS specific implementation.

          2. FsServerDefaulls#getDefaultStoragePolicy rename to FsServerDefaults#getDefaultStoragePolicyId(). Similarly storagepolicy to storagepolicyId

          3. In Hdfs.proto + optional uint32 policyId = 9 [default = 7]; also can return 0 as default value.

          +1 once above comments addressed.

          Show
          vinayrpet Vinayakumar B added a comment - Updated looks good. Have few comments. 1. + public static final byte DEFAULT_STORAGE_POLICY_ID = 7; For Non-HDFS FileSystems let the default value be 0 itself. Current default value of 7 is defined in HDFS' BlockStoragePolicySuite, this is anyway taken for FsServerDefaults in FSNameSystem. So, I think no need of Constant DEFAULT_STORAGE_POLICY_ID = 7 in CommonConfigurationsKeys. Default instance of FsServerDefaults can have 0 as the value. Let 7 be taken from FSNameSystem for HDFS specific implementation. 2. FsServerDefaulls#getDefaultStoragePolicy rename to FsServerDefaults#getDefaultStoragePolicyId() . Similarly storagepolicy to storagepolicyId 3. In Hdfs.proto + optional uint32 policyId = 9 [default = 7] ; also can return 0 as default value. +1 once above comments addressed.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Thanks Vinayakumar B for review..
          Attached updated patch..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Thanks Vinayakumar B for review.. Attached updated patch..
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 20s Maven dependency ordering for branch
          +1 mvninstall 16m 16s trunk passed
          -1 compile 11m 46s root in trunk failed.
          +1 checkstyle 2m 22s trunk passed
          +1 mvnsite 3m 16s trunk passed
          +1 mvneclipse 1m 0s trunk passed
          +1 findbugs 5m 56s trunk passed
          +1 javadoc 2m 17s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 2m 31s the patch passed
          -1 compile 9m 51s root in the patch failed.
          -1 cc 9m 51s root in the patch failed.
          -1 javac 9m 51s root in the patch failed.
          -0 checkstyle 2m 17s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426)
          +1 mvnsite 2m 53s the patch passed
          +1 mvneclipse 0m 58s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 6m 11s the patch passed
          +1 javadoc 2m 11s the patch passed
          -1 unit 9m 11s hadoop-common in the patch failed.
          +1 unit 1m 8s hadoop-hdfs-client in the patch passed.
          -1 unit 76m 12s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 46s The patch does not generate ASF License warnings.
          159m 23s



          Reason Tests
          Failed junit tests hadoop.ha.TestZKFailoverController
          Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855684/HDFS-11163-004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux bdd7a61f1cfc 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 747bafa
          Default Java 1.8.0_121
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/branch-compile-root.txt
          findbugs v3.0.0
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-compile-root.txt
          cc https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-compile-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18519/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18519/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 20s Maven dependency ordering for branch +1 mvninstall 16m 16s trunk passed -1 compile 11m 46s root in trunk failed. +1 checkstyle 2m 22s trunk passed +1 mvnsite 3m 16s trunk passed +1 mvneclipse 1m 0s trunk passed +1 findbugs 5m 56s trunk passed +1 javadoc 2m 17s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 2m 31s the patch passed -1 compile 9m 51s root in the patch failed. -1 cc 9m 51s root in the patch failed. -1 javac 9m 51s root in the patch failed. -0 checkstyle 2m 17s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426) +1 mvnsite 2m 53s the patch passed +1 mvneclipse 0m 58s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 6m 11s the patch passed +1 javadoc 2m 11s the patch passed -1 unit 9m 11s hadoop-common in the patch failed. +1 unit 1m 8s hadoop-hdfs-client in the patch passed. -1 unit 76m 12s hadoop-hdfs in the patch failed. +1 asflicense 0m 46s The patch does not generate ASF License warnings. 159m 23s Reason Tests Failed junit tests hadoop.ha.TestZKFailoverController Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855684/HDFS-11163-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux bdd7a61f1cfc 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 747bafa Default Java 1.8.0_121 compile https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/branch-compile-root.txt findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-compile-root.txt cc https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-compile-root.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18519/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18519/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18519/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 13m 37s trunk passed
          -1 compile 10m 9s root in trunk failed.
          +1 checkstyle 2m 0s trunk passed
          +1 mvnsite 2m 43s trunk passed
          +1 mvneclipse 0m 56s trunk passed
          +1 findbugs 4m 55s trunk passed
          +1 javadoc 2m 6s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 2m 2s the patch passed
          -1 compile 10m 1s root in the patch failed.
          -1 cc 10m 1s root in the patch failed.
          -1 javac 10m 1s root in the patch failed.
          -0 checkstyle 2m 2s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426)
          +1 mvnsite 2m 40s the patch passed
          +1 mvneclipse 0m 52s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 5m 20s the patch passed
          +1 javadoc 1m 56s the patch passed
          -1 unit 8m 16s hadoop-common in the patch failed.
          +1 unit 1m 7s hadoop-hdfs-client in the patch passed.
          -1 unit 69m 28s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 34s The patch does not generate ASF License warnings.
          142m 44s



          Reason Tests
          Failed junit tests hadoop.security.TestKDiag
            hadoop.hdfs.server.datanode.TestDataNodeUUID
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855684/HDFS-11163-004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 5884a3667503 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3749152
          Default Java 1.8.0_121
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/branch-compile-root.txt
          findbugs v3.0.0
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-compile-root.txt
          cc https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-compile-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18536/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18536/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 13m 37s trunk passed -1 compile 10m 9s root in trunk failed. +1 checkstyle 2m 0s trunk passed +1 mvnsite 2m 43s trunk passed +1 mvneclipse 0m 56s trunk passed +1 findbugs 4m 55s trunk passed +1 javadoc 2m 6s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 2s the patch passed -1 compile 10m 1s root in the patch failed. -1 cc 10m 1s root in the patch failed. -1 javac 10m 1s root in the patch failed. -0 checkstyle 2m 2s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426) +1 mvnsite 2m 40s the patch passed +1 mvneclipse 0m 52s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 20s the patch passed +1 javadoc 1m 56s the patch passed -1 unit 8m 16s hadoop-common in the patch failed. +1 unit 1m 7s hadoop-hdfs-client in the patch passed. -1 unit 69m 28s hadoop-hdfs in the patch failed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 142m 44s Reason Tests Failed junit tests hadoop.security.TestKDiag   hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855684/HDFS-11163-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 5884a3667503 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3749152 Default Java 1.8.0_121 compile https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/branch-compile-root.txt findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-compile-root.txt cc https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-compile-root.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18536/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18536/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18536/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 16s Maven dependency ordering for branch
          +1 mvninstall 14m 12s trunk passed
          -1 compile 10m 45s root in trunk failed.
          +1 checkstyle 2m 3s trunk passed
          +1 mvnsite 2m 43s trunk passed
          +1 mvneclipse 0m 56s trunk passed
          +1 findbugs 5m 26s trunk passed
          +1 javadoc 2m 19s trunk passed
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 2m 25s the patch passed
          -1 compile 8m 57s root in the patch failed.
          -1 cc 8m 57s root in the patch failed.
          -1 javac 8m 57s root in the patch failed.
          -0 checkstyle 1m 59s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426)
          +1 mvnsite 2m 45s the patch passed
          +1 mvneclipse 0m 55s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 5m 45s the patch passed
          +1 javadoc 2m 23s the patch passed
          +1 unit 9m 16s hadoop-common in the patch passed.
          +1 unit 1m 5s hadoop-hdfs-client in the patch passed.
          -1 unit 93m 26s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 45s The patch does not generate ASF License warnings.
          170m 32s



          Reason Tests
          Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855684/HDFS-11163-004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 95e9e022ea8a 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 3749152
          Default Java 1.8.0_121
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/branch-compile-root.txt
          findbugs v3.0.0
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/patch-compile-root.txt
          cc https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/patch-compile-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18535/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18535/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 14m 12s trunk passed -1 compile 10m 45s root in trunk failed. +1 checkstyle 2m 3s trunk passed +1 mvnsite 2m 43s trunk passed +1 mvneclipse 0m 56s trunk passed +1 findbugs 5m 26s trunk passed +1 javadoc 2m 19s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 2m 25s the patch passed -1 compile 8m 57s root in the patch failed. -1 cc 8m 57s root in the patch failed. -1 javac 8m 57s root in the patch failed. -0 checkstyle 1m 59s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426) +1 mvnsite 2m 45s the patch passed +1 mvneclipse 0m 55s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 45s the patch passed +1 javadoc 2m 23s the patch passed +1 unit 9m 16s hadoop-common in the patch passed. +1 unit 1m 5s hadoop-hdfs-client in the patch passed. -1 unit 93m 26s hadoop-hdfs in the patch failed. +1 asflicense 0m 45s The patch does not generate ASF License warnings. 170m 32s Reason Tests Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855684/HDFS-11163-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 95e9e022ea8a 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3749152 Default Java 1.8.0_121 compile https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/branch-compile-root.txt findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/patch-compile-root.txt cc https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/patch-compile-root.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18535/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18535/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          yarn compilation is not related, checkstyle issues I will fix in next patch...

          Show
          surendrasingh Surendra Singh Lilhore added a comment - yarn compilation is not related, checkstyle issues I will fix in next patch...
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 12m 39s trunk passed
          -1 compile 9m 15s root in trunk failed.
          +1 checkstyle 2m 9s trunk passed
          +1 mvnsite 3m 3s trunk passed
          +1 mvneclipse 1m 2s trunk passed
          +1 findbugs 4m 59s trunk passed
          +1 javadoc 1m 59s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 2m 13s the patch passed
          -1 compile 9m 48s root in the patch failed.
          -1 cc 9m 48s root in the patch failed.
          -1 javac 9m 48s root in the patch failed.
          -0 checkstyle 1m 57s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426)
          +1 mvnsite 2m 54s the patch passed
          +1 mvneclipse 0m 50s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 5m 33s the patch passed
          +1 javadoc 2m 2s the patch passed
          +1 unit 8m 55s hadoop-common in the patch passed.
          +1 unit 1m 1s hadoop-hdfs-client in the patch passed.
          -1 unit 115m 20s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 39s The patch does not generate ASF License warnings.
          188m 32s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness
          Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2
            org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855684/HDFS-11163-004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 09e653f8c250 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / eb5a179
          Default Java 1.8.0_121
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/branch-compile-root.txt
          findbugs v3.0.0
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/patch-compile-root.txt
          cc https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/patch-compile-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18537/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18537/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 12m 39s trunk passed -1 compile 9m 15s root in trunk failed. +1 checkstyle 2m 9s trunk passed +1 mvnsite 3m 3s trunk passed +1 mvneclipse 1m 2s trunk passed +1 findbugs 4m 59s trunk passed +1 javadoc 1m 59s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 13s the patch passed -1 compile 9m 48s root in the patch failed. -1 cc 9m 48s root in the patch failed. -1 javac 9m 48s root in the patch failed. -0 checkstyle 1m 57s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426) +1 mvnsite 2m 54s the patch passed +1 mvneclipse 0m 50s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 33s the patch passed +1 javadoc 2m 2s the patch passed +1 unit 8m 55s hadoop-common in the patch passed. +1 unit 1m 1s hadoop-hdfs-client in the patch passed. -1 unit 115m 20s hadoop-hdfs in the patch failed. +1 asflicense 0m 39s The patch does not generate ASF License warnings. 188m 32s Reason Tests Failed junit tests hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855684/HDFS-11163-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 09e653f8c250 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / eb5a179 Default Java 1.8.0_121 compile https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/branch-compile-root.txt findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/patch-compile-root.txt cc https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/patch-compile-root.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18537/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18537/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18537/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 29s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 17s Maven dependency ordering for branch
          +1 mvninstall 13m 57s trunk passed
          -1 compile 12m 18s root in trunk failed.
          +1 checkstyle 2m 26s trunk passed
          +1 mvnsite 4m 13s trunk passed
          +1 mvneclipse 1m 11s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui
          +1 findbugs 5m 27s trunk passed
          +1 javadoc 2m 44s trunk passed
          0 mvndep 0m 21s Maven dependency ordering for patch
          +1 mvninstall 2m 47s the patch passed
          +1 compile 14m 17s the patch passed
          -1 cc 14m 17s root generated 5 new + 2 unchanged - 0 fixed = 7 total (was 2)
          -1 javac 14m 17s root generated 508 new + 276 unchanged - 0 fixed = 784 total (was 276)
          -0 checkstyle 2m 22s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426)
          +1 mvnsite 4m 39s the patch passed
          +1 mvneclipse 1m 5s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui
          +1 findbugs 6m 18s the patch passed
          +1 javadoc 2m 31s the patch passed
          -1 unit 10m 14s hadoop-common in the patch failed.
          +1 unit 1m 13s hadoop-hdfs-client in the patch passed.
          -1 unit 122m 29s hadoop-hdfs in the patch failed.
          +1 unit 3m 40s hadoop-yarn-ui in the patch passed.
          +1 asflicense 0m 48s The patch does not generate ASF License warnings.
          217m 16s



          Reason Tests
          Failed junit tests hadoop.security.TestRaceWhenRelogin
            hadoop.hdfs.server.datanode.TestDataNodeUUID
            hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker
            hadoop.hdfs.TestReadStripedFileWithMissingBlocks
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2
            org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855935/temp-YARN-6278.HDFS-11163.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux 27bb527747ff 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 490abfb
          Default Java 1.8.0_121
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/branch-compile-root.txt
          findbugs v3.0.0
          cc https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/diff-compile-cc-root.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/diff-compile-javac-root.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18542/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18542/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 29s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 13m 57s trunk passed -1 compile 12m 18s root in trunk failed. +1 checkstyle 2m 26s trunk passed +1 mvnsite 4m 13s trunk passed +1 mvneclipse 1m 11s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui +1 findbugs 5m 27s trunk passed +1 javadoc 2m 44s trunk passed 0 mvndep 0m 21s Maven dependency ordering for patch +1 mvninstall 2m 47s the patch passed +1 compile 14m 17s the patch passed -1 cc 14m 17s root generated 5 new + 2 unchanged - 0 fixed = 7 total (was 2) -1 javac 14m 17s root generated 508 new + 276 unchanged - 0 fixed = 784 total (was 276) -0 checkstyle 2m 22s root: The patch generated 7 new + 424 unchanged - 2 fixed = 431 total (was 426) +1 mvnsite 4m 39s the patch passed +1 mvneclipse 1m 5s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui +1 findbugs 6m 18s the patch passed +1 javadoc 2m 31s the patch passed -1 unit 10m 14s hadoop-common in the patch failed. +1 unit 1m 13s hadoop-hdfs-client in the patch passed. -1 unit 122m 29s hadoop-hdfs in the patch failed. +1 unit 3m 40s hadoop-yarn-ui in the patch passed. +1 asflicense 0m 48s The patch does not generate ASF License warnings. 217m 16s Reason Tests Failed junit tests hadoop.security.TestRaceWhenRelogin   hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker   hadoop.hdfs.TestReadStripedFileWithMissingBlocks   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.TestDirectoryScanner Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855935/temp-YARN-6278.HDFS-11163.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 27bb527747ff 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 490abfb Default Java 1.8.0_121 compile https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/branch-compile-root.txt findbugs v3.0.0 cc https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/diff-compile-cc-root.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18542/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18542/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18542/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          I hope YARN-6278 fixed compilation error..
          Re-submitting patch...

          Show
          surendrasingh Surendra Singh Lilhore added a comment - I hope YARN-6278 fixed compilation error.. Re-submitting patch...
          Hide
          cnauroth Chris Nauroth added a comment -

          The FsServerDefaults class is annotated Public, so let's maintain the existing constructor signature and add a new constructor that supports passing default storage policy ID. The old constructor can delegate to the new constructor with default storage policy ID of 0. That also would remove the need to change FtpConfigKeys and LocalConfigKeys in this patch.

          The logic is looking good to me, but I'd still like a second opinion review before committing anything.

          Show
          cnauroth Chris Nauroth added a comment - The FsServerDefaults class is annotated Public , so let's maintain the existing constructor signature and add a new constructor that supports passing default storage policy ID. The old constructor can delegate to the new constructor with default storage policy ID of 0. That also would remove the need to change FtpConfigKeys and LocalConfigKeys in this patch. The logic is looking good to me, but I'd still like a second opinion review before committing anything.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Thanks Chris Nauroth for review.
          Attached updated patch.

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Thanks Chris Nauroth for review. Attached updated patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 1m 54s Maven dependency ordering for branch
          +1 mvninstall 12m 23s trunk passed
          +1 compile 10m 14s trunk passed
          +1 checkstyle 2m 2s trunk passed
          +1 mvnsite 2m 51s trunk passed
          +1 mvneclipse 1m 11s trunk passed
          +1 findbugs 4m 59s trunk passed
          +1 javadoc 2m 15s trunk passed
          0 mvndep 0m 13s Maven dependency ordering for patch
          +1 mvninstall 1m 52s the patch passed
          +1 compile 9m 56s the patch passed
          +1 cc 9m 56s the patch passed
          +1 javac 9m 56s the patch passed
          -0 checkstyle 2m 2s root: The patch generated 2 new + 349 unchanged - 1 fixed = 351 total (was 350)
          +1 mvnsite 2m 51s the patch passed
          +1 mvneclipse 1m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 5m 25s the patch passed
          +1 javadoc 2m 18s the patch passed
          -1 unit 8m 24s hadoop-common in the patch failed.
          +1 unit 1m 7s hadoop-hdfs-client in the patch passed.
          +1 unit 64m 15s hadoop-hdfs in the patch passed.
          +1 asflicense 0m 44s The patch does not generate ASF License warnings.
          140m 6s



          Reason Tests
          Failed junit tests hadoop.fs.sftp.TestSFTPFileSystem



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856667/HDFS-11163-006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 62867f96201c 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 28daaf0
          Default Java 1.8.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18636/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18636/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18636/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18636/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 1m 54s Maven dependency ordering for branch +1 mvninstall 12m 23s trunk passed +1 compile 10m 14s trunk passed +1 checkstyle 2m 2s trunk passed +1 mvnsite 2m 51s trunk passed +1 mvneclipse 1m 11s trunk passed +1 findbugs 4m 59s trunk passed +1 javadoc 2m 15s trunk passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 52s the patch passed +1 compile 9m 56s the patch passed +1 cc 9m 56s the patch passed +1 javac 9m 56s the patch passed -0 checkstyle 2m 2s root: The patch generated 2 new + 349 unchanged - 1 fixed = 351 total (was 350) +1 mvnsite 2m 51s the patch passed +1 mvneclipse 1m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 25s the patch passed +1 javadoc 2m 18s the patch passed -1 unit 8m 24s hadoop-common in the patch failed. +1 unit 1m 7s hadoop-hdfs-client in the patch passed. +1 unit 64m 15s hadoop-hdfs in the patch passed. +1 asflicense 0m 44s The patch does not generate ASF License warnings. 140m 6s Reason Tests Failed junit tests hadoop.fs.sftp.TestSFTPFileSystem Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856667/HDFS-11163-006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 62867f96201c 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 28daaf0 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18636/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18636/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18636/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18636/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Hi Chris Nauroth,

          The logic is looking good to me, but I'd still like a second opinion review before committing anything.

          Based on Vinayakumar B last comment , I feel he is okay with this logic..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Hi Chris Nauroth , The logic is looking good to me, but I'd still like a second opinion review before committing anything. Based on Vinayakumar B last comment , I feel he is okay with this logic..
          Hide
          cnauroth Chris Nauroth added a comment -

          The Checkstyle warnings are not worth addressing.

          The test failure is in unrelated code, and I can't repro it.

          +1 for patch revision 006. Vinayakumar B, just to make sure I'm clear, are you also +1 now? If so, I'd be happy to commit.

          Show
          cnauroth Chris Nauroth added a comment - The Checkstyle warnings are not worth addressing. The test failure is in unrelated code, and I can't repro it. +1 for patch revision 006. Vinayakumar B , just to make sure I'm clear, are you also +1 now? If so, I'd be happy to commit.
          Hide
          vinayrpet Vinayakumar B added a comment -

          +1 for 006 patch.

          Show
          vinayrpet Vinayakumar B added a comment - +1 for 006 patch.
          Hide
          cnauroth Chris Nauroth added a comment -

          Surendra Singh Lilhore, I was about to commit this when I noticed that the patch is not compatible with branch-2. Can you please provide a version of the patch for branch-2?

          Show
          cnauroth Chris Nauroth added a comment - Surendra Singh Lilhore , I was about to commit this when I noticed that the patch is not compatible with branch-2. Can you please provide a version of the patch for branch-2?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 1m 34s Maven dependency ordering for branch
          +1 mvninstall 6m 44s branch-2 passed
          +1 compile 5m 49s branch-2 passed with JDK v1.8.0_121
          +1 compile 6m 35s branch-2 passed with JDK v1.7.0_121
          +1 checkstyle 1m 31s branch-2 passed
          +1 mvnsite 2m 22s branch-2 passed
          +1 mvneclipse 0m 45s branch-2 passed
          +1 findbugs 5m 13s branch-2 passed
          +1 javadoc 2m 0s branch-2 passed with JDK v1.8.0_121
          +1 javadoc 2m 57s branch-2 passed with JDK v1.7.0_121
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 53s the patch passed
          +1 compile 5m 31s the patch passed with JDK v1.8.0_121
          +1 cc 5m 31s the patch passed
          +1 javac 5m 31s the patch passed
          +1 compile 6m 32s the patch passed with JDK v1.7.0_121
          +1 cc 6m 32s the patch passed
          +1 javac 6m 32s the patch passed
          -0 checkstyle 1m 31s root: The patch generated 2 new + 340 unchanged - 1 fixed = 342 total (was 341)
          +1 mvnsite 2m 21s the patch passed
          +1 mvneclipse 0m 44s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 5m 55s the patch passed
          +1 javadoc 1m 57s the patch passed with JDK v1.8.0_121
          +1 javadoc 2m 50s the patch passed with JDK v1.7.0_121
          +1 unit 8m 30s hadoop-common in the patch passed with JDK v1.7.0_121.
          +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_121.
          -1 unit 49m 24s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          193m 25s



          Reason Tests
          JDK v1.8.0_121 Failed junit tests hadoop.ipc.TestRPCWaitForProxy
          JDK v1.8.0_121 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859268/HDFS-11163-branch-2.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 99ab6b918116 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / dbdd97d
          Default Java 1.7.0_121
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_121 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18749/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18749/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18749/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18749/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 1m 34s Maven dependency ordering for branch +1 mvninstall 6m 44s branch-2 passed +1 compile 5m 49s branch-2 passed with JDK v1.8.0_121 +1 compile 6m 35s branch-2 passed with JDK v1.7.0_121 +1 checkstyle 1m 31s branch-2 passed +1 mvnsite 2m 22s branch-2 passed +1 mvneclipse 0m 45s branch-2 passed +1 findbugs 5m 13s branch-2 passed +1 javadoc 2m 0s branch-2 passed with JDK v1.8.0_121 +1 javadoc 2m 57s branch-2 passed with JDK v1.7.0_121 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 53s the patch passed +1 compile 5m 31s the patch passed with JDK v1.8.0_121 +1 cc 5m 31s the patch passed +1 javac 5m 31s the patch passed +1 compile 6m 32s the patch passed with JDK v1.7.0_121 +1 cc 6m 32s the patch passed +1 javac 6m 32s the patch passed -0 checkstyle 1m 31s root: The patch generated 2 new + 340 unchanged - 1 fixed = 342 total (was 341) +1 mvnsite 2m 21s the patch passed +1 mvneclipse 0m 44s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 55s the patch passed +1 javadoc 1m 57s the patch passed with JDK v1.8.0_121 +1 javadoc 2m 50s the patch passed with JDK v1.7.0_121 +1 unit 8m 30s hadoop-common in the patch passed with JDK v1.7.0_121. +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_121. -1 unit 49m 24s hadoop-hdfs in the patch failed with JDK v1.7.0_121. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 193m 25s Reason Tests JDK v1.8.0_121 Failed junit tests hadoop.ipc.TestRPCWaitForProxy JDK v1.8.0_121 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859268/HDFS-11163-branch-2.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 99ab6b918116 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / dbdd97d Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_121 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18749/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18749/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18749/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18749/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Checkstyle warnings and test failures are unrelated.
          Please review...

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Checkstyle warnings and test failures are unrelated. Please review...
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          pinging Chris Nauroth

          Show
          surendrasingh Surendra Singh Lilhore added a comment - pinging Chris Nauroth
          Hide
          cnauroth Chris Nauroth added a comment -

          Surendra Singh Lilhore, sorry, but the patches need to be rebased again. Sorry for the churn. I'll do my best to prioritize getting these committed right after you post new patches so that you don't have to rebase again.

          Show
          cnauroth Chris Nauroth added a comment - Surendra Singh Lilhore , sorry, but the patches need to be rebased again. Sorry for the churn. I'll do my best to prioritize getting these committed right after you post new patches so that you don't have to rebase again.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Thansk Chris Nauroth.
          Attached v7 trunk patch..
          I will attach branch-2 patch after getting QA report for trunk patch..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Thansk Chris Nauroth . Attached v7 trunk patch.. I will attach branch-2 patch after getting QA report for trunk patch..
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 1m 33s Maven dependency ordering for branch
          +1 mvninstall 14m 0s trunk passed
          +1 compile 16m 21s trunk passed
          +1 checkstyle 1m 58s trunk passed
          +1 mvnsite 2m 56s trunk passed
          +1 mvneclipse 1m 2s trunk passed
          +1 findbugs 5m 7s trunk passed
          +1 javadoc 2m 3s trunk passed
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 2m 6s the patch passed
          +1 compile 15m 8s the patch passed
          +1 cc 15m 8s the patch passed
          +1 javac 15m 8s the patch passed
          -0 checkstyle 2m 0s root: The patch generated 2 new + 349 unchanged - 1 fixed = 351 total (was 350)
          +1 mvnsite 2m 45s the patch passed
          +1 mvneclipse 1m 0s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 5m 26s the patch passed
          +1 javadoc 2m 4s the patch passed
          +1 unit 8m 34s hadoop-common in the patch passed.
          +1 unit 1m 18s hadoop-hdfs-client in the patch passed.
          -1 unit 66m 13s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 37s The patch does not generate ASF License warnings.
          154m 4s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.TestMaintenanceState



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:612578f
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862798/HDFS-11163-007.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 16fa49820ae6 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / aabf08d
          Default Java 1.8.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19040/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19040/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19040/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19040/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 1m 33s Maven dependency ordering for branch +1 mvninstall 14m 0s trunk passed +1 compile 16m 21s trunk passed +1 checkstyle 1m 58s trunk passed +1 mvnsite 2m 56s trunk passed +1 mvneclipse 1m 2s trunk passed +1 findbugs 5m 7s trunk passed +1 javadoc 2m 3s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 2m 6s the patch passed +1 compile 15m 8s the patch passed +1 cc 15m 8s the patch passed +1 javac 15m 8s the patch passed -0 checkstyle 2m 0s root: The patch generated 2 new + 349 unchanged - 1 fixed = 351 total (was 350) +1 mvnsite 2m 45s the patch passed +1 mvneclipse 1m 0s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 26s the patch passed +1 javadoc 2m 4s the patch passed +1 unit 8m 34s hadoop-common in the patch passed. +1 unit 1m 18s hadoop-hdfs-client in the patch passed. -1 unit 66m 13s hadoop-hdfs in the patch failed. +1 asflicense 0m 37s The patch does not generate ASF License warnings. 154m 4s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestMaintenanceState Subsystem Report/Notes Docker Image:yetus/hadoop:612578f JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862798/HDFS-11163-007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 16fa49820ae6 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / aabf08d Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19040/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19040/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19040/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19040/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Attached branch-2 v2 patch..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Attached branch-2 v2 patch..
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 10s HDFS-11163 does not apply to branch-2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862850/HDFS-11163-branch-2.002.patch
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19043/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 10s HDFS-11163 does not apply to branch-2. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862850/HDFS-11163-branch-2.002.patch Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19043/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Surendra Singh Lilhore, it looks like HDFS-11163-branch-2.002.patch still doesn't apply cleanly.

          Show
          cnauroth Chris Nauroth added a comment - Surendra Singh Lilhore , it looks like HDFS-11163 -branch-2.002.patch still doesn't apply cleanly.
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Sorry, Its my mistake. I didn't pull the latest code.
          Attached new patch for branch-2

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Sorry, Its my mistake. I didn't pull the latest code. Attached new patch for branch-2
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 34m 46s Docker mode activated.
          +1 @author 0m 1s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 2m 5s Maven dependency ordering for branch
          +1 mvninstall 11m 56s branch-2 passed
          +1 compile 9m 24s branch-2 passed with JDK v1.8.0_121
          +1 compile 9m 53s branch-2 passed with JDK v1.7.0_121
          +1 checkstyle 2m 12s branch-2 passed
          +1 mvnsite 3m 45s branch-2 passed
          +1 mvneclipse 1m 15s branch-2 passed
          +1 findbugs 7m 31s branch-2 passed
          +1 javadoc 2m 55s branch-2 passed with JDK v1.8.0_121
          +1 javadoc 3m 14s branch-2 passed with JDK v1.7.0_121
          0 mvndep 0m 20s Maven dependency ordering for patch
          +1 mvninstall 3m 0s the patch passed
          +1 compile 10m 13s the patch passed with JDK v1.8.0_121
          +1 cc 10m 13s the patch passed
          +1 javac 10m 13s the patch passed
          +1 compile 10m 20s the patch passed with JDK v1.7.0_121
          +1 cc 10m 20s the patch passed
          +1 javac 10m 20s the patch passed
          -0 checkstyle 2m 20s root: The patch generated 2 new + 340 unchanged - 1 fixed = 342 total (was 341)
          +1 mvnsite 3m 47s the patch passed
          +1 mvneclipse 1m 16s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 8m 43s the patch passed
          +1 javadoc 2m 50s the patch passed with JDK v1.8.0_121
          +1 javadoc 3m 11s the patch passed with JDK v1.7.0_121
          +1 unit 11m 34s hadoop-common in the patch passed with JDK v1.7.0_121.
          +1 unit 1m 31s hadoop-hdfs-client in the patch passed with JDK v1.7.0_121.
          -1 unit 98m 49s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          +1 asflicense 1m 36s The patch does not generate ASF License warnings.
          359m 25s



          Reason Tests
          JDK v1.8.0_121 Failed junit tests hadoop.security.TestShellBasedUnixGroupsMapping
            hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
          JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain
            hadoop.hdfs.TestMaintenanceState
          JDK v1.7.0_121 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HDFS-11163
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862907/HDFS-11163-branch-2.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux 282ae9768010 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / b04c091
          Default Java 1.7.0_121
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_121 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19047/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19047/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19047/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19047/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 34m 46s Docker mode activated. +1 @author 0m 1s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 2m 5s Maven dependency ordering for branch +1 mvninstall 11m 56s branch-2 passed +1 compile 9m 24s branch-2 passed with JDK v1.8.0_121 +1 compile 9m 53s branch-2 passed with JDK v1.7.0_121 +1 checkstyle 2m 12s branch-2 passed +1 mvnsite 3m 45s branch-2 passed +1 mvneclipse 1m 15s branch-2 passed +1 findbugs 7m 31s branch-2 passed +1 javadoc 2m 55s branch-2 passed with JDK v1.8.0_121 +1 javadoc 3m 14s branch-2 passed with JDK v1.7.0_121 0 mvndep 0m 20s Maven dependency ordering for patch +1 mvninstall 3m 0s the patch passed +1 compile 10m 13s the patch passed with JDK v1.8.0_121 +1 cc 10m 13s the patch passed +1 javac 10m 13s the patch passed +1 compile 10m 20s the patch passed with JDK v1.7.0_121 +1 cc 10m 20s the patch passed +1 javac 10m 20s the patch passed -0 checkstyle 2m 20s root: The patch generated 2 new + 340 unchanged - 1 fixed = 342 total (was 341) +1 mvnsite 3m 47s the patch passed +1 mvneclipse 1m 16s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 8m 43s the patch passed +1 javadoc 2m 50s the patch passed with JDK v1.8.0_121 +1 javadoc 3m 11s the patch passed with JDK v1.7.0_121 +1 unit 11m 34s hadoop-common in the patch passed with JDK v1.7.0_121. +1 unit 1m 31s hadoop-hdfs-client in the patch passed with JDK v1.7.0_121. -1 unit 98m 49s hadoop-hdfs in the patch failed with JDK v1.7.0_121. +1 asflicense 1m 36s The patch does not generate ASF License warnings. 359m 25s Reason Tests JDK v1.8.0_121 Failed junit tests hadoop.security.TestShellBasedUnixGroupsMapping   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain   hadoop.hdfs.TestMaintenanceState JDK v1.7.0_121 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HDFS-11163 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862907/HDFS-11163-branch-2.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 282ae9768010 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / b04c091 Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_121 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19047/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19047/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19047/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19047/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 for the latest patches. I have committed this to trunk, branch-2 and branch-2.8. Surendra Singh Lilhore, thank you for the contribution.

          Show
          cnauroth Chris Nauroth added a comment - +1 for the latest patches. I have committed this to trunk, branch-2 and branch-2.8. Surendra Singh Lilhore , thank you for the contribution.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11574 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11574/)
          HDFS-11163. Mover should move the file blocks to default storage once (cnauroth: rev 23b1a7bdf1b546c1e29d7010cf139b6d700461fc)

          • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsServerDefaults.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestMover.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11574 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11574/ ) HDFS-11163 . Mover should move the file blocks to default storage once (cnauroth: rev 23b1a7bdf1b546c1e29d7010cf139b6d700461fc) (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsServerDefaults.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestMover.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Hide
          surendrasingh Surendra Singh Lilhore added a comment -

          Thanks Chris Nauroth for review and commit. Thanks Vinayakumar B for review..

          Show
          surendrasingh Surendra Singh Lilhore added a comment - Thanks Chris Nauroth for review and commit. Thanks Vinayakumar B for review..
          Hide
          djp Junping Du added a comment -

          Hi Chris Nauroth, thanks for review and commit. As my email to hadoop dev list, we have 2.8.1 branch get cut-off for release since yesterday. Just merge the commit to branch-2.8.1 assume it is supposed to land in 2.8.1 release. Isn't it?

          Show
          djp Junping Du added a comment - Hi Chris Nauroth , thanks for review and commit. As my email to hadoop dev list, we have 2.8.1 branch get cut-off for release since yesterday. Just merge the commit to branch-2.8.1 assume it is supposed to land in 2.8.1 release. Isn't it?
          Hide
          cnauroth Chris Nauroth added a comment -

          Junping Du, sorry I missed the email update on the 2.8.1 release plan. Thank you for cherry-picking it into the new branch-2.8.1.

          Show
          cnauroth Chris Nauroth added a comment - Junping Du , sorry I missed the email update on the 2.8.1 release plan. Thank you for cherry-picking it into the new branch-2.8.1.
          Hide
          djp Junping Du added a comment -

          You are welcome, Chris Nauroth!

          Show
          djp Junping Du added a comment - You are welcome, Chris Nauroth !
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/)
          HDFS-11163. Mover should move the file blocks to default storage once (cnauroth: rev 23b1a7bdf1b546c1e29d7010cf139b6d700461fc)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestMover.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsServerDefaults.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/ ) HDFS-11163 . Mover should move the file blocks to default storage once (cnauroth: rev 23b1a7bdf1b546c1e29d7010cf139b6d700461fc) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/Mover.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestMover.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FsServerDefaults.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

            People

            • Assignee:
              surendrasingh Surendra Singh Lilhore
              Reporter:
              surendrasingh Surendra Singh Lilhore
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development