Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6955

DN should reserve disk space for a full block when creating tmp files

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HDFS-6898 is introducing disk space reservation for RBW files to avoid running out of disk space midway through block creation.

      This Jira is to introduce similar reservation for tmp files.

      1. HDFS-6955-branch-2.09.patch
        23 kB
        Kanaka Kumar Avvaru
      2. HDFS-6955-09.patch
        23 kB
        Kanaka Kumar Avvaru
      3. HDFS-6955-08.patch
        23 kB
        Kanaka Kumar Avvaru
      4. HDFS-6955-07.patch
        23 kB
        Kanaka Kumar Avvaru
      5. HDFS-6955-06.patch
        22 kB
        Kanaka Kumar Avvaru
      6. HDFS-6955-05.patch
        20 kB
        Kanaka Kumar Avvaru
      7. HDFS-6955-04.patch
        20 kB
        Kanaka Kumar Avvaru
      8. HDFS-6955-03.patch
        20 kB
        Kanaka Kumar Avvaru
      9. HDFS-6955-02.patch
        20 kB
        Kanaka Kumar Avvaru
      10. HDFS-6955-01.patch
        19 kB
        Kanaka Kumar Avvaru

        Issue Links

          Activity

          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Hi Arpit Agarwal, I would like to work on this JIRA. Please share any inputs if you have already thought of.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Hi Arpit Agarwal , I would like to work on this JIRA. Please share any inputs if you have already thought of.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Hi Kanaka Kumar Avvaru, no I haven't thought much about this one yet. The idea is to reserve space for blocks being re-replicated.

          Show
          arpitagarwal Arpit Agarwal added a comment - Hi Kanaka Kumar Avvaru , no I haven't thought much about this one yet. The idea is to reserve space for blocks being re-replicated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Attached the initial patch for reserving space for the block being re-replicated. Arpit Agarwal, please review.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Attached the initial patch for reserving space for the block being re-replicated. Arpit Agarwal , please review.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 23s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
          +1 javac 7m 47s There were no new javac warning messages.
          +1 javadoc 9m 49s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 22s The applied patch generated 3 new checkstyle issues (total was 154, now 155).
          -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 23s mvn install still works.
          +1 eclipse:eclipse 0m 41s The patch built with eclipse:eclipse.
          +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 55s Pre-build of native portion
          -1 hdfs tests 146m 6s Tests failed in hadoop-hdfs.
              191m 58s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.blockmanagement.TestNodeCount
          Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir
            org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12750505/HDFS-6955-01.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 84bf712
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11997/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11997/artifact/patchprocess/whitespace.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11997/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11997/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11997/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 23s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 7m 47s There were no new javac warning messages. +1 javadoc 9m 49s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 22s The applied patch generated 3 new checkstyle issues (total was 154, now 155). -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 41s The patch built with eclipse:eclipse. +1 findbugs 3m 3s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 55s Pre-build of native portion -1 hdfs tests 146m 6s Tests failed in hadoop-hdfs.     191m 58s   Reason Tests Failed unit tests hadoop.hdfs.server.blockmanagement.TestNodeCount Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir   org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12750505/HDFS-6955-01.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 84bf712 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11997/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11997/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11997/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11997/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11997/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Updated patch for white space errors and a checkstyle issue FsVolumeImpl.java:462:69: 'reserved' hides a field..

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Updated patch for white space errors and a checkstyle issue FsVolumeImpl.java:462:69: 'reserved' hides a field. .
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 50s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
          +1 javac 8m 2s There were no new javac warning messages.
          +1 javadoc 9m 54s There were no new javadoc warning messages.
          +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 24s The applied patch generated 3 new checkstyle issues (total was 154, now 155).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 24s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 36s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 9s Pre-build of native portion
          -1 hdfs tests 176m 49s Tests failed in hadoop-hdfs.
              222m 14s  



          Reason Tests
          Timed out tests org.apache.hadoop.cli.TestHDFSCLI



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12750865/HDFS-6955-02.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / c77bd6a
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12013/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12013/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12013/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12013/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 50s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 8m 2s There were no new javac warning messages. +1 javadoc 9m 54s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 24s The applied patch generated 3 new checkstyle issues (total was 154, now 155). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 24s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 36s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 9s Pre-build of native portion -1 hdfs tests 176m 49s Tests failed in hadoop-hdfs.     222m 14s   Reason Tests Timed out tests org.apache.hadoop.cli.TestHDFSCLI Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12750865/HDFS-6955-02.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c77bd6a checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12013/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12013/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12013/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12013/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          updated patch for required check style errors. Test failure doesn't seem to be related to this patch

          Show
          kanaka Kanaka Kumar Avvaru added a comment - updated patch for required check style errors. Test failure doesn't seem to be related to this patch
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 43s Pre-patch trunk has 4 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
          +1 javac 7m 43s There were no new javac warning messages.
          +1 javadoc 9m 54s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 21s The applied patch generated 3 new checkstyle issues (total was 153, now 154).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 10s Pre-build of native portion
          -1 hdfs tests 164m 1s Tests failed in hadoop-hdfs.
              208m 53s  



          Reason Tests
          Failed unit tests hadoop.hdfs.web.TestWebHDFS



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12752303/HDFS-6955-03.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / eee0d45
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12126/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12126/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12126/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12126/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12126/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 43s Pre-patch trunk has 4 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 9m 54s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 21s The applied patch generated 3 new checkstyle issues (total was 153, now 154). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 10s Pre-build of native portion -1 hdfs tests 164m 1s Tests failed in hadoop-hdfs.     208m 53s   Reason Tests Failed unit tests hadoop.hdfs.web.TestWebHDFS Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752303/HDFS-6955-03.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / eee0d45 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12126/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12126/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12126/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12126/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12126/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Updated patch for the check style issue reported

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Updated patch for the check style issue reported
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 2s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 4 new or modified test files.
          +1 javac 7m 54s There were no new javac warning messages.
          +1 javadoc 10m 7s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 20s The applied patch generated 2 new checkstyle issues (total was 155, now 155).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse.
          +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 16s Pre-build of native portion
          -1 hdfs tests 70m 1s Tests failed in hadoop-hdfs.
              115m 46s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
          Timed out tests org.apache.hadoop.hdfs.TestDatanodeDeath
            org.apache.hadoop.hdfs.TestDecommission



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12753357/HDFS-6955-04.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / caa04de
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12218/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12218/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12218/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12218/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 2s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 4 new or modified test files. +1 javac 7m 54s There were no new javac warning messages. +1 javadoc 10m 7s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 20s The applied patch generated 2 new checkstyle issues (total was 155, now 155). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse. +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 16s Pre-build of native portion -1 hdfs tests 70m 1s Tests failed in hadoop-hdfs.     115m 46s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints Timed out tests org.apache.hadoop.hdfs.TestDatanodeDeath   org.apache.hadoop.hdfs.TestDecommission Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753357/HDFS-6955-04.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / caa04de checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12218/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12218/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12218/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12218/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Test failure looks not related to the changes done in this patch. Check style issues can be ignored as file length was big before the patch and accessor methods are not required for "reservedSpaceForTmp"

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Test failure looks not related to the changes done in this patch. Check style issues can be ignored as file length was big before the patch and accessor methods are not required for "reservedSpaceForTmp"
          Hide
          vinayrpet Vinayakumar B added a comment -

          Thanks Kanaka Kumar Avvaru for taking up the issue.

          Patch looks almost good.

          Some nits.
          1. I think releasing reservation in cleanupBlock() not required. As by the time this method is called in BlockReceiver, already reserved space will be released. So this would release again.

          2. Since Now reservation is for both RBW and Temp files, renaming TestRbwSpaceReservation.java to something like TestSpaceReservation.java would be a good idea IMO. If you feel existing is fine, then no issues.

          Arpit Agarwal, Do you also want to take a look at the patch.?

          Show
          vinayrpet Vinayakumar B added a comment - Thanks Kanaka Kumar Avvaru for taking up the issue. Patch looks almost good. Some nits. 1. I think releasing reservation in cleanupBlock() not required. As by the time this method is called in BlockReceiver , already reserved space will be released. So this would release again. 2. Since Now reservation is for both RBW and Temp files, renaming TestRbwSpaceReservation.java to something like TestSpaceReservation.java would be a good idea IMO. If you feel existing is fine, then no issues. Arpit Agarwal , Do you also want to take a look at the patch.?
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Thanks for the review Vinayakumar B

          1. Reserved space is not released unless we call cleanupBlock(). Yes its good point to verify double release . I have updated the test case to assert this case. Please correct me if still you feel some other call is clearing space.
          2. Thanks for pointing the name correction. I have renamed in latest version of patch (v5)

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Thanks for the review Vinayakumar B 1. Reserved space is not released unless we call cleanupBlock() . Yes its good point to verify double release . I have updated the test case to assert this case. Please correct me if still you feel some other call is clearing space. 2. Thanks for pointing the name correction. I have renamed in latest version of patch (v5)
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 15m 24s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 7m 42s There were no new javac warning messages.
          +1 javadoc 9m 56s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 30s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 13s Pre-build of native portion
          +1 hdfs tests 186m 53s Tests passed in hadoop-hdfs.
              228m 44s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754404/HDFS-6955-05.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 9b68577
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12323/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12323/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12323/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 15m 24s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 56s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 30s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 13s Pre-build of native portion +1 hdfs tests 186m 53s Tests passed in hadoop-hdfs.     228m 44s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754404/HDFS-6955-05.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 9b68577 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12323/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12323/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12323/console This message was automatically generated.
          Hide
          vinayrpet Vinayakumar B added a comment -

          1. Reserved space is not released unless we call cleanupBlock(). Yes its good point to verify double release . I have updated the test case to assert this case. Please correct me if still you feel some other call is clearing space.

          Thanks for pointing out that, bytesReserved is resetting to 0 in releaseAllBytesReserved, then double calls wont harm.

          I think the following change need not required and this will not reset the value of bytesReserved in replicainfo. Instead, calling replicaInfo.releaseAllBytesReserved(); will make sure everything reset. It would be better to unify this call everywhere.

          -      
          +      FsVolumeImpl v = (FsVolumeImpl)replicaInfo.getVolume();
          +      v.releaseReservedSpaceTmp(replicaInfo.getBytesReserved());
          + 

          One more place replicaInfo.releaseAllBytesReserved(); call required is the BlockReceiver's constructor on exception. This will make sure reserved space is cleared even after some other exception during creation of block ( either rbw or tmp).

          It would be better if more tests can be added to verify these corner cases.

          Show
          vinayrpet Vinayakumar B added a comment - 1. Reserved space is not released unless we call cleanupBlock(). Yes its good point to verify double release . I have updated the test case to assert this case. Please correct me if still you feel some other call is clearing space. Thanks for pointing out that, bytesReserved is resetting to 0 in releaseAllBytesReserved , then double calls wont harm. I think the following change need not required and this will not reset the value of bytesReserved in replicainfo. Instead, calling replicaInfo.releaseAllBytesReserved(); will make sure everything reset. It would be better to unify this call everywhere. - + FsVolumeImpl v = (FsVolumeImpl)replicaInfo.getVolume(); + v.releaseReservedSpaceTmp(replicaInfo.getBytesReserved()); + One more place replicaInfo.releaseAllBytesReserved(); call required is the BlockReceiver 's constructor on exception. This will make sure reserved space is cleared even after some other exception during creation of block ( either rbw or tmp). It would be better if more tests can be added to verify these corner cases.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Thanks for the review Vinayakumar B.

          Its a good suggestion to unify the release through replicaInfo.releaseAllBytesReserved() as this call is safe for multiple invocations also. Actually there is a leak exists if exception occurs just after createRBWFile (Difficult to write test case so tried through debug & caused exceptions through code inspection)

          I have updated patch to remove the clearing logic in cleanupBlock() and use releaseAllBytesReserved() in BlockReceiver constructor on exception case which can handle both RBW & Tmp spaces clearing.

          I have done small changes in test code to ensure release additional space is not happening.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Thanks for the review Vinayakumar B . Its a good suggestion to unify the release through replicaInfo.releaseAllBytesReserved() as this call is safe for multiple invocations also. Actually there is a leak exists if exception occurs just after createRBWFile (Difficult to write test case so tried through debug & caused exceptions through code inspection) I have updated patch to remove the clearing logic in cleanupBlock() and use releaseAllBytesReserved() in BlockReceiver constructor on exception case which can handle both RBW & Tmp spaces clearing. I have done small changes in test code to ensure release additional space is not happening.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 41s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 7m 40s There were no new javac warning messages.
          +1 javadoc 10m 7s There were no new javadoc warning messages.
          +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 21s The applied patch generated 2 new checkstyle issues (total was 215, now 215).
          +1 whitespace 0m 2s The patch has no lines that end in whitespace.
          +1 install 1m 28s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 34s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 10s Pre-build of native portion
          -1 hdfs tests 169m 29s Tests failed in hadoop-hdfs.
              214m 30s  



          Reason Tests
          Failed unit tests hadoop.hdfs.web.TestWebHDFSOAuth2
          Timed out tests org.apache.hadoop.hdfs.web.TestWebHDFS



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12754981/HDFS-6955-06.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 4014ce5
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12360/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12360/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12360/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12360/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 41s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 7m 40s There were no new javac warning messages. +1 javadoc 10m 7s There were no new javadoc warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 21s The applied patch generated 2 new checkstyle issues (total was 215, now 215). +1 whitespace 0m 2s The patch has no lines that end in whitespace. +1 install 1m 28s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 34s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 10s Pre-build of native portion -1 hdfs tests 169m 29s Tests failed in hadoop-hdfs.     214m 30s   Reason Tests Failed unit tests hadoop.hdfs.web.TestWebHDFSOAuth2 Timed out tests org.apache.hadoop.hdfs.web.TestWebHDFS Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12754981/HDFS-6955-06.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 4014ce5 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12360/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12360/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12360/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12360/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Test failure looks not related to the changes done in this patch. Check style issues can be ignored as file length was big before the patch and accessor methods are not required for "reservedSpaceForTmp"

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Test failure looks not related to the changes done in this patch. Check style issues can be ignored as file length was big before the patch and accessor methods are not required for "reservedSpaceForTmp"
          Hide
          vinayrpet Vinayakumar B added a comment -

          thanks Kanaka Kumar Avvaru for the update.

          +1, Changes looks good to me in latest patch.

          Arpit Agarwal, do you also want to take a look?

          Will wait for a day before commit.
          -Thanks

          Show
          vinayrpet Vinayakumar B added a comment - thanks Kanaka Kumar Avvaru for the update. +1, Changes looks good to me in latest patch. Arpit Agarwal , do you also want to take a look? Will wait for a day before commit. -Thanks
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thanks for fixing this Kanaka Kumar Avvaru.

          Do we need the separate counter for tmp or can we just rename the reservedForRbw counter to reserved space?

          Show
          arpitagarwal Arpit Agarwal added a comment - Thanks for fixing this Kanaka Kumar Avvaru . Do we need the separate counter for tmp or can we just rename the reservedForRbw counter to reserved space?
          Hide
          vinayrpet Vinayakumar B added a comment -

          Do we need the separate counter for tmp or can we just rename the reservedForRbw counter to reserved space?

          Thats the good point Arpit Agarwal.
          We could combine it, but 'reservedSpaceForRBW' is exposed as VolumeInfo metric. If we combine it, then metric will have incompatible change.

          What do you say, is it okay to combine despite incompatible change in metric?

          Show
          vinayrpet Vinayakumar B added a comment - Do we need the separate counter for tmp or can we just rename the reservedForRbw counter to reserved space? Thats the good point Arpit Agarwal . We could combine it, but 'reservedSpaceForRBW' is exposed as VolumeInfo metric. If we combine it, then metric will have incompatible change. What do you say, is it okay to combine despite incompatible change in metric?
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Hi Vinayakumar B, good point. Perhaps we can just document that the metric tracks space reserved for both RBW and tmp replicas. A single counter would be easier to maintain.

          Show
          arpitagarwal Arpit Agarwal added a comment - Hi Vinayakumar B , good point. Perhaps we can just document that the metric tracks space reserved for both RBW and tmp replicas. A single counter would be easier to maintain.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Hi Vinayakumar B, good point. Perhaps we can just document that the metric tracks space reserved for both RBW and tmp replicas. A single counter would be easier to maintain.

          You mean, no need to rename the reservedForRbw counter to reserved space?
          just use for tmp files too as is?

          Show
          vinayrpet Vinayakumar B added a comment - Hi Vinayakumar B, good point. Perhaps we can just document that the metric tracks space reserved for both RBW and tmp replicas. A single counter would be easier to maintain. You mean, no need to rename the reservedForRbw counter to reserved space? just use for tmp files too as is?
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          We can rename the internal counter to reservedForReplicas or something similar but leave the metric as it is for compatibility. What do you think?

          Show
          arpitagarwal Arpit Agarwal added a comment - We can rename the internal counter to reservedForReplicas or something similar but leave the metric as it is for compatibility. What do you think?
          Hide
          vinayrpet Vinayakumar B added a comment -

          Yes, that would work. Thank you.

          Kanaka Kumar Avvaru, can you please update based on above suggestion by Arpit?

          Show
          vinayrpet Vinayakumar B added a comment - Yes, that would work. Thank you. Kanaka Kumar Avvaru , can you please update based on above suggestion by Arpit?
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Thanks for review Arpit Agarwal. Thanks for tracking up Vinayakumar B.

          Combining both reservations to one counter is a good suggestion. Updated patch for the same.
          reservedSpaceForRBW is recently added in JMX and not delivered in any release yet. So, we can safely change this also to reservedSpaceForReplicas

          Please review new patch.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Thanks for review Arpit Agarwal . Thanks for tracking up Vinayakumar B . Combining both reservations to one counter is a good suggestion. Updated patch for the same. reservedSpaceForRBW is recently added in JMX and not delivered in any release yet. So, we can safely change this also to reservedSpaceForReplicas Please review new patch.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 20m 1s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 9m 45s There were no new javac warning messages.
          +1 javadoc 11m 51s There were no new javadoc warning messages.
          +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 45s There were no new checkstyle issues.
          -1 whitespace 0m 1s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 57s mvn install still works.
          +1 eclipse:eclipse 0m 44s The patch built with eclipse:eclipse.
          +1 findbugs 3m 7s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 48s Pre-build of native portion
          -1 hdfs tests 160m 27s Tests failed in hadoop-hdfs.
              213m 4s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockManager
            hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
            hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
            hadoop.hdfs.web.TestWebHDFSOAuth2
            hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation
          Timed out tests org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy
            org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12755401/HDFS-6955-07.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 486d5cb
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12398/artifact/patchprocess/whitespace.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12398/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12398/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12398/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 20m 1s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 9m 45s There were no new javac warning messages. +1 javadoc 11m 51s There were no new javadoc warning messages. +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 45s There were no new checkstyle issues. -1 whitespace 0m 1s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 57s mvn install still works. +1 eclipse:eclipse 0m 44s The patch built with eclipse:eclipse. +1 findbugs 3m 7s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 48s Pre-build of native portion -1 hdfs tests 160m 27s Tests failed in hadoop-hdfs.     213m 4s   Reason Tests Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks   hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks   hadoop.hdfs.web.TestWebHDFSOAuth2   hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation Timed out tests org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy   org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12755401/HDFS-6955-07.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 486d5cb whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12398/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12398/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12398/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12398/console This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thanks for updating the patch. A few comments.

          1. Nitpick: Please add spaces around operators replicaInfo!=null
          2. It doesn't look like there is a new test case for temporary reservation. We should have test case for both full and interrupted transfer.
          3. FsVolumeImpl#recentReserved looks unused. Let's remove it.
          4. I didn't recall the state machine, do temporary blocks get finalized?
          Show
          arpitagarwal Arpit Agarwal added a comment - Thanks for updating the patch. A few comments. Nitpick: Please add spaces around operators replicaInfo!=null It doesn't look like there is a new test case for temporary reservation. We should have test case for both full and interrupted transfer. FsVolumeImpl#recentReserved looks unused. Let's remove it. I didn't recall the state machine, do temporary blocks get finalized?
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Thanks for the review Arpit Agarwal

          1. Ok. fine
          2. New tests are added for full success and volume errors. Please refer TestSpaceReservation#testTmpSpaceReserve()
          3. it is to be used in tests for verification for recent reserved space for last call to assert the actual block length instead of default block size.
          4. Yes, based on the code flow, block finalization is happening for re-replicated block also

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Thanks for the review Arpit Agarwal 1. Ok. fine 2. New tests are added for full success and volume errors. Please refer TestSpaceReservation#testTmpSpaceReserve() 3. it is to be used in tests for verification for recent reserved space for last call to assert the actual block length instead of default block size. 4. Yes, based on the code flow, block finalization is happening for re-replicated block also
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          patch v8 address TestSpaceReservation test failure in last build and comment from arpit for spaces around operators replicaInfo!=null

          Show
          kanaka Kanaka Kumar Avvaru added a comment - patch v8 address TestSpaceReservation test failure in last build and comment from arpit for spaces around operators replicaInfo!=null
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 40s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 7m 47s There were no new javac warning messages.
          +1 javadoc 10m 5s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 21s The applied patch generated 3 new checkstyle issues (total was 197, now 198).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 19s Pre-build of native portion
          -1 hdfs tests 149m 22s Tests failed in hadoop-hdfs.
              194m 35s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
            hadoop.hdfs.server.namenode.TestMetadataVersionOutput
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory
            hadoop.hdfs.server.namenode.TestMetaSave
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter
          Timed out tests org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12755469/HDFS-6955-08.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / ea4bb27
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12402/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12402/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12402/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12402/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 40s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 7m 47s There were no new javac warning messages. +1 javadoc 10m 5s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 21s The applied patch generated 3 new checkstyle issues (total was 197, now 198). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 19s Pre-build of native portion -1 hdfs tests 149m 22s Tests failed in hadoop-hdfs.     194m 35s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles   hadoop.hdfs.server.namenode.TestMetadataVersionOutput   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory   hadoop.hdfs.server.namenode.TestMetaSave   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter Timed out tests org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12755469/HDFS-6955-08.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / ea4bb27 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12402/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12402/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12402/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12402/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          deleted and uploaded v9 again to trigger jenkins build.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - deleted and uploaded v9 again to trigger jenkins build.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 16m 20s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 7m 59s There were no new javac warning messages.
          +1 javadoc 10m 10s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 32s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 11s Pre-build of native portion
          -1 hdfs tests 117m 52s Tests failed in hadoop-hdfs.
              161m 12s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory
            hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA
            hadoop.hdfs.web.TestWebHDFSOAuth2
            hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
          Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12756183/HDFS-6955-09.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 452079a
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12473/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12473/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12473/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 16m 20s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 7m 59s There were no new javac warning messages. +1 javadoc 10m 10s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 32s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 11s Pre-build of native portion -1 hdfs tests 117m 52s Tests failed in hadoop-hdfs.     161m 12s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory   hadoop.hdfs.server.namenode.ha.TestGetGroupsWithHA   hadoop.hdfs.web.TestWebHDFSOAuth2   hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12756183/HDFS-6955-09.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 452079a hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12473/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12473/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12473/console This message was automatically generated.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Test failures seems not related to the changes in this patch.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Test failures seems not related to the changes in this patch.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Seems there is a little conflict in committing this to branch-2.

          Kanaka Kumar Avvaru, Please post a branch-2 version of latest patch as well.

          -Thanks

          Show
          vinayrpet Vinayakumar B added a comment - Seems there is a little conflict in committing this to branch-2. Kanaka Kumar Avvaru , Please post a branch-2 version of latest patch as well. -Thanks
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Vinayakumar B, attached patch for branch-2 based on v9.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Vinayakumar B , attached patch for branch-2 based on v9.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Committed to trunk and branch-2.
          Thanks Kanaka Kumar Avvaru for the contribution.
          Thanks Arpit Agarwal for reviews.

          Show
          vinayrpet Vinayakumar B added a comment - Committed to trunk and branch-2. Thanks Kanaka Kumar Avvaru for the contribution. Thanks Arpit Agarwal for reviews.
          Hide
          kanaka Kanaka Kumar Avvaru added a comment -

          Thanks for the reviews & commit Vinayakumar B.
          Thanks for the reviews Arpit Agarwal.

          Show
          kanaka Kanaka Kumar Avvaru added a comment - Thanks for the reviews & commit Vinayakumar B . Thanks for the reviews Arpit Agarwal .
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8477 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8477/)
          HDFS-6955. DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8477 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8477/ ) HDFS-6955 . DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 43s Pre-patch branch-2 compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 5 new or modified test files.
          +1 javac 6m 3s There were no new javac warning messages.
          +1 javadoc 9m 57s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 23s The applied patch generated 2 new checkstyle issues (total was 200, now 199).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 13s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 1m 33s Pre-build of native portion
          -1 hdfs tests 189m 48s Tests failed in hadoop-hdfs.
              231m 11s  



          Reason Tests
          Failed unit tests hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.server.namenode.TestFileTruncate



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12757236/%0AHDFS-6955-branch-2.09.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision branch-2 / d94c092
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12539/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12539/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12539/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12539/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 43s Pre-patch branch-2 compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 5 new or modified test files. +1 javac 6m 3s There were no new javac warning messages. +1 javadoc 9m 57s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 23s The applied patch generated 2 new checkstyle issues (total was 200, now 199). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 13s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 29s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 33s Pre-build of native portion -1 hdfs tests 189m 48s Tests failed in hadoop-hdfs.     231m 11s   Reason Tests Failed unit tests hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.namenode.TestFileTruncate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12757236/%0AHDFS-6955-branch-2.09.patch Optional Tests javadoc javac unit findbugs checkstyle git revision branch-2 / d94c092 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12539/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12539/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12539/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12539/console This message was automatically generated.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1148 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1148/)
          HDFS-6955. DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1148 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1148/ ) HDFS-6955 . DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #407 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/407/)
          HDFS-6955. DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #407 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/407/ ) HDFS-6955 . DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #414 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/414/)
          HDFS-6955. DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #414 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/414/ ) HDFS-6955 . DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2354 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2354/)
          HDFS-6955. DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2354 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2354/ ) HDFS-6955 . DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #390 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/390/)
          HDFS-6955. DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #390 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/390/ ) HDFS-6955 . DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2329 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2329/)
          HDFS-6955. DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2329 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2329/ ) HDFS-6955 . DN should reserve disk space for a full block when creating tmp files (Contributed by Kanaka Kumar Avvaru) (vinayakumarb: rev 92c1af1646b1d91a2ab7821e4f7d450e3b6e10bb) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestRbwSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalVolumeImpl.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java

            People

            • Assignee:
              kanaka Kanaka Kumar Avvaru
              Reporter:
              arpitagarwal Arpit Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development