Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10271

Extra bytes are getting released from reservedSpace for append

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      1. File already have some bytes available in block. (ex: 1024B)
      2. Re-open the file for append, (Here reserving for (BlockSize-1024) bytes)
      3. write one byte and flush,
      4. close()

      After close(), releasing BlockSize-1 bytes from reservedspace instead of BlockSize-1025 bytes.
      Extra bytes released from reservedSpace may create problem for other writers.

      1. HDFS-10271-01.patch
        5 kB
        Brahma Reddy Battula
      2. HDFS-10271-branch-2.7-01.patch
        5 kB
        Brahma Reddy Battula

        Issue Links

          Activity

          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Attaching the patch.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Attaching the patch.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Kindly Review.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Kindly Review.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Attaching the patch for branch-2.7.
          In Branch-2.7 reservedSpaceForReplicas is still reservedSpaceForRBW.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Attaching the patch for branch-2.7. In Branch-2.7 reservedSpaceForReplicas is still reservedSpaceForRBW .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 9m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 5s branch-2.7 passed
          +1 compile 1m 18s branch-2.7 passed with JDK v1.8.0_77
          +1 compile 1m 9s branch-2.7 passed with JDK v1.7.0_95
          +1 checkstyle 0m 28s branch-2.7 passed
          +1 mvnsite 1m 6s branch-2.7 passed
          +1 mvneclipse 0m 13s branch-2.7 passed
          -1 findbugs 3m 14s hadoop-hdfs-project/hadoop-hdfs in branch-2.7 has 1 extant Findbugs warnings.
          +1 javadoc 1m 22s branch-2.7 passed with JDK v1.8.0_77
          +1 javadoc 2m 6s branch-2.7 passed with JDK v1.7.0_95
          +1 mvninstall 1m 4s the patch passed
          +1 compile 1m 14s the patch passed with JDK v1.8.0_77
          +1 javac 1m 14s the patch passed
          +1 compile 1m 3s the patch passed with JDK v1.7.0_95
          +1 javac 1m 3s the patch passed
          -1 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 127 unchanged - 2 fixed = 128 total (was 129)
          +1 mvnsite 1m 2s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          -1 whitespace 0m 1s The patch has 1200 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 whitespace 0m 29s The patch has 246 line(s) with tabs.
          +1 findbugs 3m 27s the patch passed
          +1 javadoc 1m 21s the patch passed with JDK v1.8.0_77
          +1 javadoc 2m 8s the patch passed with JDK v1.7.0_95
          -1 unit 68m 5s hadoop-hdfs in the patch failed with JDK v1.8.0_77.
          -1 unit 63m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          -1 asflicense 1m 6s Patch generated 65 ASF License warnings.
          175m 36s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.hdfs.web.TestWebHdfsTimeouts
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.qjournal.TestSecureNNWithQJM
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.datanode.TestBlockReplacement



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797688/HDFS-10271-branch-2.7-01.patch
          JIRA Issue HDFS-10271
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f91d3cf5352a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 262d769
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15112/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15112/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 9m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 5s branch-2.7 passed +1 compile 1m 18s branch-2.7 passed with JDK v1.8.0_77 +1 compile 1m 9s branch-2.7 passed with JDK v1.7.0_95 +1 checkstyle 0m 28s branch-2.7 passed +1 mvnsite 1m 6s branch-2.7 passed +1 mvneclipse 0m 13s branch-2.7 passed -1 findbugs 3m 14s hadoop-hdfs-project/hadoop-hdfs in branch-2.7 has 1 extant Findbugs warnings. +1 javadoc 1m 22s branch-2.7 passed with JDK v1.8.0_77 +1 javadoc 2m 6s branch-2.7 passed with JDK v1.7.0_95 +1 mvninstall 1m 4s the patch passed +1 compile 1m 14s the patch passed with JDK v1.8.0_77 +1 javac 1m 14s the patch passed +1 compile 1m 3s the patch passed with JDK v1.7.0_95 +1 javac 1m 3s the patch passed -1 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: patch generated 1 new + 127 unchanged - 2 fixed = 128 total (was 129) +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 11s the patch passed -1 whitespace 0m 1s The patch has 1200 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 0m 29s The patch has 246 line(s) with tabs. +1 findbugs 3m 27s the patch passed +1 javadoc 1m 21s the patch passed with JDK v1.8.0_77 +1 javadoc 2m 8s the patch passed with JDK v1.7.0_95 -1 unit 68m 5s hadoop-hdfs in the patch failed with JDK v1.8.0_77. -1 unit 63m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 asflicense 1m 6s Patch generated 65 ASF License warnings. 175m 36s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.qjournal.TestSecureNNWithQJM JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.datanode.TestBlockReplacement Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797688/HDFS-10271-branch-2.7-01.patch JIRA Issue HDFS-10271 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f91d3cf5352a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 262d769 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15112/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/15112/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15112/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 11m 5s trunk passed
          +1 compile 1m 33s trunk passed with JDK v1.8.0_77
          +1 compile 1m 13s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 1m 24s trunk passed
          +1 mvneclipse 0m 23s trunk passed
          +1 findbugs 2m 47s trunk passed
          +1 javadoc 1m 58s trunk passed with JDK v1.8.0_77
          +1 javadoc 3m 8s trunk passed with JDK v1.7.0_95
          +1 mvninstall 1m 16s the patch passed
          +1 compile 1m 28s the patch passed with JDK v1.8.0_77
          +1 javac 1m 28s the patch passed
          +1 compile 1m 3s the patch passed with JDK v1.7.0_95
          +1 javac 1m 3s the patch passed
          +1 checkstyle 0m 30s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 121 unchanged - 1 fixed = 121 total (was 122)
          +1 mvnsite 1m 21s the patch passed
          +1 mvneclipse 0m 20s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 7s the patch passed
          +1 javadoc 1m 52s the patch passed with JDK v1.8.0_77
          +1 javadoc 3m 4s the patch passed with JDK v1.7.0_95
          -1 unit 124m 6s hadoop-hdfs in the patch failed with JDK v1.8.0_77.
          -1 unit 82m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 1m 4s Patch does not generate ASF License warnings.
          250m 10s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.hdfs.qjournal.TestSecureNNWithQJM
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.server.datanode.TestDataNodeUUID
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
            hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.TestErasureCodeBenchmarkThroughput
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.web.TestWebHdfsTimeouts
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.server.datanode.TestDataNodeUUID
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2
            org.apache.hadoop.hdfs.TestDFSClientRetries
            org.apache.hadoop.hdfs.server.balancer.TestBalancer
            org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding
            org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797684/HDFS-10271-01.patch
          JIRA Issue HDFS-10271
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c9c8a1fbf0fd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 594c70f
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15111/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15111/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15111/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15111/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15111/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15111/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 11m 5s trunk passed +1 compile 1m 33s trunk passed with JDK v1.8.0_77 +1 compile 1m 13s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 24s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 2m 47s trunk passed +1 javadoc 1m 58s trunk passed with JDK v1.8.0_77 +1 javadoc 3m 8s trunk passed with JDK v1.7.0_95 +1 mvninstall 1m 16s the patch passed +1 compile 1m 28s the patch passed with JDK v1.8.0_77 +1 javac 1m 28s the patch passed +1 compile 1m 3s the patch passed with JDK v1.7.0_95 +1 javac 1m 3s the patch passed +1 checkstyle 0m 30s hadoop-hdfs-project/hadoop-hdfs: patch generated 0 new + 121 unchanged - 1 fixed = 121 total (was 122) +1 mvnsite 1m 21s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 7s the patch passed +1 javadoc 1m 52s the patch passed with JDK v1.8.0_77 +1 javadoc 3m 4s the patch passed with JDK v1.7.0_95 -1 unit 124m 6s hadoop-hdfs in the patch failed with JDK v1.8.0_77. -1 unit 82m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 1m 4s Patch does not generate ASF License warnings. 250m 10s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestErasureCodeBenchmarkThroughput   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2   org.apache.hadoop.hdfs.TestDFSClientRetries   org.apache.hadoop.hdfs.server.balancer.TestBalancer   org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding   org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797684/HDFS-10271-01.patch JIRA Issue HDFS-10271 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c9c8a1fbf0fd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 594c70f Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15111/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15111/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15111/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_77.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15111/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15111/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15111/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Can somebody from HDFS please help review / commit this? It's one of the last few blocking tickets for 2.7.3!

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Can somebody from HDFS please help review / commit this? It's one of the last few blocking tickets for 2.7.3!
          Hide
          raviprak Ravi Prakash added a comment -

          Hi Vinod!

          Counting of reservedSpace on Datanodes is all messed up right now I don't know if you should block on getting all these fixes in, because they may be a while.

          These issues already exist in 2.7.* (and I haven't heard very many complaints). There exists a workaround (restart the Datanode).

          I'm going to push out Target Version to 2.7.4 . Please revert if you disagree

          Show
          raviprak Ravi Prakash added a comment - Hi Vinod! Counting of reservedSpace on Datanodes is all messed up right now I don't know if you should block on getting all these fixes in, because they may be a while. These issues already exist in 2.7.* (and I haven't heard very many complaints). There exists a workaround (restart the Datanode). I'm going to push out Target Version to 2.7.4 . Please revert if you disagree
          Hide
          vinayrpet Vinayakumar B added a comment -

          This seems to be a pretty straight-forward bug.
          Simple Append op releases more reserved space than it actually should.

          Thanks Brahma Reddy Battula for nice find.
          +1 for the patch.
          I will commit it shortly.

          Show
          vinayrpet Vinayakumar B added a comment - This seems to be a pretty straight-forward bug. Simple Append op releases more reserved space than it actually should. Thanks Brahma Reddy Battula for nice find. +1 for the patch. I will commit it shortly.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Committed to trunk, branch-2, branch-2.8, branch-2.7 and branch-2.6.

          Thanks Brahma Reddy Battula for the contribution.
          Thanks Ravi Prakash for the participation.

          Show
          vinayrpet Vinayakumar B added a comment - Committed to trunk, branch-2, branch-2.8, branch-2.7 and branch-2.6. Thanks Brahma Reddy Battula for the contribution. Thanks Ravi Prakash for the participation.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9596 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9596/)
          HDFS-10271. Extra bytes are getting released from reservedSpace for (vinayakumarb: rev a9a607f8fc0d996af3fb37f7efa7591d6655900d)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9596 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9596/ ) HDFS-10271 . Extra bytes are getting released from reservedSpace for (vinayakumarb: rev a9a607f8fc0d996af3fb37f7efa7591d6655900d) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestSpaceReservation.java
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Vinayakumar B thanks a lot for review and commit..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Vinayakumar B thanks a lot for review and commit..
          Show
          raviprak Ravi Prakash added a comment - Here are the other places that I thought reserved space should have been modified : https://issues.apache.org/jira/browse/HDFS-9530?focusedCommentId=15231164&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15231164 .
          Hide
          vinayrpet Vinayakumar B added a comment -

          I think brahma has replied for those questions in HDFS-9530 here.
          Perhaps, this issue is straight forward, and related to append.
          So we shall continue that discussion in HDFS-9530 itself.

          Show
          vinayrpet Vinayakumar B added a comment - I think brahma has replied for those questions in HDFS-9530 here . Perhaps, this issue is straight forward, and related to append. So we shall continue that discussion in HDFS-9530 itself.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

            People

            • Assignee:
              brahmareddy Brahma Reddy Battula
              Reporter:
              brahmareddy Brahma Reddy Battula
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development