Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.7.4, 3.0.0-alpha2
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      Hide
      The fix for HDFS-11056 reads meta file to load last partial chunk checksum when a block is converted from finalized/temporary to rbw. However, it did not close the file explicitly, which may cause number of open files reaching system limit. This jira fixes it by closing the file explicitly after the meta file is read.
      Show
      The fix for HDFS-11056 reads meta file to load last partial chunk checksum when a block is converted from finalized/temporary to rbw. However, it did not close the file explicitly, which may cause number of open files reaching system limit. This jira fixes it by closing the file explicitly after the meta file is read.

      Description

      The following code failed to close the file after it is read.

      FsVolumeImpl#loadLastPartialChunkChecksum
          RandomAccessFile raf = new RandomAccessFile(metaFile, "r");
          raf.seek(offsetInChecksum);
          raf.read(lastChecksum, 0, checksumSize);
          return lastChecksum;
      

      This must be fixed because every append operation uses this piece of code. Without an explicit close, open files can reach system limit before RandomAccessFile objects are garbage collected.

      1. HDFS-11229.001.patch
        1 kB
        Wei-Chiu Chuang
      2. HDFS-11229.branch-2.patch
        1 kB
        Wei-Chiu Chuang

        Issue Links

          Activity

          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Upload patch v001. Use try-with-resources statement to close the file.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Upload patch v001. Use try-with-resources statement to close the file.
          Hide
          yzhangal Yongjun Zhang added a comment -

          HI Wei-Chiu,

          Thanks for the patch, I thought we want to do:

          RandomAccessFile raf = null;
          try {
            raf = new RandomAccessFile(metaFile, "r");		
            raf.seek(offsetInChecksum);		
            raf.read(lastChecksum, 0, checksumSize);
          } finally {
            if (raf != null) {
              try {
                raf.close();
              } catch (IOException e) {
                // issue warning message here
              }
            }
          }
          

          I thought about using IOUtils.cleanup(raf); however, this method calls LOG.debug, to issue message. I think WARN is better.

          Show
          yzhangal Yongjun Zhang added a comment - HI Wei-Chiu, Thanks for the patch, I thought we want to do: RandomAccessFile raf = null ; try { raf = new RandomAccessFile(metaFile, "r" ); raf.seek(offsetInChecksum); raf.read(lastChecksum, 0, checksumSize); } finally { if (raf != null ) { try { raf.close(); } catch (IOException e) { // issue warning message here } } } I thought about using IOUtils.cleanup(raf); however, this method calls LOG.debug, to issue message. I think WARN is better.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Ha, I got you, in java 7, "Any variable declared in the try block will automatically have close() called when the try block exits.", so your patch looks good to me now. Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Ha, I got you, in java 7, "Any variable declared in the try block will automatically have close() called when the try block exits.", so your patch looks good to me now. Thanks.
          Hide
          andrew.wang Andrew Wang added a comment -

          +1 LGTM too. Please also set the targets versions in the future, since this looks like a 3.0.0-alpha2 blocker as well.

          Show
          andrew.wang Andrew Wang added a comment - +1 LGTM too. Please also set the targets versions in the future, since this looks like a 3.0.0-alpha2 blocker as well.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 23s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 25s trunk passed
          +1 mvnsite 0m 52s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 42s trunk passed
          +1 javadoc 0m 40s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          +1 checkstyle 0m 23s the patch passed
          +1 mvnsite 0m 50s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 47s the patch passed
          +1 javadoc 0m 36s the patch passed
          -1 unit 62m 35s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          81m 47s



          Reason Tests
          Failed junit tests hadoop.hdfs.web.TestWebHdfsFileSystemContract



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11229
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842608/HDFS-11229.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 8e973edff6bc 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 5bd7dec
          Default Java 1.8.0_111
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17812/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17812/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17812/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 23s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 36s the patch passed -1 unit 62m 35s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 81m 47s Reason Tests Failed junit tests hadoop.hdfs.web.TestWebHdfsFileSystemContract Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11229 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842608/HDFS-11229.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8e973edff6bc 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5bd7dec Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17812/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17812/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17812/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          The failed test looks unrelated and can't be reproduced locally. I will file a jira though.

          Show
          jojochuang Wei-Chiu Chuang added a comment - The failed test looks unrelated and can't be reproduced locally. I will file a jira though.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Branch 2 patch

          Show
          jojochuang Wei-Chiu Chuang added a comment - Branch 2 patch
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10979 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10979/)
          HDFS-11229. HDFS-11056 failed to close meta file. Contributed by (weichiu: rev 2a28e8cf0469a373a99011f0fa540474e60528c8)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10979 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10979/ ) HDFS-11229 . HDFS-11056 failed to close meta file. Contributed by (weichiu: rev 2a28e8cf0469a373a99011f0fa540474e60528c8) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 59s branch-2 passed
          +1 compile 0m 49s branch-2 passed with JDK v1.8.0_111
          +1 compile 0m 43s branch-2 passed with JDK v1.7.0_121
          +1 checkstyle 0m 30s branch-2 passed
          +1 mvnsite 0m 55s branch-2 passed
          +1 mvneclipse 0m 17s branch-2 passed
          +1 findbugs 2m 3s branch-2 passed
          +1 javadoc 0m 58s branch-2 passed with JDK v1.8.0_111
          +1 javadoc 1m 46s branch-2 passed with JDK v1.7.0_121
          +1 mvninstall 0m 55s the patch passed
          +1 compile 0m 47s the patch passed with JDK v1.8.0_111
          +1 javac 0m 47s the patch passed
          +1 compile 0m 46s the patch passed with JDK v1.7.0_121
          +1 javac 0m 46s the patch passed
          +1 checkstyle 0m 31s the patch passed
          +1 mvnsite 0m 57s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 18s the patch passed
          +1 javadoc 0m 58s the patch passed with JDK v1.8.0_111
          +1 javadoc 1m 36s the patch passed with JDK v1.7.0_121
          -1 unit 75m 53s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
          +1 asflicense 0m 25s The patch does not generate ASF License warnings.
          179m 8s



          Reason Tests
          JDK v1.8.0_111 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
          JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
            hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Issue HDFS-11229
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842626/HDFS-11229.branch-2.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 011afe8f4436 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 85adc66
          Default Java 1.7.0_121
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17815/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
          JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17815/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17815/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 59s branch-2 passed +1 compile 0m 49s branch-2 passed with JDK v1.8.0_111 +1 compile 0m 43s branch-2 passed with JDK v1.7.0_121 +1 checkstyle 0m 30s branch-2 passed +1 mvnsite 0m 55s branch-2 passed +1 mvneclipse 0m 17s branch-2 passed +1 findbugs 2m 3s branch-2 passed +1 javadoc 0m 58s branch-2 passed with JDK v1.8.0_111 +1 javadoc 1m 46s branch-2 passed with JDK v1.7.0_121 +1 mvninstall 0m 55s the patch passed +1 compile 0m 47s the patch passed with JDK v1.8.0_111 +1 javac 0m 47s the patch passed +1 compile 0m 46s the patch passed with JDK v1.7.0_121 +1 javac 0m 46s the patch passed +1 checkstyle 0m 31s the patch passed +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 18s the patch passed +1 javadoc 0m 58s the patch passed with JDK v1.8.0_111 +1 javadoc 1m 36s the patch passed with JDK v1.7.0_121 -1 unit 75m 53s hadoop-hdfs in the patch failed with JDK v1.7.0_121. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 179m 8s Reason Tests JDK v1.8.0_111 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength JDK v1.7.0_121 Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandby   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HDFS-11229 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842626/HDFS-11229.branch-2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 011afe8f4436 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 85adc66 Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17815/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17815/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17815/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          The test failures are not related.

          Show
          jojochuang Wei-Chiu Chuang added a comment - The test failures are not related.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Committed the patch to trunk, branch-2, branch-2.8 and branch-2.7.
          Thanks Yongjun Zhang and Andrew Wang for the quick review!

          Show
          jojochuang Wei-Chiu Chuang added a comment - Committed the patch to trunk, branch-2, branch-2.8 and branch-2.7. Thanks Yongjun Zhang and Andrew Wang for the quick review!

            People

            • Assignee:
              jojochuang Wei-Chiu Chuang
              Reporter:
              jojochuang Wei-Chiu Chuang
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development