Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11711

DN should not delete the block On "Too many open files" Exception

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Seen the following scenario in one of our customer environment

      • while jobclient writing "job.xml" there are pipeline failures and written to only one DN.
      • when mapper reading the "job.xml", DN got "Too many open files" (as system exceed limit) and block got deleted. Hence mapper failed to read and job got failed.
      1. HDFS-11711-branch-2-003.patch
        7 kB
        Brahma Reddy Battula
      2. HDFS-11711-branch-2-002.patch
        7 kB
        Brahma Reddy Battula
      3. HDFS-11711-004.patch
        7 kB
        Brahma Reddy Battula
      4. HDFS-11711-003.patch
        7 kB
        Brahma Reddy Battula
      5. HDFS-11711-002.patch
        7 kB
        Brahma Reddy Battula
      6. HDFS-11711.patch
        1 kB
        Brahma Reddy Battula

        Issue Links

          Activity

          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          it should just throw a new type of exception in these two cases.

          Looks this better, we can have different type of exception .Instead of deleting on FNFE, Validate the file existence before opening stream, and then throw different exception..?

          Show
          brahmareddy Brahma Reddy Battula added a comment - it should just throw a new type of exception in these two cases. Looks this better, we can have different type of exception .Instead of deleting on FNFE, Validate the file existence before opening stream, and then throw different exception..?
          Hide
          jojochuang Wei-Chiu Chuang added a comment - - edited

          Brahma Reddy Battula sorry i didn't make myself clear.
          To begin with, this behavior was caused by HDFS-8492, which throws FileNotFoundException("BlockId " + blockId + " is not valid.").
          I was just thinking that "Too many open files" error is thrown within Java library, so there's no guarantee this would be compatible between different operating systems, or across different Java versions, or different JVM/JDK implementation.

          IMHO, the more compatible approach would be that we check if FNFE has "BlockId " + blockId + " is not valid.", and only delete the block when that's the case.

          Edit: HDFS-3100 throws FileNotFoundException("Meta-data not found for " + block) when meta file checksum is not found. So this should be checked as well.

          Or, it should just throw a new type of exception in these two cases.

          Show
          jojochuang Wei-Chiu Chuang added a comment - - edited Brahma Reddy Battula sorry i didn't make myself clear. To begin with, this behavior was caused by HDFS-8492 , which throws FileNotFoundException("BlockId " + blockId + " is not valid."). I was just thinking that "Too many open files" error is thrown within Java library, so there's no guarantee this would be compatible between different operating systems, or across different Java versions, or different JVM/JDK implementation. IMHO, the more compatible approach would be that we check if FNFE has "BlockId " + blockId + " is not valid.", and only delete the block when that's the case. Edit: HDFS-3100 throws FileNotFoundException("Meta-data not found for " + block) when meta file checksum is not found. So this should be checked as well. Or, it should just throw a new type of exception in these two cases.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Committed to trunk,branch-2,branch-2.8 and branch-2.8.1. Test failures are unrelated

          Thanks to all for your great reviews.

          Must throw File not found would be better updated to Must throw FileNotFoundException,

          Should throw too many open would be better updated to Should throw too many open files

          Addressed these minor nits while committing.

          But I wish there's a more portable way to check for Too many open files error.

          do you mean, it's already there..?

          Show
          brahmareddy Brahma Reddy Battula added a comment - Committed to trunk , branch-2 , branch-2.8 and branch-2.8.1 . Test failures are unrelated Thanks to all for your great reviews. Must throw File not found would be better updated to Must throw FileNotFoundException, Should throw too many open would be better updated to Should throw too many open files Addressed these minor nits while committing. But I wish there's a more portable way to check for Too many open files error. do you mean, it's already there..?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 34s branch-2 passed
          +1 compile 0m 48s branch-2 passed with JDK v1.8.0_131
          +1 compile 0m 46s branch-2 passed with JDK v1.7.0_131
          +1 checkstyle 0m 31s branch-2 passed
          +1 mvnsite 0m 56s branch-2 passed
          +1 findbugs 2m 6s branch-2 passed
          +1 javadoc 0m 42s branch-2 passed with JDK v1.8.0_131
          +1 javadoc 1m 4s branch-2 passed with JDK v1.7.0_131
          +1 mvninstall 0m 49s the patch passed
          +1 compile 0m 45s the patch passed with JDK v1.8.0_131
          +1 javac 0m 45s the patch passed
          +1 compile 0m 43s the patch passed with JDK v1.7.0_131
          +1 javac 0m 43s the patch passed
          +1 checkstyle 0m 25s the patch passed
          +1 mvnsite 0m 48s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 9s the patch passed
          +1 javadoc 0m 36s the patch passed with JDK v1.8.0_131
          +1 javadoc 0m 57s the patch passed with JDK v1.7.0_131
          -1 unit 48m 15s hadoop-hdfs in the patch failed with JDK v1.7.0_131.
          -1 asflicense 0m 20s The patch generated 1 ASF License warnings.
          129m 4s



          Reason Tests
          JDK v1.8.0_131 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.balancer.TestBalancerRPCDelay
            hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5e40efe
          JIRA Issue HDFS-11711
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871755/HDFS-11711-branch-2-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 99e2d0ea8fca 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 72f0fdf
          Default Java 1.7.0_131
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19815/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19815/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/19815/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19815/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 34s branch-2 passed +1 compile 0m 48s branch-2 passed with JDK v1.8.0_131 +1 compile 0m 46s branch-2 passed with JDK v1.7.0_131 +1 checkstyle 0m 31s branch-2 passed +1 mvnsite 0m 56s branch-2 passed +1 findbugs 2m 6s branch-2 passed +1 javadoc 0m 42s branch-2 passed with JDK v1.8.0_131 +1 javadoc 1m 4s branch-2 passed with JDK v1.7.0_131 +1 mvninstall 0m 49s the patch passed +1 compile 0m 45s the patch passed with JDK v1.8.0_131 +1 javac 0m 45s the patch passed +1 compile 0m 43s the patch passed with JDK v1.7.0_131 +1 javac 0m 43s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 48s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 9s the patch passed +1 javadoc 0m 36s the patch passed with JDK v1.8.0_131 +1 javadoc 0m 57s the patch passed with JDK v1.7.0_131 -1 unit 48m 15s hadoop-hdfs in the patch failed with JDK v1.7.0_131. -1 asflicense 0m 20s The patch generated 1 ASF License warnings. 129m 4s Reason Tests JDK v1.8.0_131 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.balancer.TestBalancerRPCDelay   hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain Subsystem Report/Notes Docker Image:yetus/hadoop:5e40efe JIRA Issue HDFS-11711 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871755/HDFS-11711-branch-2-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 99e2d0ea8fca 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 72f0fdf Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/19815/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19815/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/19815/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19815/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 20m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 21s branch-2 passed
          +1 compile 0m 42s branch-2 passed with JDK v1.8.0_131
          +1 compile 0m 46s branch-2 passed with JDK v1.7.0_131
          +1 checkstyle 0m 30s branch-2 passed
          +1 mvnsite 0m 56s branch-2 passed
          +1 findbugs 2m 6s branch-2 passed
          +1 javadoc 0m 41s branch-2 passed with JDK v1.8.0_131
          +1 javadoc 1m 3s branch-2 passed with JDK v1.7.0_131
          +1 mvninstall 0m 50s the patch passed
          +1 compile 0m 47s the patch passed with JDK v1.8.0_131
          +1 javac 0m 46s the patch passed
          +1 compile 0m 49s the patch passed with JDK v1.7.0_131
          +1 javac 0m 49s the patch passed
          +1 checkstyle 0m 29s the patch passed
          +1 mvnsite 1m 0s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 31s the patch passed
          +1 javadoc 0m 39s the patch passed with JDK v1.8.0_131
          +1 javadoc 1m 7s the patch passed with JDK v1.7.0_131
          -1 unit 70m 23s hadoop-hdfs in the patch failed with JDK v1.7.0_131.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          184m 44s



          Reason Tests
          JDK v1.7.0_131 Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs
            hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain
          JDK v1.7.0_131 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5e40efe
          JIRA Issue HDFS-11711
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871755/HDFS-11711-branch-2-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux ed79381f43d5 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 5c81958
          Default Java 1.7.0_131
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19814/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19814/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19814/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 20m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 21s branch-2 passed +1 compile 0m 42s branch-2 passed with JDK v1.8.0_131 +1 compile 0m 46s branch-2 passed with JDK v1.7.0_131 +1 checkstyle 0m 30s branch-2 passed +1 mvnsite 0m 56s branch-2 passed +1 findbugs 2m 6s branch-2 passed +1 javadoc 0m 41s branch-2 passed with JDK v1.8.0_131 +1 javadoc 1m 3s branch-2 passed with JDK v1.7.0_131 +1 mvninstall 0m 50s the patch passed +1 compile 0m 47s the patch passed with JDK v1.8.0_131 +1 javac 0m 46s the patch passed +1 compile 0m 49s the patch passed with JDK v1.7.0_131 +1 javac 0m 49s the patch passed +1 checkstyle 0m 29s the patch passed +1 mvnsite 1m 0s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 31s the patch passed +1 javadoc 0m 39s the patch passed with JDK v1.8.0_131 +1 javadoc 1m 7s the patch passed with JDK v1.7.0_131 -1 unit 70m 23s hadoop-hdfs in the patch failed with JDK v1.7.0_131. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 184m 44s Reason Tests JDK v1.7.0_131 Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs   hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain JDK v1.7.0_131 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:5e40efe JIRA Issue HDFS-11711 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871755/HDFS-11711-branch-2-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ed79381f43d5 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 5c81958 Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/19814/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19814/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19814/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11836 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11836/)
          HDFS-11711. DN should not delete the block On "Too many open files" (brahma: rev 1869e1771c7eeea46ccb822ce6f7081d994bb12c)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11836 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11836/ ) HDFS-11711 . DN should not delete the block On "Too many open files" (brahma: rev 1869e1771c7eeea46ccb822ce6f7081d994bb12c) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMetrics.java
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Updating the branch-2 patch which address the minor nits also.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Updating the branch-2 patch which address the minor nits also.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          +1 too. Thanks for the patch. I think the fix is good. But I wish there's a more portable way to check for Too many open files error.

          Show
          jojochuang Wei-Chiu Chuang added a comment - +1 too. Thanks for the patch. I think the fix is good. But I wish there's a more portable way to check for Too many open files error.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 25s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 14m 3s trunk passed
          +1 compile 0m 51s trunk passed
          +1 checkstyle 0m 38s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 findbugs 1m 46s trunk passed
          +1 javadoc 0m 40s trunk passed
          +1 mvninstall 0m 53s the patch passed
          +1 compile 0m 49s the patch passed
          +1 javac 0m 49s the patch passed
          +1 checkstyle 0m 35s the patch passed
          +1 mvnsite 0m 53s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 51s the patch passed
          +1 javadoc 0m 40s the patch passed
          -1 unit 64m 16s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          91m 1s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11711
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871741/HDFS-11711-004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 703a3c0342ea 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b65100c
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19810/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19810/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19810/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 3s trunk passed +1 compile 0m 51s trunk passed +1 checkstyle 0m 38s trunk passed +1 mvnsite 0m 56s trunk passed +1 findbugs 1m 46s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 53s the patch passed +1 compile 0m 49s the patch passed +1 javac 0m 49s the patch passed +1 checkstyle 0m 35s the patch passed +1 mvnsite 0m 53s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 51s the patch passed +1 javadoc 0m 40s the patch passed -1 unit 64m 16s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 91m 1s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11711 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871741/HDFS-11711-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 703a3c0342ea 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b65100c Default Java 1.8.0_131 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/19810/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19810/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19810/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks Brahma Reddy Battula for the updated patch.
          Found two nits:

          try {
          +        new BlockSender(b, 0, -1, false, true, true,
          +            cluster.getDataNodes().get(0), null,
          +            CachingStrategy.newDefaultStrategy());
          +        fail("Must throw File not found");
          +      } catch (FileNotFoundException fe) {
          +        assertTrue("Should throw too many open",
          +            fe.getMessage().contains("Too many open files"));
          +      }
          

          Must throw File not found would be better updated to Must throw FileNotFoundException,
          Should throw too many open would be better updated to Should throw too many open files
          You can just update this while committing, . Thanks

          Show
          linyiqun Yiqun Lin added a comment - Thanks Brahma Reddy Battula for the updated patch. Found two nits: try { + new BlockSender(b, 0, -1, false , true , true , + cluster.getDataNodes().get(0), null , + CachingStrategy.newDefaultStrategy()); + fail( "Must throw File not found" ); + } catch (FileNotFoundException fe) { + assertTrue( "Should throw too many open" , + fe.getMessage().contains( "Too many open files" )); + } Must throw File not found would be better updated to Must throw FileNotFoundException , Should throw too many open would be better updated to Should throw too many open files You can just update this while committing, . Thanks
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Updated the patch to increase the third param value.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Updated the patch to increase the third param value.
          Hide
          shahrs87 Rushabh S Shah added a comment - - edited

          The third param vaule should be more than the second param i

          Yiqun Lin: Good catch. We should fix this.
          Does it make sense to open a new jira for GenericTestUtils to have some assert or preconditions check to make sure 3rd param is always greater than 2nd param.

          Show
          shahrs87 Rushabh S Shah added a comment - - edited The third param vaule should be more than the second param i Yiqun Lin : Good catch. We should fix this. Does it make sense to open a new jira for GenericTestUtils to have some assert or preconditions check to make sure 3rd param is always greater than 2nd param.
          Hide
          linyiqun Yiqun Lin added a comment -

          Hi Brahma Reddy Battula, just one comment from me:
          The third param vaule should be more than the second param in {{GenericTestUtils#waitFor(Supplier<Boolean> check,
          int checkEveryMillis, int waitForMillis)}}, now 600ms < 1000ms, maybe you just want to use 6000 here?

          +  private void verifyBlockLocations(DistributedFileSystem fs, Path p,
          +      int expected) throws IOException, TimeoutException, InterruptedException {
          +    final LocatedBlock lb =
          +        fs.getClient().getLocatedBlocks(p.toString(), 0).get(0);
          +    GenericTestUtils.waitFor(new Supplier<Boolean>() {
          +      @Override
          +      public Boolean get() {
          +        return lb.getLocations().length == expected;
          +      }
          +    }, 1000, 600);
          
          Show
          linyiqun Yiqun Lin added a comment - Hi Brahma Reddy Battula , just one comment from me: The third param vaule should be more than the second param in {{GenericTestUtils#waitFor(Supplier<Boolean> check, int checkEveryMillis, int waitForMillis)}}, now 600ms < 1000ms, maybe you just want to use 6000 here? + private void verifyBlockLocations(DistributedFileSystem fs, Path p, + int expected) throws IOException, TimeoutException, InterruptedException { + final LocatedBlock lb = + fs.getClient().getLocatedBlocks(p.toString(), 0).get(0); + GenericTestUtils.waitFor( new Supplier< Boolean >() { + @Override + public Boolean get() { + return lb.getLocations().length == expected; + } + }, 1000, 600);
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Actually I can go-head with Vinay earlier +1..

          Since only test code changed after Vinayakumar B's +1, it should be fine.

          Show
          shahrs87 Rushabh S Shah added a comment - Actually I can go-head with Vinay earlier +1.. Since only test code changed after Vinayakumar B 's +1, it should be fine.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Rushabh S Shah I will take care check-style while committing the patch.Testfailures are unrelated and thanks for your verification.

          I will hold the commit,till somebody else review..Actually I can go-head with Vinay earlier +1..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Rushabh S Shah I will take care check-style while committing the patch. Testfailures are unrelated and thanks for your verification. I will hold the commit,till somebody else review..Actually I can go-head with Vinay earlier +1..
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Brahma: the latest patch looks good.
          There is one checkstyle warning (in branch-2 patch) and 1 more warning (in trunk patch) that needs to be addressed. I wouldn't submit a new patch and waste build resources for that.
          The committer can fix it while checking in.
          I verified the tests (that failed from branch-2 patch) passes locally for me.
          Brahma: Can you verify test failures from trunk patch are not related to your patch ?
          Other than that, +1 (non-binding) from me.
          Thanks !

          Show
          shahrs87 Rushabh S Shah added a comment - Brahma : the latest patch looks good. There is one checkstyle warning (in branch-2 patch) and 1 more warning (in trunk patch) that needs to be addressed. I wouldn't submit a new patch and waste build resources for that. The committer can fix it while checking in. I verified the tests (that failed from branch-2 patch) passes locally for me. Brahma : Can you verify test failures from trunk patch are not related to your patch ? Other than that, +1 (non-binding) from me. Thanks !
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 14m 49s trunk passed
          +1 compile 1m 2s trunk passed
          +1 checkstyle 0m 39s trunk passed
          +1 mvnsite 0m 58s trunk passed
          +1 findbugs 1m 49s trunk passed
          +1 javadoc 0m 45s trunk passed
          +1 mvninstall 0m 59s the patch passed
          +1 compile 0m 53s the patch passed
          +1 javac 0m 53s the patch passed
          -0 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 52 unchanged - 0 fixed = 53 total (was 52)
          +1 mvnsite 0m 58s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 52s the patch passed
          +1 javadoc 0m 39s the patch passed
          -1 unit 85m 41s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          113m 43s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
            hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
            hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11711
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871534/HDFS-11711-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 85d059e6a20e 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 66c6fd8
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19793/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19793/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19793/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19793/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 49s trunk passed +1 compile 1m 2s trunk passed +1 checkstyle 0m 39s trunk passed +1 mvnsite 0m 58s trunk passed +1 findbugs 1m 49s trunk passed +1 javadoc 0m 45s trunk passed +1 mvninstall 0m 59s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed -0 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 52 unchanged - 0 fixed = 53 total (was 52) +1 mvnsite 0m 58s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 52s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 85m 41s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 113m 43s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11711 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871534/HDFS-11711-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 85d059e6a20e 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 66c6fd8 Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19793/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19793/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19793/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19793/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 18s branch-2 passed
          +1 compile 0m 46s branch-2 passed with JDK v1.8.0_131
          +1 compile 0m 46s branch-2 passed with JDK v1.7.0_131
          +1 checkstyle 0m 31s branch-2 passed
          +1 mvnsite 0m 57s branch-2 passed
          +1 findbugs 2m 3s branch-2 passed
          +1 javadoc 0m 40s branch-2 passed with JDK v1.8.0_131
          +1 javadoc 1m 3s branch-2 passed with JDK v1.7.0_131
          +1 mvninstall 0m 48s the patch passed
          +1 compile 0m 42s the patch passed with JDK v1.8.0_131
          +1 javac 0m 42s the patch passed
          +1 compile 0m 45s the patch passed with JDK v1.7.0_131
          +1 javac 0m 45s the patch passed
          -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 56 unchanged - 0 fixed = 58 total (was 56)
          +1 mvnsite 0m 54s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 15s the patch passed
          +1 javadoc 0m 40s the patch passed with JDK v1.8.0_131
          +1 javadoc 1m 3s the patch passed with JDK v1.7.0_131
          -1 unit 57m 8s hadoop-hdfs in the patch failed with JDK v1.7.0_131.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          134m 31s



          Reason Tests
          JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.datanode.metrics.TestDataNodeOutlierDetectionViaMetrics
            hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain
          JDK v1.7.0_131 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5e40efe
          JIRA Issue HDFS-11711
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871526/HDFS-11711-branch-2-002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux df2439aaa080 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / e889c82
          Default Java 1.7.0_131
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19792/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19792/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19792/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19792/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 18s branch-2 passed +1 compile 0m 46s branch-2 passed with JDK v1.8.0_131 +1 compile 0m 46s branch-2 passed with JDK v1.7.0_131 +1 checkstyle 0m 31s branch-2 passed +1 mvnsite 0m 57s branch-2 passed +1 findbugs 2m 3s branch-2 passed +1 javadoc 0m 40s branch-2 passed with JDK v1.8.0_131 +1 javadoc 1m 3s branch-2 passed with JDK v1.7.0_131 +1 mvninstall 0m 48s the patch passed +1 compile 0m 42s the patch passed with JDK v1.8.0_131 +1 javac 0m 42s the patch passed +1 compile 0m 45s the patch passed with JDK v1.7.0_131 +1 javac 0m 45s the patch passed -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 56 unchanged - 0 fixed = 58 total (was 56) +1 mvnsite 0m 54s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 15s the patch passed +1 javadoc 0m 40s the patch passed with JDK v1.8.0_131 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_131 -1 unit 57m 8s hadoop-hdfs in the patch failed with JDK v1.7.0_131. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 134m 31s Reason Tests JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.datanode.metrics.TestDataNodeOutlierDetectionViaMetrics   hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain JDK v1.7.0_131 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:5e40efe JIRA Issue HDFS-11711 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871526/HDFS-11711-branch-2-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux df2439aaa080 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / e889c82 Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19792/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19792/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19792/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19792/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Uploading patch to fix the check-style issues.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Uploading patch to fix the check-style issues.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 14m 15s trunk passed
          +1 compile 0m 51s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 findbugs 1m 49s trunk passed
          +1 javadoc 0m 40s trunk passed
          +1 mvninstall 0m 50s the patch passed
          +1 compile 0m 48s the patch passed
          +1 javac 0m 48s the patch passed
          -0 checkstyle 0m 34s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 52 unchanged - 0 fixed = 56 total (was 52)
          +1 mvnsite 0m 51s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 46s the patch passed
          +1 javadoc 0m 37s the patch passed
          -1 unit 63m 2s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          89m 39s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy
            hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11711
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871510/HDFS-11711-002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 5ba772d5389a 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 66c6fd8
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19791/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19791/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19791/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19791/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 15s trunk passed +1 compile 0m 51s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 0m 56s trunk passed +1 findbugs 1m 49s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 50s the patch passed +1 compile 0m 48s the patch passed +1 javac 0m 48s the patch passed -0 checkstyle 0m 34s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 52 unchanged - 0 fixed = 56 total (was 52) +1 mvnsite 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 63m 2s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 89m 39s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090   hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy   hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11711 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12871510/HDFS-11711-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5ba772d5389a 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 66c6fd8 Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19791/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19791/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19791/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19791/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Attaching the branch-2 Patch.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Attaching the branch-2 Patch.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Can we add some unit tests for this behavior ?

          Uploaded the patch with unit testcase.. Kindly review.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Can we add some unit tests for this behavior ? Uploaded the patch with unit testcase.. Kindly review.
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Hi Brahma Reddy Battula
          The patch looks good to me.
          Can we add some unit tests for this behavior ?

          Show
          shahrs87 Rushabh S Shah added a comment - Hi Brahma Reddy Battula The patch looks good to me. Can we add some unit tests for this behavior ?
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Vinayakumar B thanks for review.

          Also this should be marked as critical

          done.

          Will commit tomorrow unless there no objections.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Vinayakumar B thanks for review. Also this should be marked as critical done. Will commit tomorrow unless there no objections.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Changes looks good to me. +1.
          Also this should be marked as critical, as 'replica' deletion may lead to missing blocks if in case other nodes are not available.

          Show
          vinayrpet Vinayakumar B added a comment - Changes looks good to me. +1. Also this should be marked as critical, as 'replica' deletion may lead to missing blocks if in case other nodes are not available.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Test Failures and ASF Liceneses are unrelated.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Test Failures and ASF Liceneses are unrelated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 24s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 13m 18s trunk passed
          +1 compile 0m 46s trunk passed
          +1 checkstyle 0m 35s trunk passed
          +1 mvnsite 0m 53s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          -1 findbugs 1m 38s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
          +1 javadoc 0m 39s trunk passed
          +1 mvninstall 0m 47s the patch passed
          +1 compile 0m 44s the patch passed
          +1 javac 0m 44s the patch passed
          -0 checkstyle 0m 33s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 37 unchanged - 1 fixed = 39 total (was 38)
          +1 mvnsite 0m 50s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 43s the patch passed
          +1 javadoc 0m 38s the patch passed
          -1 unit 70m 14s hadoop-hdfs in the patch failed.
          -1 asflicense 0m 20s The patch generated 1 ASF License warnings.
          95m 43s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11711
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867317/HDFS-11711.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 0a1f06fe9bb9 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 2ba9903
          Default Java 1.8.0_121
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19380/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19380/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19380/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19380/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/19380/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19380/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 13m 18s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 35s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 14s trunk passed -1 findbugs 1m 38s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 0m 39s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 33s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 37 unchanged - 1 fixed = 39 total (was 38) +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 43s the patch passed +1 javadoc 0m 38s the patch passed -1 unit 70m 14s hadoop-hdfs in the patch failed. -1 asflicense 0m 20s The patch generated 1 ASF License warnings. 95m 43s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11711 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867317/HDFS-11711.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0a1f06fe9bb9 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2ba9903 Default Java 1.8.0_121 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19380/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19380/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19380/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19380/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/19380/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19380/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Uploaded the patch..Kindly Review.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Uploaded the patch..Kindly Review.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          FYI .
          Following is the log from DN

          2017-04-25 07:02:44,610 | ERROR | DataXceiver for client DFSClient_NONMAPREDUCE_222700060_28 at /192.168.100.48:18124 [Sending block BP-262396492-192.168.100.42-1490663057778:blk_1078953155_5605334] | datanode5:25009:DataXceiver error processing READ_BLOCK operation  src: /192.168.100.48:18124 dst: /192.168.100.48:25009 | DataXceiver.java:304
          java.io.FileNotFoundException: /srv/BigData/hadoop/data9/dn/current/BP-262396492-192.168.100.42-1490663057778/current/finalized/subdir79/subdir132/blk_1078953155_5605334.meta (Too many open files)
          	at java.io.FileInputStream.open0(Native Method)
          	at java.io.FileInputStream.open(FileInputStream.java:195)
          	at java.io.FileInputStream.<init>(FileInputStream.java:138)
          	at org.apache.hadoop.io.nativeio.NativeIO.getShareDeleteFileInputStream(NativeIO.java:757)
          	at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.getMetaDataInputStream(FsDatasetImpl.java:229)
          	at org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:290)
          	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:617)
          	at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:116)
          	at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:71)
          	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:265)
          	at java.lang.Thread.run(Thread.java:745)
          2017-04-25 07:02:44,610 | INFO  | Async disk worker #87 for volume /srv/BigData/hadoop/data9/dn/current | Deleted BP-262396492-192.168.100.42-1490663057778 blk_1078953155_5605334 file
          

          I feel, we should handle Too Many Open Files here

               } catch (FileNotFoundException e) {
                    // The replica is on its volume map but not on disk
                    datanode.notifyNamenodeDeletedBlock(block, replica.getStorageUuid());
                    datanode.data.invalidate(block.getBlockPoolId(),
                        new Block[]{block.getLocalBlock()});
                    throw e;
          
          Show
          brahmareddy Brahma Reddy Battula added a comment - FYI . Following is the log from DN 2017-04-25 07:02:44,610 | ERROR | DataXceiver for client DFSClient_NONMAPREDUCE_222700060_28 at /192.168.100.48:18124 [Sending block BP-262396492-192.168.100.42-1490663057778:blk_1078953155_5605334] | datanode5:25009:DataXceiver error processing READ_BLOCK operation src: /192.168.100.48:18124 dst: /192.168.100.48:25009 | DataXceiver.java:304 java.io.FileNotFoundException: /srv/BigData/hadoop/data9/dn/current/BP-262396492-192.168.100.42-1490663057778/current/finalized/subdir79/subdir132/blk_1078953155_5605334.meta (Too many open files) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.<init>(FileInputStream.java:138) at org.apache.hadoop.io.nativeio.NativeIO.getShareDeleteFileInputStream(NativeIO.java:757) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.getMetaDataInputStream(FsDatasetImpl.java:229) at org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:290) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:617) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:116) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:71) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:265) at java.lang.Thread.run(Thread.java:745) 2017-04-25 07:02:44,610 | INFO | Async disk worker #87 for volume /srv/BigData/hadoop/data9/dn/current | Deleted BP-262396492-192.168.100.42-1490663057778 blk_1078953155_5605334 file I feel, we should handle Too Many Open Files here } catch (FileNotFoundException e) { // The replica is on its volume map but not on disk datanode.notifyNamenodeDeletedBlock(block, replica.getStorageUuid()); datanode.data.invalidate(block.getBlockPoolId(), new Block[]{block.getLocalBlock()}); throw e;

            People

            • Assignee:
              brahmareddy Brahma Reddy Battula
              Reporter:
              brahmareddy Brahma Reddy Battula
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development