Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10347

Namenode report bad block method doesn't log the bad block or datanode.

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Minor
        • Resolution: Fixed
        • Affects Version/s: 2.7.0
        • Fix Version/s: 2.8.0, 2.7.3, 3.0.0-alpha1
        • Component/s: namenode
        • Labels:
          None
        • Target Version/s:
        • Hadoop Flags:
          Reviewed

          Description

          Currently the method FSNamesystem#reportBadBlocks doesn't log any information regarding the bad block id or the datanode on which corrupt block is detected.
          It would be helpful to log that information to debug.

          1. Text File
            HDFS-10347.patch
            1 kB
            Rushabh S Shah

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 13s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
            +1 mvninstall 6m 59s trunk passed
            +1 compile 0m 53s trunk passed with JDK v1.8.0_91
            +1 compile 0m 42s trunk passed with JDK v1.7.0_95
            +1 checkstyle 0m 23s trunk passed
            +1 mvnsite 0m 51s trunk passed
            +1 mvneclipse 0m 13s trunk passed
            +1 findbugs 1m 59s trunk passed
            +1 javadoc 1m 5s trunk passed with JDK v1.8.0_91
            +1 javadoc 1m 49s trunk passed with JDK v1.7.0_95
            +1 mvninstall 0m 48s the patch passed
            +1 compile 0m 48s the patch passed with JDK v1.8.0_91
            +1 javac 0m 48s the patch passed
            +1 compile 0m 41s the patch passed with JDK v1.7.0_95
            +1 javac 0m 41s the patch passed
            +1 checkstyle 0m 19s the patch passed
            +1 mvnsite 0m 51s the patch passed
            +1 mvneclipse 0m 10s the patch passed
            +1 whitespace 0m 0s Patch has no whitespace issues.
            +1 findbugs 2m 13s the patch passed
            +1 javadoc 1m 9s the patch passed with JDK v1.8.0_91
            +1 javadoc 1m 40s the patch passed with JDK v1.7.0_95
            -1 unit 58m 29s hadoop-hdfs in the patch failed with JDK v1.8.0_91.
            -1 unit 55m 26s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
            +1 asflicense 0m 22s Patch does not generate ASF License warnings.
            140m 12s



            Reason Tests
            JDK v1.8.0_91 Failed junit tests hadoop.hdfs.TestHFlush
            JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush
              hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:cf2ee45
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801489/HDFS-10347.patch
            JIRA Issue HDFS-10347
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux a4578694edda 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / 9d4d302
            Default Java 1.7.0_95
            Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
            findbugs v3.0.0
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/15325/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/15325/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
            unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15325/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15325/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
            JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15325/testReport/
            modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15325/console
            Powered by Apache Yetus 0.2.0 http://yetus.apache.org

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 59s trunk passed +1 compile 0m 53s trunk passed with JDK v1.8.0_91 +1 compile 0m 42s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 59s trunk passed +1 javadoc 1m 5s trunk passed with JDK v1.8.0_91 +1 javadoc 1m 49s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 48s the patch passed +1 compile 0m 48s the patch passed with JDK v1.8.0_91 +1 javac 0m 48s the patch passed +1 compile 0m 41s the patch passed with JDK v1.7.0_95 +1 javac 0m 41s the patch passed +1 checkstyle 0m 19s the patch passed +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 13s the patch passed +1 javadoc 1m 9s the patch passed with JDK v1.8.0_91 +1 javadoc 1m 40s the patch passed with JDK v1.7.0_95 -1 unit 58m 29s hadoop-hdfs in the patch failed with JDK v1.8.0_91. -1 unit 55m 26s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 140m 12s Reason Tests JDK v1.8.0_91 Failed junit tests hadoop.hdfs.TestHFlush JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801489/HDFS-10347.patch JIRA Issue HDFS-10347 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a4578694edda 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9d4d302 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15325/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15325/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15325/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_91.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15325/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15325/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15325/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
            Hide
            Permalink
            kihwal Kihwal Lee added a comment - - edited

            +1 straightforward logging change.
            I ran TestProcessCorruptBlocks and saw this in the log.

            *DIR* reportBadBlocks for block: BP-1292934650-x.x.x.x-1461963215660:blk_1073741825_1001
 on datanode: 127.0.0.1:41290
            Show
            kihwal Kihwal Lee added a comment - - edited +1 straightforward logging change. I ran TestProcessCorruptBlocks and saw this in the log. *DIR* reportBadBlocks for block: BP-1292934650-x.x.x.x-1461963215660:blk_1073741825_1001 on datanode: 127.0.0.1:41290
            Hide
            Permalink
            liuml07 Mingliang Liu added a comment -

            +1 (non-binding) as the patch is simple and clear.

            My minor concern is we're dumping state change log while holding the lock, which is not ideal.

            Show
            liuml07 Mingliang Liu added a comment - +1 (non-binding) as the patch is simple and clear. My minor concern is we're dumping state change log while holding the lock, which is not ideal.
            Hide
            Permalink
            kihwal Kihwal Lee added a comment -

            Committed this to trunk through branch-2.7. Thanks for improving this, Rushabh.

            Show
            kihwal Kihwal Lee added a comment - Committed this to trunk through branch-2.7. Thanks for improving this, Rushabh.
            Hide
            Permalink
            hudson Hudson added a comment -

            SUCCESS: Integrated in Hadoop-trunk-Commit #9697 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9697/)
            HDFS-10347. Namenode report bad block method doesn't log the bad block (kihwal: rev 7da540d03eccc2f97950bf47e8b35ce8c889d1e0)

            • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
            Show
            hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9697 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9697/ ) HDFS-10347 . Namenode report bad block method doesn't log the bad block (kihwal: rev 7da540d03eccc2f97950bf47e8b35ce8c889d1e0) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
            Hide
            Permalink
            shahrs87 Rushabh S Shah added a comment -

            There are 2 approaches:
            1. Go through the array of LocatedBlock outside the lock and print every block and then again go through that array to mark them corrupt.
            2. Go through array only once and take a negligible hit for logging within write lock.
            We are not looking up any map or data structure so I think the impact is almost negligible.
            Any thoughts ?

            Show
            shahrs87 Rushabh S Shah added a comment - There are 2 approaches: 1. Go through the array of LocatedBlock outside the lock and print every block and then again go through that array to mark them corrupt. 2. Go through array only once and take a negligible hit for logging within write lock. We are not looking up any map or data structure so I think the impact is almost negligible. Any thoughts ?
            Hide
            Permalink
            shahrs87 Rushabh S Shah added a comment -

            Thanks Kihwal Lee for reviewing and committing.
            Thanks Mingliang Liu for additional reviews.

            Show
            shahrs87 Rushabh S Shah added a comment - Thanks Kihwal Lee for reviewing and committing. Thanks Mingliang Liu for additional reviews.
            Hide
            Permalink
            liuml07 Mingliang Liu added a comment -

            If we are locking/unlocking for each reported block/node instead of the whole RPC request, it will be much easier. I agree that the overhead is not big deal here. Thanks for the patch.

            Show
            liuml07 Mingliang Liu added a comment - If we are locking/unlocking for each reported block/node instead of the whole RPC request, it will be much easier. I agree that the overhead is not big deal here. Thanks for the patch.
            Hide
            Permalink
            vinodkv Vinod Kumar Vavilapalli added a comment -

            Closing the JIRA as part of 2.7.3 release.

            Show
            vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

              People

              • Assignee:
                shahrs87 Rushabh S Shah
                Reporter:
                shahrs87 Rushabh S Shah
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development