Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10569

A bug causes OutOfIndex error in BlockListAsLongs

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      An obvious bug in LongsDecoder.getBlockListAsLongs(), the size of var longs is +2 to the size of values, but the for-loop accesses values using longs index. This will cause OutOfIndex.

      1. HDFS-10569.001.patch
        0.8 kB
        Weiwei Yang
      2. HDFS-10569.002.patch
        3 kB
        Weiwei Yang
      3. HDFS-10569.003.patch
        3 kB
        Weiwei Yang
      4. HDFS-10569.004.patch
        3 kB
        Weiwei Yang

        Issue Links

          Activity

          Hide
          kihwal Kihwal Lee added a comment -

          It looks like index2BlockId() was used to get the right index before the change.
          Daryn Sharp, please have a look.

          Show
          kihwal Kihwal Lee added a comment - It looks like index2BlockId() was used to get the right index before the change. Daryn Sharp , please have a look.
          Hide
          daryn Daryn Sharp added a comment -

          It's a dead/uncalled code path. The NN uses the iterator interface to decode/process the obsolete encoding, not this method. This is why the unit tests don't catch it.

          Otherwise, the block report would be corrupted since it's 2 longs for size followed by a tuple of 3 longs per block. Shift that down and the block report is completely mangled. A NN would never come out of safemode.

          So we could throw UnsupportedOperationException, or if it's fixed, a unit test should be added.

          Show
          daryn Daryn Sharp added a comment - It's a dead/uncalled code path. The NN uses the iterator interface to decode/process the obsolete encoding, not this method. This is why the unit tests don't catch it. Otherwise, the block report would be corrupted since it's 2 longs for size followed by a tuple of 3 longs per block. Shift that down and the block report is completely mangled. A NN would never come out of safemode. So we could throw UnsupportedOperationException, or if it's fixed, a unit test should be added.
          Hide
          cheersyang Weiwei Yang added a comment -

          Well this did happen to us, when we tried to back port HDFS-10312 to 2.7.1. To avoid others running into same issue, I still want this to be fixed. I will add an UT for this. Does that make sense?

          Show
          cheersyang Weiwei Yang added a comment - Well this did happen to us, when we tried to back port HDFS-10312 to 2.7.1. To avoid others running into same issue, I still want this to be fixed. I will add an UT for this. Does that make sense?
          Hide
          cheersyang Weiwei Yang added a comment -

          It looks like the 2nd test in TestBlockListAsLongs#testDatanodeDetect was still using new-style BP, that's why the code was not called in UT. I modified that part in 002 patch. Without the patch, it will throw OutOfIndex error, the patch fixes this issue when DN sends old-style BR.

          Please take a look at the new patch and the test case. Thanks a lot.

          Show
          cheersyang Weiwei Yang added a comment - It looks like the 2nd test in TestBlockListAsLongs#testDatanodeDetect was still using new-style BP, that's why the code was not called in UT. I modified that part in 002 patch. Without the patch, it will throw OutOfIndex error, the patch fixes this issue when DN sends old-style BR. Please take a look at the new patch and the test case. Thanks a lot.
          Hide
          cheersyang Weiwei Yang added a comment - - edited

          Hello Daryn Sharp, Kihwal Lee

          Please help to check if my latest comment and the modifications to the UT makes sense.

          Show
          cheersyang Weiwei Yang added a comment - - edited Hello Daryn Sharp , Kihwal Lee Please help to check if my latest comment and the modifications to the UT makes sense.
          Hide
          cheersyang Weiwei Yang added a comment -

          Not sure why Jenkins job was not triggered, upload a new (but same as v2) patch to trigger the job running.

          Show
          cheersyang Weiwei Yang added a comment - Not sure why Jenkins job was not triggered, upload a new (but same as v2) patch to trigger the job running.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 34s trunk passed
          +1 compile 0m 45s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 57s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 48s trunk passed
          +1 javadoc 0m 58s trunk passed
          +1 mvninstall 0m 53s the patch passed
          +1 compile 0m 44s the patch passed
          +1 javac 0m 44s the patch passed
          -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 58 unchanged - 0 fixed = 59 total (was 58)
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 58s the patch passed
          +1 javadoc 1m 0s the patch passed
          -1 unit 60m 43s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          81m 24s



          Reason Tests
          Failed junit tests hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:85209cc
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12815275/HDFS-10569.003.patch
          JIRA Issue HDFS-10569
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6fe1b87eb01d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 846ada2
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15951/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15951/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15951/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15951/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 34s trunk passed +1 compile 0m 45s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 48s trunk passed +1 javadoc 0m 58s trunk passed +1 mvninstall 0m 53s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 58 unchanged - 0 fixed = 59 total (was 58) +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 58s the patch passed +1 javadoc 1m 0s the patch passed -1 unit 60m 43s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 81m 24s Reason Tests Failed junit tests hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer Subsystem Report/Notes Docker Image:yetus/hadoop:85209cc JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12815275/HDFS-10569.003.patch JIRA Issue HDFS-10569 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6fe1b87eb01d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 846ada2 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15951/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15951/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15951/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15951/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cheersyang Weiwei Yang added a comment -

          Uploaded v4 patch to fix the check style issue.

          Show
          cheersyang Weiwei Yang added a comment - Uploaded v4 patch to fix the check style issue.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 45s trunk passed
          +1 compile 1m 0s trunk passed
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 47s trunk passed
          +1 javadoc 0m 56s trunk passed
          +1 mvninstall 0m 49s the patch passed
          +1 compile 0m 45s the patch passed
          +1 javac 0m 45s the patch passed
          +1 checkstyle 0m 24s the patch passed
          +1 mvnsite 0m 50s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 51s the patch passed
          +1 javadoc 0m 54s the patch passed
          -1 unit 58m 34s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          80m 16s



          Reason Tests
          Failed junit tests hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:85209cc
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12815484/HDFS-10569.004.patch
          JIRA Issue HDFS-10569
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2225b47b56cf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 846ada2
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15956/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15956/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15956/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 45s trunk passed +1 compile 1m 0s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 49s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 51s the patch passed +1 javadoc 0m 54s the patch passed -1 unit 58m 34s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 80m 16s Reason Tests Failed junit tests hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer Subsystem Report/Notes Docker Image:yetus/hadoop:85209cc JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12815484/HDFS-10569.004.patch JIRA Issue HDFS-10569 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2225b47b56cf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 846ada2 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15956/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15956/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15956/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          cheersyang Weiwei Yang added a comment -

          Hello Daryn Sharp, Kihwal Lee, I got the patch ready, let me know if it looks good to you. Thanks.

          Show
          cheersyang Weiwei Yang added a comment - Hello Daryn Sharp , Kihwal Lee , I got the patch ready, let me know if it looks good to you. Thanks.
          Hide
          kihwal Kihwal Lee added a comment -

          The failed test is passing for me with the patch. HDFS-10572 probably fixed it.
          +1 the patch looks good.

          Show
          kihwal Kihwal Lee added a comment - The failed test is passing for me with the patch. HDFS-10572 probably fixed it. +1 the patch looks good.
          Hide
          kihwal Kihwal Lee added a comment -

          Thanks for reporting and fixing it, Weiwei Yang.

          Show
          kihwal Kihwal Lee added a comment - Thanks for reporting and fixing it, Weiwei Yang .
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10208 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10208/)
          HDFS-10569. A bug causes OutOfIndex error in BlockListAsLongs. (kihwal: rev 6f63566694f8cec64a469448a8fa00ce921ce367)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/BlockListAsLongs.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestBlockListAsLongs.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10208 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10208/ ) HDFS-10569 . A bug causes OutOfIndex error in BlockListAsLongs. (kihwal: rev 6f63566694f8cec64a469448a8fa00ce921ce367) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/BlockListAsLongs.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestBlockListAsLongs.java
          Hide
          cheersyang Weiwei Yang added a comment -

          My pleasure thanks Kihwal Lee

          Show
          cheersyang Weiwei Yang added a comment - My pleasure thanks Kihwal Lee

            People

            • Assignee:
              cheersyang Weiwei Yang
              Reporter:
              cheersyang Weiwei Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development