Hadoop Common
  1. Hadoop Common
  2. HADOOP-1443

TestFileCorruption fails with ArrayIndexOutOfBoundsException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      org.apache.hadoop.dfs.TestFileCorruption.testFileCorruption failed once on Windows with this exception:

      org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 1
      at org.apache.hadoop.dfs.FSNamesystem.getBlockLocations(FSNamesystem.java:472)
      at org.apache.hadoop.dfs.FSNamesystem.getBlockLocations(FSNamesystem.java:436)
      at org.apache.hadoop.dfs.NameNode.getBlockLocations(NameNode.java:272)
      at org.apache.hadoop.dfs.NameNode.open(NameNode.java:259)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:567)

      at org.apache.hadoop.ipc.Client.call(Client.java:471)
      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:165)
      at org.apache.hadoop.dfs.$Proxy0.open(Unknown Source)
      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
      at org.apache.hadoop.dfs.$Proxy0.open(Unknown Source)
      at org.apache.hadoop.dfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:590)
      at org.apache.hadoop.dfs.DFSClient$DFSInputStream.<init>(DFSClient.java:582)
      at org.apache.hadoop.dfs.DFSClient.open(DFSClient.java:273)
      at org.apache.hadoop.dfs.DistributedFileSystem$RawDistributedFileSystem.open(DistributedFileSystem.java:136)
      at org.apache.hadoop.fs.ChecksumFileSystem$FSInputChecker.<init>(ChecksumFileSystem.java:114)
      at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:340)
      at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:234)
      at org.apache.hadoop.dfs.DFSTestUtil.checkFiles(DFSTestUtil.java:132)
      at org.apache.hadoop.dfs.TestFileCorruption.testFileCorruption(TestFileCorruption.java:66)

      1. EmptyFile.patch
        5 kB
        Konstantin Shvachko
      2. 1443.patch
        0.5 kB
        dhruba borthakur

        Activity

        Hide
        dhruba borthakur added a comment -

        I am marking this as a blocker for 0.13 release. Investigation under progress.

        Show
        dhruba borthakur added a comment - I am marking this as a blocker for 0.13 release. Investigation under progress.
        Hide
        dhruba borthakur added a comment -

        This looks like a bug that occurs when a zero-length file is read. Possible fix is attached. A test case for zero-length files will follow shortly (either as part of this issue or a separate issue).

        Show
        dhruba borthakur added a comment - This looks like a bug that occurs when a zero-length file is read. Possible fix is attached. A test case for zero-length files will follow shortly (either as part of this issue or a separate issue).
        Hide
        Doug Cutting added a comment -

        It'd be nice to have the test case included in this issue, so we can verify both the bug and that it's fixed. But perhaps it's not trivial to create such a test?

        Show
        Doug Cutting added a comment - It'd be nice to have the test case included in this issue, so we can verify both the bug and that it's fixed. But perhaps it's not trivial to create such a test?
        Hide
        Konstantin Shvachko added a comment -

        This does not work if you getBlockLocation for the offset at the start of the second block.
        I'm working on it. I'll create test cases.

        Show
        Konstantin Shvachko added a comment - This does not work if you getBlockLocation for the offset at the start of the second block. I'm working on it. I'll create test cases.
        Hide
        Konstantin Shvachko added a comment -

        The bug is related to a corner case. An empty file in hdfs is represented by one block
        of size 0, which is replicated on the required number of nodes, and the replicas are
        represented respectively by empty files. This case was not handled correctly.
        I fixed the bug and created test cases that verify correctness of 2 things:

        • empty file open and read;
        • reading beyond the file end.
        Show
        Konstantin Shvachko added a comment - The bug is related to a corner case. An empty file in hdfs is represented by one block of size 0, which is replicated on the required number of nodes, and the replicas are represented respectively by empty files. This case was not handled correctly. I fixed the bug and created test cases that verify correctness of 2 things: empty file open and read; reading beyond the file end.
        Show
        Hadoop QA added a comment - +1 http://issues.apache.org/jira/secure/attachment/12358751/EmptyFile.patch applied and successfully tested against trunk revision r543622. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/236/testReport/ Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/236/console
        Hide
        dhruba borthakur added a comment -

        +1 code looks good.

        Two minor comments:

        1. There is a comment in the test saying "// create and write a file that contains three blocks of data". This might not be correct.

        2. This patch checks for negative offsets and lengths. Is there a way to enhance the test so that it triggers negative lengths and negative offsets and verify that they generate the expected exceptions?

        Show
        dhruba borthakur added a comment - +1 code looks good. Two minor comments: 1. There is a comment in the test saying "// create and write a file that contains three blocks of data". This might not be correct. 2. This patch checks for negative offsets and lengths. Is there a way to enhance the test so that it triggers negative lengths and negative offsets and verify that they generate the expected exceptions?
        Hide
        Doug Cutting added a comment -

        I just committed this. Thanks, Konstantin!

        Show
        Doug Cutting added a comment - I just committed this. Thanks, Konstantin!
        Hide
        Doug Cutting added a comment -

        This patch has conflicts with the 0.13 branch. We need a new version of the patch for that branch that does not conflict.

        Show
        Doug Cutting added a comment - This patch has conflicts with the 0.13 branch. We need a new version of the patch for that branch that does not conflict.
        Hide
        Doug Cutting added a comment -

        My current theory is that this was caused by HADOOP-894, which is not in 0.13. If that's the case, then this should not be merged into the 0.13 branch, and the "Fix Version" should be 0.14, not 0.13. Does that sound right to others, or is this really a problem in 0.13?

        Show
        Doug Cutting added a comment - My current theory is that this was caused by HADOOP-894 , which is not in 0.13. If that's the case, then this should not be merged into the 0.13 branch, and the "Fix Version" should be 0.14, not 0.13. Does that sound right to others, or is this really a problem in 0.13?
        Hide
        dhruba borthakur added a comment -

        I agree. This bug does not exist in 0.13.

        Show
        dhruba borthakur added a comment - I agree. This bug does not exist in 0.13.
        Hide
        Doug Cutting added a comment -

        Moving to 0.14 and downgrading.

        Show
        Doug Cutting added a comment - Moving to 0.14 and downgrading.
        Hide
        Konstantin Shvachko added a comment -

        Yes 0.14 is right for this patch if 894 is not in 0.13.

        Show
        Konstantin Shvachko added a comment - Yes 0.14 is right for this patch if 894 is not in 0.13.

          People

          • Assignee:
            Konstantin Shvachko
            Reporter:
            Nigel Daley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development