Hadoop Common
  1. Hadoop Common
  2. HADOOP-2658

Design and Implement a Test Plan to support appends to HDFS files

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.0
    • Component/s: test
    • Labels:
      None

      Description

      HADOOP-1700 describes the design of supporting appends to HDFS files. This feature needs extensive testing, especially because the design explicitly analyzes many failure scenarios. A detailed test plan and test cases are needed to make this feature a reality.

      1. TestFileAppend3_TC7.java
        4 kB
        Tsz Wo Nicholas Sze
      2. AppendTestCase.doc
        106 kB
        dhruba borthakur
      3. AppendTestCase.doc
        107 kB
        dhruba borthakur
      4. append_testplan.txt
        4 kB
        Nigel Daley
      5. 20081017testplan.txt
        5 kB
        Tsz Wo Nicholas Sze
      6. 20081008testplan.txt
        4 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          Nigel Daley added a comment -

          Dhruba, weren't you going to write a test plan?

          Show
          Nigel Daley added a comment - Dhruba, weren't you going to write a test plan?
          Hide
          dhruba borthakur added a comment -

          Test Plan in the works, but not ready yet. Will post in the next 4 days.

          Show
          dhruba borthakur added a comment - Test Plan in the works, but not ready yet. Will post in the next 4 days.
          Hide
          stack added a comment -

          +1 on the appendtestcase.doc (You might want to add author, date, and link to this issue Dhruba).

          Show
          stack added a comment - +1 on the appendtestcase.doc (You might want to add author, date, and link to this issue Dhruba).
          Hide
          dhruba borthakur added a comment -

          Incorporated feedback from Stack. Thanks.

          Show
          dhruba borthakur added a comment - Incorporated feedback from Stack. Thanks.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          There are a few append-related tests failed occasionally. Linking the to this issue.

          Show
          Tsz Wo Nicholas Sze added a comment - There are a few append-related tests failed occasionally. Linking the to this issue.
          Hide
          Nigel Daley added a comment -

          Attaching latest test plan in a non-proprietary file format.

          Show
          Nigel Daley added a comment - Attaching latest test plan in a non-proprietary file format.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          20081008testplan.txt

          I went thought each test in the test plan. We already have unit tests for some of the items. Some items need new unit tests. The remaining items need manual tests.

          Show
          Tsz Wo Nicholas Sze added a comment - 20081008testplan.txt I went thought each test in the test plan. We already have unit tests for some of the items. Some items need new unit tests. The remaining items need manual tests.
          Hide
          Chris Douglas added a comment - - edited

          [ edit - sorry, wrong JIRA ]

          Show
          Chris Douglas added a comment - - edited [ edit - sorry, wrong JIRA ]
          Hide
          Chris Douglas added a comment -

          Manual tests for items 4 and 14 passed, but 7 failed with the following exception:

          08/10/13 04:53:20 WARN hdfs.DFSClient: DFSOutputStream ResponseProcessor exception  for block blk_-2897952013639714624_1002java.io.IOException: Connection reset by peer
                  at sun.nio.ch.FileDispatcher.read0(Native Method)
                  at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
                  at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233)
                  at sun.nio.ch.IOUtil.read(IOUtil.java:206)
                  at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)
                  at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
                  at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:140)
                  at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:150)
                  at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:123)
                  at java.io.DataInputStream.readFully(DataInputStream.java:178)
                  at java.io.DataInputStream.readLong(DataInputStream.java:399)
                  at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:2307)
          
          08/10/13 04:53:20 WARN hdfs.DFSClient: Error Recovery for block blk_-2897952013639714624_1002 bad datanode[0] 67.195.209.61:50010
          08/10/13 04:53:20 WARN hdfs.DFSClient: Error Recovery for block blk_-2897952013639714624_1002 in pipeline 67.195.209.61:50010, 67.195.209.109:50010: bad datanode 67.195.209.61:50010
          08/10/13 04:53:20 WARN hdfs.DFSClient: DFSOutputStream ResponseProcessor exception  for block blk_-2897952013639714624_1003java.io.IOException: Connection reset by peer
                  at sun.nio.ch.FileDispatcher.read0(Native Method)
                  at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
                  at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233)
                  at sun.nio.ch.IOUtil.read(IOUtil.java:206)
                  at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)
                  at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
                  at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:140)
                  at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:150)
                  at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:123)
                  at java.io.DataInputStream.readFully(DataInputStream.java:178)
                  at java.io.DataInputStream.readLong(DataInputStream.java:399)
                  at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:2307)
          
          08/10/13 04:53:20 WARN hdfs.DFSClient: Error Recovery for block blk_-2897952013639714624_1003 bad datanode[0] 67.195.209.109:50010
          

          The remaining, good replica was lost.

          Show
          Chris Douglas added a comment - Manual tests for items 4 and 14 passed, but 7 failed with the following exception: 08/10/13 04:53:20 WARN hdfs.DFSClient: DFSOutputStream ResponseProcessor exception for block blk_-2897952013639714624_1002java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233) at sun.nio.ch.IOUtil.read(IOUtil.java:206) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236) at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:140) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:150) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:123) at java.io.DataInputStream.readFully(DataInputStream.java:178) at java.io.DataInputStream.readLong(DataInputStream.java:399) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:2307) 08/10/13 04:53:20 WARN hdfs.DFSClient: Error Recovery for block blk_-2897952013639714624_1002 bad datanode[0] 67.195.209.61:50010 08/10/13 04:53:20 WARN hdfs.DFSClient: Error Recovery for block blk_-2897952013639714624_1002 in pipeline 67.195.209.61:50010, 67.195.209.109:50010: bad datanode 67.195.209.61:50010 08/10/13 04:53:20 WARN hdfs.DFSClient: DFSOutputStream ResponseProcessor exception for block blk_-2897952013639714624_1003java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233) at sun.nio.ch.IOUtil.read(IOUtil.java:206) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236) at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:140) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:150) at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:123) at java.io.DataInputStream.readFully(DataInputStream.java:178) at java.io.DataInputStream.readLong(DataInputStream.java:399) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:2307) 08/10/13 04:53:20 WARN hdfs.DFSClient: Error Recovery for block blk_-2897952013639714624_1003 bad datanode[0] 67.195.209.109:50010 The remaining, good replica was lost.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          TestFileAppend3_TC7.java: implemented TC7. Trunk cannot pass this test.

          Show
          Tsz Wo Nicholas Sze added a comment - TestFileAppend3_TC7.java: implemented TC7. Trunk cannot pass this test.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          TC7 failed is because DataNode.getBlockMetaDataInfo(Block) does not aware the block is corrupted and it returns a BlockMetaDataInfo with the corrupted information.

          Show
          Tsz Wo Nicholas Sze added a comment - TC7 failed is because DataNode.getBlockMetaDataInfo(Block) does not aware the block is corrupted and it returns a BlockMetaDataInfo with the corrupted information.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The problem discovered by TC7 is now fixed by HADOOP-4423.

          See 20081017testplan.txt for the test plan updates.

          Show
          Tsz Wo Nicholas Sze added a comment - The problem discovered by TC7 is now fixed by HADOOP-4423 . See 20081017testplan.txt for the test plan updates.
          Hide
          Robert Chansler added a comment -

          OK by Nigel and I!

          Show
          Robert Chansler added a comment - OK by Nigel and I!

            People

            • Assignee:
              dhruba borthakur
              Reporter:
              dhruba borthakur
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development