Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1606 Provide a stronger data guarantee in the write pipeline
  3. HDFS-1761

Add a new DataTransferProtocol operation, Op.TRANSFER_BLOCK, instead of using RPC

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Add a new DataTransferProtocol operation, Op.TRANSFER_BLOCK, for transferring RBW/Finalized with acknowledgement and without using RPC.

      Description

      TestTransferRbw failed in build #264

      org.apache.hadoop.hdfs.server.datanode.ReplicaNotFoundException: 65536 = numBytes < visible = 90564, r=ReplicaInPipeline, blk_8440021909252053811_1001, TEMPORARY
        getNumBytes()     = 65536
        getBytesOnDisk()  = 0
        getVisibleLength()= -1
        getVolume()       = /grid/0/hudson/hudson-slave/workspace/PreCommit-HDFS-Build/trunk/build/test/data/dfs/data/data3/current/finalized
        getBlockFile()    = /grid/0/hudson/hudson-slave/workspace/PreCommit-HDFS-Build/trunk/build/test/data/dfs/data/data3/tmp/blk_8440021909252053811
        bytesAcked=0
        bytesOnDisk=0
      	at org.apache.hadoop.hdfs.server.datanode.FSDataset.convertTemporaryToRbw(FSDataset.java:1383)
      	at org.apache.hadoop.hdfs.server.datanode.DataNode.convertTemporaryToRbw(DataNode.java:2023)
      	at org.apache.hadoop.hdfs.server.datanode.TestTransferRbw.testTransferRbw(TestTransferRbw.java:121)
      

        Attachments

        1. h1761_20110329.patch
          34 kB
          Tsz Wo Nicholas Sze
        2. h1761_20110325.patch
          42 kB
          Tsz Wo Nicholas Sze
        3. h1761_20110319.patch
          8 kB
          Tsz Wo Nicholas Sze
        4. h1761_20110318.patch
          8 kB
          Tsz Wo Nicholas Sze
        5. h1761_20110317.patch
          3 kB
          Tsz Wo Nicholas Sze
        6. build264.log
          36 kB
          Tsz Wo Nicholas Sze

          Activity

            People

            • Assignee:
              szetszwo Tsz Wo Nicholas Sze
              Reporter:
              szetszwo Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: