[HDFS-8288] Refactor DFSStripedOutputStream and StripedDataStreamer

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: hdfs-client
• Target Version/s: HDFS-7285
• Assignee: Tsz-wo Sze
• Reporter: Tsz-wo Sze
• Created: 29/Apr/15 19:07
• Updated: 05/May/15 23:29
• Resolved: 05/May/15 23:29

Description

• DFSStripedOutputStream has a list of StripedDataStreamer(s). The streamers share a data structure List<BlockingQueue<LocatedBlock>> stripeBlocks for communicate located block and end block information.
  For example,

  //StripedDataStreamer.endBlock()
        // before retrieving a new block, transfer the finished block to
        // leading streamer
        LocatedBlock finishedBlock = new LocatedBlock(
            new ExtendedBlock(block.getBlockPoolId(), block.getBlockId(),
                block.getNumBytes(), block.getGenerationStamp()), null);
        try {
          boolean offSuccess = stripedBlocks.get(0).offer(finishedBlock, 30,
              TimeUnit.SECONDS);
  

  It is unnecessary to create a LocatedBlock object for an end block since the locations passed is null. Also, the return value is ignored (i.e. offSuccess is not used).

• DFSStripedOutputStream has another data structure cellBuffers for computing parity. It should be refactored to a class.

Issue Links

• is part of: HDFS-7672 Erasure Coding: consolidate streamer coordination logic and handle failure when writing striped blocks