[HDFS-7265] Use a throttler for replica write in datanode

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Component/s: datanode
• Assignee: Tsz-wo Sze
• Reporter: Tsz-wo Sze
• Created: 19/Oct/14 01:39
• Updated: 15/Feb/16 19:37
• Resolved: 08/May/15 20:32

Description

BlockReceiver process packets in BlockReceiver.receivePacket() as follows

1. read from socket
2. enqueue the ack
3. write to downstream
4. write to disk

The above steps is repeated for each packet in a single thread. When there are a lot of concurrent writes in a datanode, the write time in #4 becomes very long. As a result, it leads to SocketTimeoutException since it cannot read from the socket for a long time.

Issue Links

• is superceded by: HDFS-7270 Add congestion signaling capability to DataNode write protocol
• relates to: HDFS-9796 Add throttler for datanode bandwidth