[HDFS-16898] Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.4
• Fix Version/s: 3.4.0, 3.3.7
• Component/s: datanode
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.7
• Hadoop Flags: Reviewed
• Assignee: farmmamba
• Reporter: farmmamba
• Created: 29/Jan/23 13:11
• Updated: 28/Jan/24 07:23
• Resolved: 08/Feb/23 02:36

Description

Now in method processCommandFromActor,  we have code like below:

 

writeLock();
try {
  if (actor == bpServiceToActive) {
    return processCommandFromActive(cmd, actor);
  } else {
    return processCommandFromStandby(cmd, actor);
  }
} finally {
  writeUnlock();
} 

if method processCommandFromActive costs much time, the write lock would not release.

 

It maybe block the updateActorStatesFromHeartbeat method in offerService，furthermore, it can cause the lastcontact of datanode very high, even dead when lastcontact beyond 600s.

bpos.updateActorStatesFromHeartbeat(
    this, resp.getNameNodeHaState());

here we can make write lock fine-grain in processCommandFromActor method to address this problem

 

Issue Links

• links to: GitHub Pull Request #5330
• links to: GitHub Pull Request #5408