[HDFS-16414] Improve asynchronous edit: reduce the time that edit waits to enter editPendingQ

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Not A Bug
• Affects Version/s: 2.9.2
• Component/s: fs async, namenode
• Assignee: JiangHua Zhu
• Reporter: JiangHua Zhu
• Created: 06/Jan/22 07:54
• Updated: 06/Jan/22 08:25
• Resolved: 06/Jan/22 08:25

Description

When dfs.namenode.edits.asynclogging=true, FSEditLogAsync starts to work, which is very helpful for improving the speed of NameNode processing Call.
But there is a strange phenomenon here. When a lot of calls enter the NameNode through RPC within a period of time, the handler will be forced to wait, in seconds:
FSEditLogAsync#enqueueEdit():
if (Thread.holdsLock(this)) {
int permits = overflowMutex.drainPermits();
try {
do

{ this.wait(1000); // will be notified by next logSync. }

while (!editPendingQ.offer(edit));
} finally

{ overflowMutex.release(permits); }

}

We should reduce this.wait(1000) here so that edit can be processed quickly.