[HDFS-16931] Observer nn delete blocks asynchronously when tail OP_DELETE editlog - ASF JIRA

XML

Word

Printable

JSON

Release Note:

Hide
In our HDFS cluster that holds hundreds of millions of metadata, all of which is EC data. When Observer NN replays the OP_DELETE operation which deletes a large directory, involving tens of millions of metadata. Because the operation is synchronous, the Observer NN is stuck for nearly 50 minutes. ~~HDFS-16043~~ has solved the asynchronous deletion on the active NN. This is to solve the asynchronous deletion on the Observer NN.

Show
In our HDFS cluster that holds hundreds of millions of metadata, all of which is EC data. When Observer NN replays the OP_DELETE operation which deletes a large directory, involving tens of millions of metadata. Because the operation is synchronous, the Observer NN is stuck for nearly 50 minutes. HDFS-16043 has solved the asynchronous deletion on the active NN. This is to solve the asynchronous deletion on the Observer NN.

links to

GitHub Pull Request #5424