[HDFS-15915] Race condition with async edits logging due to updating txId outside of the namesystem log

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0, 2.10.2, 3.2.3, 3.3.2
• Component/s: hdfs, namenode
• Target Version/s: 2.10.2
• Hadoop Flags: Reviewed
• Assignee: Konstantin Shvachko
• Reporter: Konstantin Shvachko
• Created: 23/Mar/21 22:09
• Updated: 30/Oct/24 08:41
• Resolved: 26/May/21 23:07

Description

FSEditLogAsync creates an FSEditLogOp and populates its fields inside FSNamesystem.writeLock. But one essential field the transaction id of the edits op remains unset until the time when the operation is scheduled for synching. At that time beginTransaction() will set the the FSEditLogOp.txid and increment the global transaction count. On busy NameNode this event can fall outside the write lock.
This causes problems for Observer reads. It also can potentially reshuffle transactions and Standby will apply them in a wrong order.

Issue Links

• breaks: HADOOP-19164 Hadoop CLI MiniCluster is broken
• breaks: HDFS-16050 Some dynamometer tests fail
• breaks: YARN-10808 TestHBaseTimelineStorageUtils fails by NoClassDefFoundError
• is related to: HBASE-28944 TestShadedHBaseTestingUtility fails with NCDFE: org/mockito/stubbing/Answer