[HDFS-16910] Fix incorrectly initializing RandomAccessFile caused flush performance decreased for JN

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: namenode
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Haiyang Hu
• Reporter: Haiyang Hu
• Created: 07/Feb/23 02:35
• Updated: 16/Oct/24 18:29
• Resolved: 09/Feb/23 02:48

Description

At present, after our cluster backport patch HDFS-15882,
when set shouldSyncWritesAndSkipFsync to false, there will be flush performance degradation caused by JN.

Root Cause：
when setting shouldSyncWritesAndSkipFsync to false, the mode of init RandomAccessFile will be `rws`.
even if fc.force(false) is executed when flushAndSync is executed (hopefully, only requires updates to the file's content to be written to storage and the metadata is not update),
but since the mode of RandomAccessFile is `rws`, It will requires updates to both the file's content and its metadata to be written,
there will be flush performance degradation caused by JN.

Fix:
Need to update RandomAccessFile's mode from `rws` to `rwd`:

rwd: Open for reading and writing, as with "rw", and also require that every update to the file's content be written synchronously to the underlying storage device.

if (shouldSyncWritesAndSkipFsync) {
rp = new RandomAccessFile(name, "rwd");
} else {
rp = new RandomAccessFile(name, "rw");
}

In this way, when flushAndSync is executed,
if shouldSyncWritesAndSkipFsync is false and the mode of RandomAccessFile is 'rw', it will call fc.force(false) to execute,
otherwise should use `rwd` to perform the operation.

Issue Links

• is caused by: HDFS-15882 Fix incorrectly initializing RandomAccessFile based on configuration options
• links to: GitHub Pull Request #5359