Details
Minor Description
In a production environment, there may be some differences in each JouranlNode (e.g. network condition, disk condition, and so on). For example, If a JN's network is much worse than other JNs, then the time taken by the NN to write this JN will be much greater than other JNs, in this case, it will cause the IPC Logger thread corresponding to this JN to have many pending edits, when the pending edits exceeds the maximum limit (default 10MB), the new edits about to write to this JN will be silently dropped, and will result gaps in the editlog segment, which causing this JN and NN repeatedly reporting the following errors:
org.apache.hadoop.hdfs.qjournal.protocol.JournalOutOfSyncException: Can't write txid 1904164873 expecting nextTxId=1904164871
Unfortunately, the above error message can not help us quickly find the root cause, It took more time to find the cause, so it's better to add a warning log here, like this:
2019-08-02 04:55:05,879 WARN org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager:Pending edits to 192.168.202.13:8485 is going to exceed limit size:10240, current queued edits size:10224, will silently drop 174 bytes of edits!
This is just a very small improvement.