[HADOOP-18365] Updated addresses are still accessed using the old IP address

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.5
• Fix Version/s: 3.4.0, 3.3.5
• Component/s: common
• Labels: pull-request-available

Environment

Demonstrated in a Kubernetes environment running Java 11.

• Target Version/s: 3.4.0, 3.3.5
• Hadoop Flags: Reviewed
• Assignee: Steve Vaughan
• Reporter: Steve Vaughan
• Created: 25/Jul/22 13:42
• Updated: 28/Jan/24 00:20
• Resolved: 22/Aug/22 16:57

Description

When the IPC Client recognizes that an IP address has changed, it updates the server field and logs a message:
Address change detected. Old: journalnode-1.journalnode.hdfs.svc.cluster.local/10.1.0.178:8485 New: journalnode-1.journalnode.hdfs.svc.cluster.local/10.1.0.182:8485
Although the change is detected, the client will continue to connect to the old IP address, resulting in repeated log messages.  This is seen in managed environments when JournalNode syncing is enabled and a JournalNode is restarted, with the remaining nodes in the set repeatedly logging this message when syncing to the restarted JournalNode.

The source of the problem is that the remoteId.address is not updated.

Issue Links

• causes: HADOOP-18694 Client.Connection#updateAddress needs to ensure that address is resolved before updating
• is a child of: HADOOP-18396 Issues running in dynamic / managed environments
• is duplicated by: HADOOP-18346 Suggest to update the IP address of ConnectionId when server of Connection got updated
• links to: GitHub Pull Request #4692
• links to: GitHub Pull Request #4768