Major Description
Currently (as of 3.2.2, but reading through the release notes this doesn't seem to have changed since then) DataNodes use the same properties for deciding which port to bind each service to, as for deciding which ports are included in the `DatanodeRegistration` sent to the NameNode. Further, NameNodes overwrite the DataNode's IP address with the incoming address during registration.
Both of these prevent external users from connecting to DataNodes that are hosted behind some sort of NAT (such as Kubernetes).
I have created a spike branch (https://github.com/stackabletech/hadoop/tree/spike/override-datanode-id, based on v3.2.2) that I have confirmed solves this problem for us. There's clearly some work to be done integrating this properly (such as using the regular Hadoop config system and falling back to the old behaviour if no override is configured). I'd be happy to take that on to the best of my ability (with the caveats that I'm not super familiar with the Hadoop codebase, and that my Java is quite rusty at this point) if the overall direction seems acceptable.