Would be great if distcp supported cross-cluster rack awareness.
For example, we have hdfs cluster1 and hdfs cluster2.
Both clusters span three switches, and both have rack awareness enabled.
And also both clusters name same switches same way.
So when distcp runs data replication job, it could replicate hdfs blocks
only to counterpart datanodes on destination cluster that are in the same physical network
switch, minimizing latencies and maximizing bandwidth.
It could be an option, activate through `distcp` clommand-line switch.
We have multiple clusters with default replication of 3 and all those cluster live in same three different "racks" / "top of the rack switches".
This could drastically minimize inter-switch network traffic during huge distcp jobs.