[HDFS-16880] modify invokeSingleXXX interface in order to pass actual file src to namenode for debug info.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.3.4
• Component/s: rbf
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: farmmamba
• Created: 30/Dec/22 07:47
• Updated: 06/May/23 13:45
• Resolved: 06/May/23 13:45

Description

We found lots of INFO level log like below:

2022-12-30 15:31:04,169 INFO org.apache.hadoop.hdfs.StateChange: DIR* completeFile: / is closed by DFSClient_attempt_1671783180362_213003_m_000077_0_1102875551_1
2022-12-30 15:31:04,186 INFO org.apache.hadoop.hdfs.StateChange: DIR* completeFile: / is closed by DFSClient_NONMAPREDUCE_1198313144_27480

It lost the real path of completeFile. Actually this is caused by : 

org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient#invokeSingle(java.lang.String, org.apache.hadoop.hdfs.server.federation.router.RemoteMethod)

In this method, it instantiates a RemoteLocationContext object:

RemoteLocationContext loc = new RemoteLocation(nsId, "/", "/");

and then execute: Object[] params = method.getParams(loc);

The problem is right here, becasuse we always use new RemoteParam(), so, 

context.getDest() always return "/"; That's why we saw lots of incorrect logs.

 

After diving into invokeSingleXXX source code, I found the following RPCs classified as need actual src and not need actual src.

 

need src path RPC:

addBlock、abandonBlock、getAdditionalDatanode、complete

not need src path RPC:

updateBlockForPipeline、reportBadBlocks、getBlocks、updatePipeline、invokeAtAvailableNs（invoked by: getServerDefaults、getBlockKeys、getTransactionID、getMostRecentCheckpointTxId、versionRequest、getStoragePolicies）

 

After changes, the src can be pass to NN correctly.

 

 

Issue Links

• is duplicated by: HDFS-16865 RBF: The source path is always / after RBF proxied the complete, addBlock and getAdditionalDatanode RPC.
• links to: GitHub Pull Request #5262