Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.9.2
-
None
-
None
Description
Some RBF-related information has been reported in HDFS-15078 and HDFS-15079, which may be similar to the situation here.
When the client connects to the NameNode, such as creating a file, sometimes RPC will prompt some exception information:
2021-12-15 14:57:14,502 [2246654536] - WARN [IPC Server handler 58 on 8025:Server$Responder@1518] - IPC Server handler 58 on 8025, call Call#870382577 Retry#0 org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 10.196.73.49:33700: output error
2021-12-15 14:57:14,502 [2246654536] - INFO [IPC Server handler 58 on 8025:Server$Handler@2619] - IPC Server handler 58 on 8025 caught an exception
java.nio.channels.ClosedChannelException
at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:270)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:461)
at org.apache.hadoop.ipc.Server.channelWrite(Server.java:3173)
at org.apache.hadoop.ipc.Server.access$1700(Server.java:136)
at org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:1468)
at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:1538)
at org.apache.hadoop.ipc.Server$Connection.sendResponse(Server.java:2517)
at org.apache.hadoop.ipc.Server$Connection.access$300(Server.java:1610)
at org.apache.hadoop.ipc.Server$RpcCall.doResponse(Server.java:935)
at org.apache.hadoop.ipc.Server$Call.sendResponse(Server.java:769)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:880)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:817)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2606)
This information exists in the log recorded by the NameNode, which is somewhat different from HDFS-15078&HDFS-15079, but they have a common feature, that is, they all appear when accessing RPC.
I think this problem can be solved in a unified way.
Attachments
Issue Links
- is related to
-
HDFS-15079 RBF: Client maybe get an unexpected result with network anomaly
-
- Resolved
-
-
HDFS-15078 RBF: Should check connection channel before sending RPC to namenode
-
- Patch Available
-