Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
2.7.1
-
None
-
None
-
CentOS Linux release 7.1.1503 (Core)
Description
In our Cluster, I found some abnormal in ZKFC log
[2017-01-10T01:42:37.168+08:00] [INFO] hadoop.ha.SshFenceByTcpPort.doFence(SshFenceByTcpPort.java 147) [Health Monitor for NameNode at xxx-xxx-172xxx.hadoop.xxx.com/xxx.xxx.172.xxx:8021-EventThread] : Indeterminate response from trying to kill service. Verifying whether it is running using nc... [2017-01-10T01:42:37.234+08:00] [WARN] hadoop.ha.SshFenceByTcpPort.pump(StreamPumper.java 88) [nc -z xxx-xxx-172xx.hadoop.xx.com 8021 via ssh: StreamPumper for STDERR] : nc -z xxx-xxx-172xx.hadoop.xxx.com 8021 via ssh: nc: invalid option -- 'z' [2017-01-10T01:42:37.235+08:00] [WARN] hadoop.ha.SshFenceByTcpPort.pump(StreamPumper.java 88) [nc -z xxx-xxx-172xx.hadoop.xxx.com 8021 via ssh: StreamPumper for STDERR] : nc -z xxx-xxx-17224.hadoop.xxx.com 8021 via ssh: Ncat: Try `--help' or man(1) ncat for more information, usage options and help. QUITTING.
When I perform nc an exception occurs, the return value is 2, and cannot confirm sshfence success,this may lead to some problems
SshFenceByTcpPort.java
rc = execCommand(session, "nc -z " + serviceAddr.getHostName() + " " + serviceAddr.getPort()); if (rc == 0) { // the service is still listening - we are unable to fence LOG.warn("Unable to fence - it is running but we cannot kill it"); return false; } else { LOG.info("Verified that the service is down."); return true; }
Attachments
Issue Links
- duplicates
-
HDFS-3618 SSH fencing option may incorrectly succeed if nc (netcat) command not present
- Patch Available