[HBASE-23333] Include simple Call.toShortString() in sendCall exceptions

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1, 2.2.0, 2.3.0, 2.1.6
• Fix Version/s: 3.0.0-alpha-1, 2.3.0, 2.2.3, 2.1.9
• Component/s: Client, Operability
• Assignee: Nick Dimiduk
• Reporter: Nick Dimiduk
• Created: 23/Nov/19 00:07
• Updated: 24/Jun/22 19:09
• Resolved: 03/Jan/20 20:58

Description

Failure diagnosis isn't very straightforward with call stack traces like

org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to c501d28b0dfa/172.17.0.2:45657 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508, waitTime=60006, rpcTimeout=60000
	at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:204)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:392)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:97)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:423)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:419)
	at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
	at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
	at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:680)
	at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:755)
	at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:483)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=508, waitTime=60006, rpcTimeout=60000
	at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
	... 4 more

Probably the "affectsVersions" goes back farther than this.

See if we can provide more calling context, even stack trace from the call origin, in these exceptions.

Issue Links

• is related to: HBASE-22316 Record the stack trace for current thread in FutureUtils.get
• links to: GitHub Pull Request #872
• links to: GitHub Pull Request #984
• links to: GitHub Pull Request #985
• links to: GitHub Pull Request #986