[YARN-4558] Yarn client retries on some non-retriable exceptions

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.7.1
• Assignee: Unassigned
• Reporter: Sergey Shelukhin
• Created: 07/Jan/16 21:51
• Updated: 05/Feb/16 15:35

Description

Seems the problem is in RMProxy where the policy is built.

Thread 23594: (state = BLOCKED)
- java.lang.Thread.sleep(long) @bci=0 (Interpreted frame)
- org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(java.lang.Object, java.lang.reflect.Method, java.lang.Object[]) @bci=603, line=155 (Interpreted frame)
- com.sun.proxy.$Proxy32.getClusterNodes(org.apache.hadoop.yarn.api.protocolrecords.GetClusterNodesRequest) @bci=16 (Interpreted frame)
- org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNodeReports(org.apache.hadoop.yarn.api.records.NodeState[]) @bci=66, line=515 (Interpreted frame)

produces

2016-01-07 02:50:45,111 [main] WARN  ipc.Client - Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
2016-01-07 02:51:15,126 [main] WARN  ipc.Client - Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
...

Issue Links

• relates to: HADOOP-12697 IPC retry policies should recognise that SASL auth failures are unrecoverable
• relates to: HADOOP-10036 RetryInvocationHandler should recognise that there is no point retrying to auth failures