[HDFS-3972] Trash emptier fails in secure HA cluster

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.0.1-alpha, 3.0.0-alpha1
• Fix Version/s: 2.0.2-alpha
• Component/s: namenode
• Hadoop Flags: Reviewed
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 25/Sep/12 19:29
• Updated: 12/May/16 18:17
• Resolved: 26/Sep/12 21:27

Description

In a secure HA cluster, we're seeing the following issue on the NN when the trash emptier tries to run:

WARN org.apache.hadoop.fs.TrashPolicyDefault: Trash can't list homes: java.io.IOException: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host \
is: "xxxxx"; destination host is: "xxxx":8020; Sleeping.

The issue seems to be that the trash emptier thread sends RPCs back to itself, but isn't wrapped in a doAs. Credit goes to Stephen Chu for discovering this.