[SPARK-7110] when use saveAsNewAPIHadoopFile, sometimes it throws "Delegation Token can be issued only with kerberos or web authentication"

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.1.0
• Component/s: YARN
• Assignee: Sean R. Owen
• Reporter: gu-chi
• Created: 24/Apr/15 03:42
• Updated: 26/May/15 06:54
• Resolved: 26/May/15 06:54

Description

Under yarn-client mode, this issue random occurs. Authentication method is set to kerberos, and use "saveAsNewAPIHadoopFile" in PairRDDFunctions to save data to HDFS, then exception comes as:
org.apache.hadoop.ipc.RemoteException(java.io.IOException): Delegation Token can be issued only with kerberos or web authentication

Issue Links

• duplicates: SPARK-3778 newAPIHadoopRDD doesn't properly pass credentials for secure hdfs on yarn
• links to: [Github] Pull Request #6052 (srowen)