[SPARK-8379] LeaseExpiredException when using dynamic partition with speculative execution

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 1.3.1, 1.4.0
• Fix Version/s: 1.4.1, 1.5.0
• Component/s: SQL
• Target Version/s: 1.5.0
• Assignee: jeanlyn
• Reporter: jeanlyn
• Shepherd: liancheng
• Created: 15/Jun/15 15:59
• Updated: 19/Sep/15 13:58
• Resolved: 21/Jun/15 07:14

Description

when inserting to table using dynamic partitions with spark.speculation=true and there is a skew data of some partitions trigger the speculative tasks ,it will throws the exception like

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): Lease mismatch on /tmp/hive-jeanlyn/hive_2015-06-15_15-20-44_734_8801220787219172413-1/-ext-10000/ds=2015-06-15/type=2/part-00301.lzo owned by DFSClient_attempt_201506031520_0011_m_000189_0_-1513487243_53 but is accessed by DFSClient_attempt_201506031520_0011_m_000042_0_-1275047721_57

Issue Links

• is duplicated by: SPARK-6067 Spark sql hive dynamic partitions job will fail if task fails
• links to: [Github] Pull Request #6833 (jeanlyn)