[HIVE-20695] HoS Query fails with hive.exec.parallel=true

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.1
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Spark
• Target Version/s: 4.0.0
• Assignee: Yongzhi Chen
• Reporter: Yongzhi Chen
• Created: 05/Oct/18 14:24
• Updated: 17/Nov/22 08:54
• Resolved: 13/Oct/18 03:51

Description

Hive queries which fail when running a HiveOnSpark job:

ERROR : Failed to execute spark task, with exception 'java.lang.Exception(Failed to submit Spark work, please retry later)'
java.lang.Exception: Failed to submit Spark work, please retry later
        at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.execute(RemoteHiveSparkClient.java:186)
        at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.submit(SparkSessionImpl.java:71)
        at org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:107)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /tmp/hive/dbname/_spark_session_dir/e202c452-8793-4e4e-ad55-61e3d4965c69/somename.jar (inode 725730760): File does not exist. [Lease.  Holder: DFSClient_NONMAPREDUCE_-1981084042_486659, pending creates: 7]
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3755)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:3556)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3412)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:688)

Issue Links

• relates to: HIVE-20737 Local SparkContext is shared between user sessions and should be closed only when there is no active