Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20695

HoS Query fails with hive.exec.parallel=true

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.1
    • Fix Version/s: 4.0.0
    • Component/s: Spark
    • Labels:
      None
    • Target Version/s:

      Description

      Hive queries which fail when running a HiveOnSpark job:

      ERROR : Failed to execute spark task, with exception 'java.lang.Exception(Failed to submit Spark work, please retry later)'
      java.lang.Exception: Failed to submit Spark work, please retry later
              at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.execute(RemoteHiveSparkClient.java:186)
              at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.submit(SparkSessionImpl.java:71)
              at org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:107)
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99)
              at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
      Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /tmp/hive/dbname/_spark_session_dir/e202c452-8793-4e4e-ad55-61e3d4965c69/somename.jar (inode 725730760): File does not exist. [Lease.  Holder: DFSClient_NONMAPREDUCE_-1981084042_486659, pending creates: 7]
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3755)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:3556)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3412)
              at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:688)
      

        Attachments

        1. HIVE-20695.4.patch
          4 kB
          Yongzhi Chen
        2. HIVE-20695.3.patch
          4 kB
          Yongzhi Chen
        3. HIVE-20695.1.patch
          2 kB
          Yongzhi Chen

          Issue Links

            Activity

              People

              • Assignee:
                ychena Yongzhi Chen
                Reporter:
                ychena Yongzhi Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: