Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-10476

Hive query should fail when it fails to initialize a session in SetSparkReducerParallelism [Spark Branch]

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: spark-branch
    • Fix Version/s: 1.3.0, 2.0.0
    • Component/s: Spark
    • Labels:
      None

      Description

      Currently, for a Hive query HoS need to get a session
      a session twice, once in SparkSetReducerParallelism, and another when submitting the actual job.
      The issue is that sometimes there's problem when launching a Yarn application (e.g., don't have permission), then user will have to wait for two timeouts, because both session initializations will fail. This turned out to happen frequently.

      This JIRA proposes to fail the query in SparkSetReducerParallelism, when it cannot initialize the session.

        Attachments

        1. HIVE-10476.2-spark.patch
          0.9 kB
          Chao Sun
        2. HIVE-10476.1-spark.patch
          0.9 kB
          Chao Sun

          Issue Links

            Activity

              People

              • Assignee:
                csun Chao Sun
                Reporter:
                csun Chao Sun
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: