Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7893

Find a way to get a job identifier when submitting a spark job [Spark Branch]

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Spark
    • Labels:

      Description

      Currently we use the foreach RDD action to submit a spark job. In order to implement job monitoring functionality (HIVE-7438), we need to get a job identifier when submitting the job, so that we can later register some listener for that specific job.

      This task requires facilitation from spark side (SPARK-2636). I've tried to use AsyncRDDActions instead of the traditional actions, and it proved to be a possible way to get the job ID we need.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lirui Rui Li
                Reporter:
                lirui Rui Li
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: