Hive
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-7439

Spark job monitoring and error reporting [Spark Branch]

    Details

      Description

      After Hive submits a job to Spark cluster, we need to report to user the job progress, such as the percentage done, to the user. This is especially important for long running queries. Moreover, if there is an error during job submission or execution, it's also crucial for hive to fetch the error log and/or stacktrace and feedback it to the user.

      Please refer design doc on wiki for more information.

      CLEAR LIBRARY CACHE

      1. hive on spark job status.PNG
        34 kB
        Chengxiang Li
      2. HIVE-7439.3-spark.patch
        27 kB
        Chengxiang Li
      3. HIVE-7439.3-spark.patch
        27 kB
        Xuefu Zhang
      4. HIVE-7439.2-spark.patch
        27 kB
        Chengxiang Li
      5. HIVE-7439.2-spark.patch
        27 kB
        Xuefu Zhang
      6. HIVE-7439.1-spark.patch
        26 kB
        Chengxiang Li

        Issue Links

          Activity

          Xuefu Zhang created issue -
          Chengxiang Li made changes -
          Field Original Value New Value
          Assignee Chengxiang Li [ chengxiang li ]
          Chengxiang Li made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Chengxiang Li made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          Brock Noland made changes -
          Summary Spark job monitoring and error reporting Spark job monitoring and error reporting [Spark Branch]
          Brock Noland made changes -
          Labels Spark-M3
          Brock Noland made changes -
          Link This issue relates to HIVE-7874 [ HIVE-7874 ]
          Chengxiang Li made changes -
          Attachment HIVE-7439.1-spark.patch [ 12674141 ]
          Chengxiang Li made changes -
          Attachment hive on spark job status.PNG [ 12674143 ]
          Chengxiang Li made changes -
          Attachment HIVE-7439.2-spark.patch [ 12674333 ]
          Attachment hive on spark job status.PNG [ 12674334 ]
          Chengxiang Li made changes -
          Attachment hive on spark job status.PNG [ 12674143 ]
          Chengxiang Li made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Xuefu Zhang made changes -
          Attachment HIVE-7439.2-spark.patch [ 12674413 ]
          Chengxiang Li made changes -
          Attachment HIVE-7439.3-spark.patch [ 12674471 ]
          Xuefu Zhang made changes -
          Description After Hive submits a job to Spark cluster, we need to report to user the job progress, such as the percentage done, to the user. This is especially important for long running queries. Moreover, if there is an error during job submission or execution, it's also crucial for hive to fetch the error log and/or stacktrace and feedback it to the user.

          Please refer design doc on wiki for more information.
          After Hive submits a job to Spark cluster, we need to report to user the job progress, such as the percentage done, to the user. This is especially important for long running queries. Moreover, if there is an error during job submission or execution, it's also crucial for hive to fetch the error log and/or stacktrace and feedback it to the user.

          Please refer design doc on wiki for more information.

          CLEAR LIBRARY CACHE
          Xuefu Zhang made changes -
          Attachment HIVE-7439.3-spark.patch [ 12674525 ]
          Xuefu Zhang made changes -
          Remote Link This issue links to "https://reviews.apache.org/r/26545/ (Web Link)" [ 19246 ]
          Rui Li made changes -
          Link This issue relates to SPARK-3902 [ SPARK-3902 ]
          Xuefu Zhang made changes -
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s spark-branch [ 12327352 ]
          Lefty Leverenz made changes -
          Link This issue relates to HIVE-7893 [ HIVE-7893 ]
          Lefty Leverenz made changes -
          Labels Spark-M3 Spark-M3 TODOC-SPARK
          Lefty Leverenz made changes -
          Link This issue is related to HIVE-8834 [ HIVE-8834 ]

            People

            • Assignee:
              Chengxiang Li
              Reporter:
              Xuefu Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development