Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6762

ControlledJob#toString failed with NPE when job status is not successfully updated

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.2
    • 2.9.0, 3.0.0-alpha1
    • None
    • None
    • Reviewed

    Description

      This issue was found from a cluster where Pig query occasionally failed on NPE. Pig uses JobControl API to track MR job status, but sometimes Job History Server failed to flush job meta files to HDFS which caused the status update failed. Then we get NPE in org.apache.hadoop.mapreduce.Job.getJobName. The result of this situation is quite confusing: Pig query failed, job history is missing, but the job status on Yarn is succeed.

      Attachments

        1. MAPREDUCE-6762.001.patch
          3 kB
          Weiwei Yang

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. PIG-4967 NPE in PigJobControl.run() when job status is null

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-7179 Job status not available when job finished

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              cheersyang Weiwei Yang
              cheersyang Weiwei Yang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: