Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6762

ControlledJob#toString failed with NPE when job status is not successfully updated

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.2
    • 2.9.0, 3.0.0-alpha1
    • None
    • None
    • Reviewed

    Description

      This issue was found from a cluster where Pig query occasionally failed on NPE. Pig uses JobControl API to track MR job status, but sometimes Job History Server failed to flush job meta files to HDFS which caused the status update failed. Then we get NPE in org.apache.hadoop.mapreduce.Job.getJobName. The result of this situation is quite confusing: Pig query failed, job history is missing, but the job status on Yarn is succeed.

      Attachments

        1. MAPREDUCE-6762.001.patch
          3 kB
          Weiwei Yang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            cheersyang Weiwei Yang
            cheersyang Weiwei Yang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment