Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3937

Job history may get disabled due to overly long job names

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.17.0, 0.17.1, 0.18.0, 0.19.0
    • 0.19.0
    • None
    • None
    • Reviewed

    Description

      Since Hadoop 0.17, the job history logs include the job's name in the filename. However, this can lead to overly long filenames, because job names may be arbitrarily long. When a filename is too long for the underlying OS, file creation fails and the JobHistory class silently disables history from that point on. This can lead to days of lost history until somebody notices the error in the log.

      Proposed solution: Trim the job name to a reasonable length when selecting a filename for the history file.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            matei@eecs.berkeley.edu Matei Zaharia Assign to me
            matei@eecs.berkeley.edu Matei Zaharia
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment