Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2846

a small % of all tasks fail with DefaultTaskController

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed
    • Fixed a race condition in writing the log index file that caused tasks to 'fail'.

    Description

      After upgrading our test 0.20.203 grid to 0.20.204-rc2, we ran terasort to verify operation. While the job completed successfully, approx 10% of the tasks failed with task runner execution errors and the inability to create symlinks for attempt logs.

      Attachments

        1. sync.patch
          0.9 kB
          Owen O'Malley
        2. sync-trunk.patch
          1 kB
          Owen O'Malley

        Issue Links

        is related to

        Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-2804 "Creation of symlink to attempt log dir failed." message is not useful

        • Blocker - Blocks development and/or testing work, production could not run
        • Closed
        relates to

        Sub-task - The sub-task of the issue MAPREDUCE-2415 Distribute TaskTracker userlogs onto multiple disks

        • Major - Major loss of function.
        • Closed

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            omalley Owen O'Malley Assign to me
            aw Allen Wittenauer
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment