Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.22.0
    • Component/s: security, tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Moved Task log cleanup into a separate thread in TaskTracker.
      Added configuration "mapreduce.job.userlog.retain.hours" to specify the time(in hours) for which the user-logs are to be retained after the job completion.
      Show
      Moved Task log cleanup into a separate thread in TaskTracker. Added configuration "mapreduce.job.userlog.retain.hours" to specify the time(in hours) for which the user-logs are to be retained after the job completion.

      Description

      Task logs' cleanup is being done in Child now. This is undesirable atleast for two reasons: 1) failures while cleaning up will affect the user's tasks, and 2) the task's wall time will get affected due to operations that TT actually should own.

      1. patch-927.txt
        31 kB
        Amareshwari Sriramadasu
      2. patch-927-1.txt
        31 kB
        Amareshwari Sriramadasu
      3. patch-927-2.txt
        43 kB
        Amareshwari Sriramadasu
      4. patch-927-3.txt
        51 kB
        Amareshwari Sriramadasu
      5. patch-927-4.txt
        50 kB
        Amareshwari Sriramadasu
      6. patch-927-5-dist.txt
        152 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Vinod Kumar Vavilapalli made changes -
          Fix Version/s 0.22.0 [ 12314184 ]
          Fix Version/s 0.21.0 [ 12314045 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Tom White made changes -
          Fix Version/s 0.21.0 [ 12314045 ]
          Fix Version/s 0.22.0 [ 12314184 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment patch-927-5-dist.txt [ 12439009 ]
          Amareshwari Sriramadasu made changes -
          Release Note Moved Task log cleanup into a separate thread in TaskTracker.
          Added configuration "mapreduce.job.userlog.retain.hours" to specify the time(in hours) for which the user-logs are to be retained after the job completion.
          Priority Blocker [ 1 ] Major [ 3 ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Amareshwari Sriramadasu made changes -
          Attachment patch-927-4.txt [ 12438257 ]
          Amareshwari Sriramadasu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Amareshwari Sriramadasu made changes -
          Attachment patch-927-3.txt [ 12438174 ]
          Amareshwari Sriramadasu made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Amareshwari Sriramadasu made changes -
          Link This issue is blocked by MAPREDUCE-1553 [ MAPREDUCE-1553 ]
          Amareshwari Sriramadasu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 0.22.0 [ 12314184 ]
          Fix Version/s 0.21.0 [ 12314045 ]
          Amareshwari Sriramadasu made changes -
          Attachment patch-927-2.txt [ 12437694 ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue is part of MAPREDUCE-1100 [ MAPREDUCE-1100 ]
          Amareshwari Sriramadasu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Amareshwari Sriramadasu made changes -
          Attachment patch-927-1.txt [ 12436547 ]
          Amareshwari Sriramadasu made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Amareshwari Sriramadasu made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Amareshwari Sriramadasu made changes -
          Attachment patch-927.txt [ 12436285 ]
          Amareshwari Sriramadasu made changes -
          Assignee Amareshwari Sriramadasu [ amareshwari ]
          Amareshwari Sriramadasu made changes -
          Component/s security [ 12313041 ]
          Amareshwari Sriramadasu made changes -
          Parent MAPREDUCE-1100 [ 12437951 ]
          Issue Type Bug [ 1 ] Sub-task [ 7 ]
          Amareshwari Sriramadasu made changes -
          Resolution Duplicate [ 3 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue is part of MAPREDUCE-1100 [ MAPREDUCE-1100 ]
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Fix Version/s 0.21.0 [ 12314045 ]
          Affects Version/s 0.21.0 [ 12314045 ]
          Priority Major [ 3 ] Blocker [ 1 ]
          Vinod Kumar Vavilapalli created issue -

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development