Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1100

User's task-logs filling up local disks on the TaskTrackers

    Details

      Description

      Some user's jobs are filling up TT disks by outrageous logging. mapreduce.task.userlog.limit.kb is not enabled on the cluster. Disks are getting filled up before task-log cleanup via mapred.task.userlog.retain.hours can kick in.

      1. reducetask-log-level.patch
        4 kB
        Todd Lipcon
      2. patch-1100-fix-ydist.2.txt
        13 kB
        Vinod Kumar Vavilapalli
      3. MAPREDUCE-1100-20091216.2.txt
        53 kB
        Arun C Murthy
      4. MAPREDUCE-1100-20091106.txt
        79 kB
        Vinod Kumar Vavilapalli
      5. MAPREDUCE-1100-20091102.txt
        51 kB
        Vinod Kumar Vavilapalli

        Activity

        Vinod Kumar Vavilapalli created issue -
        Vinod Kumar Vavilapalli made changes -
        Field Original Value New Value
        Assignee Vinod K V [ vinodkv ]
        Vinod Kumar Vavilapalli made changes -
        Attachment MAPREDUCE-1100-20091102.txt [ 12423815 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment MAPREDUCE-1100-20091106.txt [ 12424193 ]
        Vinod Kumar Vavilapalli made changes -
        Link This issue incorporates MAPREDUCE-927 [ MAPREDUCE-927 ]
        Arun C Murthy made changes -
        Attachment MAPREDUCE-1100-20091216.2.txt [ 12428200 ]
        Vinod Kumar Vavilapalli made changes -
        Link This issue incorporates MAPREDUCE-927 [ MAPREDUCE-927 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment patch-1100-fix-ydist.2.txt [ 12438394 ]
        Eli Collins made changes -
        Fix Version/s 0.21.1 [ 12315272 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Affects Version/s 0.20.2 [ 12314205 ]
        Affects Version/s 0.20.1 [ 12314047 ]
        Eli Collins made changes -
        Priority Major [ 3 ] Blocker [ 1 ]
        Todd Lipcon made changes -
        Attachment reducetask-log-level.patch [ 12471047 ]
        Konstantin Shvachko made changes -
        Priority Blocker [ 1 ] Major [ 3 ]
        Arun C Murthy made changes -
        Target Version/s 0.22.0 [ 12314184 ]
        Arun C Murthy made changes -
        Labels critical-0.22.0
        Konstantin Shvachko made changes -
        Fix Version/s 0.22.1 [ 12319242 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Allen Wittenauer made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Won't Fix [ 2 ]

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            Vinod Kumar Vavilapalli
          • Votes:
            0 Vote for this issue
            Watchers:
            25 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development