Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Updates to task timing information were fixed in some code paths that led to inconsistent metering of task run times. Also, checks were added to guard against inconsistencies and record information about state if they should happen to occur.

      Description

      For some jobs the mapSlotSeconds is incorrect.

      negative value

      09/09/01 18:31:44 INFOmapred.JobInProgress$JobSummary: jobId=job_200908270718_4568,submitTime=1251823543976,launchTime=1251823554310,finishTime=1251829904565, numMaps=7965,numSlotsPerMap=1,numReduces=40,numSlotsPerReduce=1,user=wile,queue=runner,status=SUCCEEDED, mapSlotSeconds=-2503133523,reduceSlotsSeconds=186536,clusterMapCapacity=11262,clusterReduceCapacity=3754

      or too high

      09/09/02 23:59:57 INFO mapred.JobInProgress$JobSummary: jobId=job_200908270718_5861,submitTime=1251935672924,launchTime=1251935687698,finishTime=1251935997949, numMaps=1026,numSlotsPerMap=1,numReduces=10,numSlotsPerReduce=1,user=dfsload,queue=gridops,status=SUCCEEDED,
      mapSlotSeconds=1251949742,reduceSlotsSeconds=537,clusterMapCapacity=11262,clusterReduceCapacity=3754

      1. mapreduce-964-1.patch
        2 kB
        Sreekanth Ramakrishnan
      2. mapreduce-964-ydist.patch
        5 kB
        Sreekanth Ramakrishnan
      3. mapreduce-964-2.patch
        6 kB
        Sreekanth Ramakrishnan
      4. mapreduce-964-ydist.patch
        6 kB
        Sreekanth Ramakrishnan
      5. mapreduce-964-3.patch
        6 kB
        Sreekanth Ramakrishnan
      6. MAPREDUCE-964-4.patch
        7 kB
        Hemanth Yamijala
      7. mapreduce-964-ydist-1.patch
        0.5 kB
        Sreekanth Ramakrishnan

        Activity

        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hemanth Yamijala made changes -
        Release Note Updates to task timing information were fixed in some code paths that led to inconsistent metering of task run times. Also, checks were added to guard against inconsistencies and record information about state if they should happen to occur.
        Sreekanth Ramakrishnan made changes -
        Attachment mapreduce-964-ydist-1.patch [ 12420893 ]
        Hemanth Yamijala made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.21.0 [ 12314045 ]
        Resolution Fixed [ 1 ]
        Hemanth Yamijala made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hemanth Yamijala made changes -
        Attachment MAPREDUCE-964-4.patch [ 12420557 ]
        Hemanth Yamijala made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Sreekanth Ramakrishnan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Sreekanth Ramakrishnan made changes -
        Attachment mapreduce-964-3.patch [ 12420540 ]
        Sreekanth Ramakrishnan made changes -
        Attachment mapreduce-964-ydist.patch [ 12420539 ]
        Sreekanth Ramakrishnan made changes -
        Attachment mapreduce-964-2.patch [ 12420534 ]
        Sreekanth Ramakrishnan made changes -
        Attachment mapreduce-964-ydist.patch [ 12420533 ]
        Sreekanth Ramakrishnan made changes -
        Attachment mapreduce-964-1.patch [ 12420449 ]
        Hemanth Yamijala made changes -
        Assignee Sreekanth Ramakrishnan [ sreekanth ]
        Hong Tang made changes -
        Field Original Value New Value
        Priority Major [ 3 ] Critical [ 2 ]
        Rajiv Chittajallu created issue -

          People

          • Assignee:
            Sreekanth Ramakrishnan
            Reporter:
            Rajiv Chittajallu
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development