Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4100

Sometimes gridmix emulates data larger much larger then acutal counter for map only jobs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 2.1.0-beta
    • Component/s: contrib/gridmix
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Bug fixed in compression emulation feature for map only jobs.
    • Tags:
      gridmix compression-emulation map-only-jobs

      Description

      While running 1400+ jobs trace I encountered this issue.
      For map-only jobs, observed that some Maps generating data of around 9 GB (From HDFS_BYTES_WRITTEN) whereas actual value is around 5GB in trace.
      This can sometimes also cause jobs to fail intermittently.

      Other GridMix version coming be Hadoop-1.1.X and above might also effected

        Attachments

        1. mapreduce-4100-v1.2.patch
          8 kB
          Amar Kamat

          Activity

            People

            • Assignee:
              amar_kamat Amar Kamat
              Reporter:
              karams Karam Singh
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: