Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2722

Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed input is used

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.0-beta
    • Component/s: contrib/gridmix
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Makes Gridmix use the uncompressed input data size while simulating map tasks in the case where compressed input data was used in original job.

      Description

      When compressed input was used by original job's map task, then the simulated job's map task's hdfsBytesRead counter is wrong if compression emulation is enabled. This issue is because hdfsBytesRead of map task of original job is considered as uncompressed map input size by Gridmix.

        Attachments

        1. 2722.v1.patch
          5 kB
          Ravi Gummadi
        2. 2722.v2.1.patch
          6 kB
          Ravi Gummadi
        3. MR2722.patch
          1 kB
          Ravi Gummadi

          Activity

            People

            • Assignee:
              ravidotg Ravi Gummadi
              Reporter:
              ravidotg Ravi Gummadi
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: