Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2722

Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed input is used

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.1.0-beta
    • contrib/gridmix
    • None
    • Reviewed
    • Makes Gridmix use the uncompressed input data size while simulating map tasks in the case where compressed input data was used in original job.

    Description

      When compressed input was used by original job's map task, then the simulated job's map task's hdfsBytesRead counter is wrong if compression emulation is enabled. This issue is because hdfsBytesRead of map task of original job is considered as uncompressed map input size by Gridmix.

      Attachments

        1. 2722.v2.1.patch
          6 kB
          Ravi Gummadi
        2. 2722.v1.patch
          5 kB
          Ravi Gummadi
        3. MR2722.patch
          1 kB
          Ravi Gummadi

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ravidotg Ravi Gummadi
            ravidotg Ravi Gummadi
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment