Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4087

[Gridmix] GenerateDistCacheData job of Gridmix can become slow in some cases

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0, 3.0.0, 2.1.0-beta
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixes the issue of GenerateDistCacheData job slowness.

      Description

      In map() method of GenerateDistCacheData job of Gridmix, val.setSize() is done every time based on the bytes to be written to a distributed cache file. When we try to write data to next distributed cache file in the same map task, the size of random data generated in each iteration can become small based on the particular case. This can make this dist cache data generation slow.

      1. 4087.trunk.patch
        1 kB
        Ravi Gummadi
      2. 4087.patch
        1 kB
        Ravi Gummadi

        Activity

        Ravi Gummadi created issue -
        Ravi Gummadi made changes -
        Field Original Value New Value
        Attachment 4087.patch [ 12520592 ]
        Ravi Gummadi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Release Note Fixes the issue of Generate Dist Cache Data generation job slowness.
        Ravi Gummadi made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Ravi Gummadi made changes -
        Attachment 4087.trunk.patch [ 12520595 ]
        Ravi Gummadi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hadoop Flags Reviewed [ 10343 ]
        Ravi Gummadi made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Release Note Fixes the issue of Generate Dist Cache Data generation job slowness. Fixes the issue of GenerateDistCacheData job slowness.
        Resolution Fixed [ 1 ]
        Matt Foley made changes -
        Fix Version/s 1.1.0 [ 12317960 ]
        Fix Version/s 3.0.0 [ 12320355 ]
        Matt Foley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Thomas Graves made changes -
        Fix Version/s 2.0.5-beta [ 12324032 ]

          People

          • Assignee:
            Ravi Gummadi
            Reporter:
            Ravi Gummadi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development