Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1403

Save file-sizes of each of the artifacts in DistributedCache in the JobConf

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Added private configuration variables: mapred.cache.files.filesizes and mapred.cache.archives.filesizes to store sizes of distributed cache artifacts per job. This can be used by tools like Gridmix in simulation runs.

      Description

      It would be a useful metric to collect... potentially GridMix could use it to emulate jobs which use the DistributedCache.

      1. mr-1403-trunk-v4.patch
        10 kB
        Luke Lu
      2. mr-1403-trunk-v3.patch
        10 kB
        Luke Lu
      3. mr-1403-trunk-v2.patch
        11 kB
        Luke Lu
      4. MR-1403-trunk-1.patch
        11 kB
        Luke Lu
      5. MAPREDUCE-1403_yhadoop20-2.patch
        7 kB
        Hemanth Yamijala
      6. MAPREDUCE-1403_yhadoop20-1.patch
        7 kB
        Hemanth Yamijala
      7. MAPREDUCE-1403_yhadoop20.patch
        7 kB
        Arun C Murthy

        Activity

        Arun C Murthy created issue -
        Hong Tang made changes -
        Field Original Value New Value
        Assignee Arun C Murthy [ acmurthy ] Hong Tang [ hong.tang ]
        Arun C Murthy made changes -
        Assignee Hong Tang [ hong.tang ] Arun C Murthy [ acmurthy ]
        Arun C Murthy made changes -
        Attachment MAPREDUCE-1403_yhadoop20.patch [ 12435330 ]
        Hemanth Yamijala made changes -
        Attachment MAPREDUCE-1403_yhadoop20-1.patch [ 12435768 ]
        Hemanth Yamijala made changes -
        Release Note Added private configuration variables: mapred.cache.files.filesizes and mapred.cache.archives.filesizes to store sizes of distributed cache artifacts per job. This can be used by tools like Gridmix in simulation runs.
        Hemanth Yamijala made changes -
        Attachment MAPREDUCE-1403_yhadoop20-2.patch [ 12436842 ]
        Luke Lu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.22.0 [ 12314184 ]
        Luke Lu made changes -
        Attachment MR-1403-trunk-1.patch [ 12436997 ]
        Chris Douglas made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Luke Lu made changes -
        Attachment mr-1403-trunk-v2.patch [ 12438333 ]
        Luke Lu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Luke Lu made changes -
        Attachment mr-1403-trunk-v3.patch [ 12438342 ]
        Chris Douglas made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Chris Douglas made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Luke Lu made changes -
        Attachment mr-1403-trunk-v4.patch [ 12438471 ]
        Chris Douglas made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Tom White made changes -
        Fix Version/s 0.21.0 [ 12314045 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Arun C Murthy
            Reporter:
            Arun C Murthy
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development