Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2141

Add an "extra data" field to Task for use by Mesos

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None

      Description

      In order to support running Hadoop on the Mesos cluster manager (http://mesos.berkeley.edu/), I'd like to add an extra String field to the Task class to allow extra data (a Mesos task ID) to be associated with each task. This should have no impact on normal operation other than making the serialized form of Task a few bytes longer. In the Mesos support patch for Hadoop, this field is set by a pluggable Hadoop scheduler implementation to allow code on the TaskTracker side to see which Mesos task corresponds to each Hadoop task.

      1. mapreduce-2141-v1.patch
        1 kB
        Matei Zaharia
      2. mapreduce-2141-v2.patch
        2 kB
        Matei Zaharia

        Activity

        Matei Zaharia created issue -
        Matei Zaharia made changes -
        Field Original Value New Value
        Attachment mapreduce-2141-v1.patch [ 12457514 ]
        Matei Zaharia made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Matei Zaharia made changes -
        Attachment mapreduce-2141-v2.patch [ 12457646 ]
        Matei Zaharia made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Matei Zaharia made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Matei Zaharia
            Reporter:
            Matei Zaharia
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development