Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-933

Application defined InputSplits do not work

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      If an application defines its own InputSplit, the task tracker chokes when it cannot deserialize the InputSplit when it deserializes MapTasks it receives from the JobTracker. This is because the TaskTracker does not resolve classes from the job jar file. The attached patch delays resolution of the InputSplit until it is running in the context of the child process where it can resolve the InputSplit class.

        Attachments

        1. MapTask.patch
          4 kB
          Benjamin Reed
        2. JobInProgress.patch
          0.8 kB
          Benjamin Reed
        3. client-split-fixed.patch
          35 kB
          Nigel Daley
        4. client-split-3.patch
          38 kB
          Owen O'Malley
        5. client-split.patch
          36 kB
          Owen O'Malley

          Activity

            People

            • Assignee:
              omalley Owen O'Malley
              Reporter:
              breed Benjamin Reed
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: