Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-933

Application defined InputSplits do not work

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.10.1
    • 0.12.0
    • None
    • None

    Description

      If an application defines its own InputSplit, the task tracker chokes when it cannot deserialize the InputSplit when it deserializes MapTasks it receives from the JobTracker. This is because the TaskTracker does not resolve classes from the job jar file. The attached patch delays resolution of the InputSplit until it is running in the context of the child process where it can resolve the InputSplit class.

      Attachments

        1. MapTask.patch
          4 kB
          Benjamin Reed
        2. JobInProgress.patch
          0.8 kB
          Benjamin Reed
        3. client-split.patch
          36 kB
          Owen O'Malley
        4. client-split-fixed.patch
          35 kB
          Nigel Daley
        5. client-split-3.patch
          38 kB
          Owen O'Malley

        Activity

          People

            omalley Owen O'Malley
            breed Benjamin Reed
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: