Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1983

jobs using pipes interface with tasks not using java output format have a good chance of not updating progress and timing out

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.15.0
    • Fix Version/s: 0.15.0
    • Component/s: None
    • Labels:
      None

      Description

      When using C++-pipes interface, mappers/reducers not emitting any key-values pairs, but running longer than 'mapred.task.timeout' might timeout even when they send periodical status updates upstream.

      The cause of the problem is that all upstream messages are buffered. Progress and status updates should be flushed immediately.

        Attachments

        1. pipe-flush.patch
          3 kB
          Owen O'Malley

          Activity

            People

            • Assignee:
              omalley Owen O'Malley
              Reporter:
              ckunz Christian Kunz
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: