[MAPREDUCE-1073] Progress reported for pipes tasks is incorrect.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.20.1
• Component/s: pipes
• Assignee: Dick King
• Reporter: Sreekanth Ramakrishnan
• Created: 07/Oct/09 08:27
• Updated: 29/Jan/12 03:14

Description

Currently in pipes, org.apache.hadoop.mapred.pipes.PipesMapRunner.run(RecordReader<K1, V1>, OutputCollector<K2, V2>, Reporter) we do the following:

        while (input.next(key, value)) {
          downlink.mapItem(key, value);
          if(skipping) {
            downlink.flush();
          }
        }

This would result in consumption of all the records for current task and taking task progress to 100% whereas the actual pipes application would be trailing behind.