Progress is not updated when the New Reducer is running reduce phase

• Key: HADOOP-5882
• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Priority: Blocker
• Assignee: Amareshwari Sriramadasu
• Reporter: Jothi Padmanabhan
• Affects Version/s: 0.20.0
• Fix Version/s: 0.20.1

Description

The old reducer calls informReduceProgress on a key change to update the progress. This call or equivalent should be called for the new reducer as well.

Comments

[Amareshwari Sriramadasu]

Marking it a blocker, I see reducers failing because they couldn't report status for 600 seconds, during reduce phase.

[Owen O'Malley]

This needs to be addressed like the MapTask does it where the progress is updated when the code calls next. In the new api, the loop is handled in user code and therefore can't require the extra call to update progress.

[Amareshwari Sriramadasu]

Patch adding progress during reduce phase. Tested the patch manually to see progress updates during reduce phase.

[Amareshwari Sriramadasu]

test-patch result:

     [exec]
     [exec] -1 overall.
     [exec]
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec]
     [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
     [exec]                         Please justify why no tests are needed for this patch.
     [exec]
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec]
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec]
     [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
     [exec]
     [exec]     -1 release audit.  The applied patch generated 497 release audit warnings (more than the trunk's current 495 warnings).
     [exec]

-1 release audit. There are no new files added in the patch. I dont know why it is giving -1 release audit.
-1 tests included. It is difficult to a unit test for observing progress updates during reduce phase. Patch has been tested manually

All mapred unit tests passed on my machine.

[Hadoop QA]

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12408794/patch-5882.txt
against trunk revision 778388.

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 Eclipse classpath. The patch retains Eclipse classpath integrity.

-1 release audit. The applied patch generated 493 release audit warnings (more than the trunk's current 491 warnings).

-1 core tests. The patch failed core unit tests.

-1 contrib tests. The patch failed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/398/testReport/
Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/398/artifact/trunk/current/releaseAuditDiffWarnings.txt
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/398/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/398/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/398/console

This message is automatically generated.

[Amareshwari Sriramadasu]

Test failures are not related to the patch.

[Sharad Agarwal]

Changing the ReduceContext and having the constructor with null progress object does not seem very elegant. Would it work, if we just intercept RawKeyValueIterator.next()? Something like:

final RawKeyValueIterator rawIter = rIter;
    rIter = new RawKeyValueIterator() {
      public void close() throws IOException {
        rawIter.close();
      }
      public DataInputBuffer getKey() throws IOException {
        return rawIter.getKey();
      }
      public Progress getProgress() {
        return rawIter.getProgress();
      }
      public DataInputBuffer getValue() throws IOException {
        return rawIter.getValue();
      }
      public boolean next() throws IOException {
        reducePhase.set(rawIter.getProgress().getProgress());
        return rawIter.next();
      }
    };

This looks much simpler to me.

[Amareshwari Sriramadasu]

Code changes as suggested Sharad, wraping RawKeyValueIterator to report progress. Tested the patch on cluster, it works fine

[Hadoop QA]

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12409363/patch-5882-1.txt
against trunk revision 780114.

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 Eclipse classpath. The patch retains Eclipse classpath integrity.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

-1 core tests. The patch failed core unit tests.

-1 contrib tests. The patch failed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/437/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/437/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/437/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/437/console

This message is automatically generated.

[Amareshwari Sriramadasu]

Test failures are not related to the patch.
All tests passed on my machine

[Tsz Wo (Nicholas), SZE]

Seems that this was committed to 0.20 only but not 0.21.

[Amareshwari Sriramadasu]

Seems that this was committed to 0.20 only but not 0.21.

It is committed to 0.21, but not 0.20.1

The patch has been verified by running sort with new api, and observing progress update in redude from 66% to 100% (the reduce phase).

[Amareshwari Sriramadasu]

Patch does not apply to 0.20. I'm testing the patch for 0.20. Will upload a patch soon.

[Amareshwari Sriramadasu]

Patch does not apply to 0.20. I'm testing the patch for 0.20. Will upload a patch soon.

[Amareshwari Sriramadasu]

Uploaded patch doesnt solve the issue. Patch was wrongly created. I will upload correct patch for trunk and 0.20.

[Amareshwari Sriramadasu]

Patch for branch 0.20.1

[Amareshwari Sriramadasu]

patch for trunk

[Amareshwari Sriramadasu]

ant test and test patch passed on my machine.

[Sharad Agarwal]

I committed this to trunk and branch 0.20.1. Thanks Amareshwari!

[Hudson]

Integrated in Hadoop-trunk #863 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/863/)