Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: contrib/mrunit
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently the library creates a new Configuration object in the MockMapContext and MocKReduceContext constructors, rather than allowing the developer to configure and pass their own

      1. MAPREDUCE-1569.patch
        17 kB
        Chris White
      2. MAPREDUCE-1569.patch
        15 kB
        Chris White
      3. MAPREDUCE-1569.patch
        15 kB
        Chris White

        Issue Links

          Activity

          Hide
          Chris White added a comment -

          Initial patch to resolve this issue (for review)

          Show
          Chris White added a comment - Initial patch to resolve this issue (for review)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12438155/MAPREDUCE-1569.patch
          against trunk revision 920250.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 12 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/508/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12438155/MAPREDUCE-1569.patch against trunk revision 920250. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/508/console This message is automatically generated.
          Hide
          Chris White added a comment -

          Fixed build problems

          Show
          Chris White added a comment - Fixed build problems
          Hide
          Chris White added a comment -

          Attempting to restart Hudson job

          Show
          Chris White added a comment - Attempting to restart Hudson job
          Hide
          Chris White added a comment -

          Uploading patch again in attempt to restart Hudson build

          Show
          Chris White added a comment - Uploading patch again in attempt to restart Hudson build
          Hide
          Chris White added a comment -

          Another attempt at patch submission

          Show
          Chris White added a comment - Another attempt at patch submission
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12438280/MAPREDUCE-1569.patch
          against trunk revision 921230.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 12 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/31/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/31/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/31/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/31/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12438280/MAPREDUCE-1569.patch against trunk revision 921230. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/31/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/31/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/31/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/31/console This message is automatically generated.
          Hide
          Aaron Kimball added a comment -

          Hi Chris,

          This looks like a good start! The test failure in Hudson is unrelated to this patch.

          Here's some comments on the patch itself:

          • Can you wrap your new lines at 80 columns?
          • The withConfiguration() methods will need to be pushed down (as they are) into all the specific classes (MapDriver, ReduceDriver, etc). But I think the getConfiguration() method can live in TestDriver, eliminating redundant code.
          • For API consistency, we should have a public void setConfiguration(Configuration conf) method of TestDriver as well. (And withConfiguration() should call setConfiguration() rather than manipulating the field explicitly.)
          • withConfiguration()'s Javadoc @return needs to be filled out.
          • For MockMapContext/MockReduceContext, instead of changing the existing constructor, can you add another constructor that takes the configuration as an argument? The existing constructor can call the new one with a new Configuration() there.

          When you fix these, please just attach another patch to this issue and redo the cancel patch / submit patch dance.

          Show
          Aaron Kimball added a comment - Hi Chris, This looks like a good start! The test failure in Hudson is unrelated to this patch. Here's some comments on the patch itself: Can you wrap your new lines at 80 columns? The withConfiguration() methods will need to be pushed down (as they are) into all the specific classes (MapDriver, ReduceDriver, etc). But I think the getConfiguration() method can live in TestDriver, eliminating redundant code. For API consistency, we should have a public void setConfiguration(Configuration conf) method of TestDriver as well. (And withConfiguration() should call setConfiguration() rather than manipulating the field explicitly.) withConfiguration()'s Javadoc @return needs to be filled out. For MockMapContext/MockReduceContext, instead of changing the existing constructor, can you add another constructor that takes the configuration as an argument? The existing constructor can call the new one with a new Configuration() there. When you fix these, please just attach another patch to this issue and redo the cancel patch / submit patch dance.
          Hide
          Chris White added a comment -

          Cancelling current patch to upload new patch

          Show
          Chris White added a comment - Cancelling current patch to upload new patch
          Hide
          Chris White added a comment -

          Patch with Aaron's review comments actioned

          Show
          Chris White added a comment - Patch with Aaron's review comments actioned
          Hide
          Chris White added a comment -

          Patch with review comments actioned

          Show
          Chris White added a comment - Patch with review comments actioned
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12438682/MAPREDUCE-1569.patch
          against trunk revision 923907.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 12 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/33/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/33/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/33/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/33/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12438682/MAPREDUCE-1569.patch against trunk revision 923907. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/33/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/33/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/33/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/33/console This message is automatically generated.
          Hide
          Aaron Kimball added a comment -

          New patch looks good; +1.

          Show
          Aaron Kimball added a comment - New patch looks good; +1.
          Hide
          Chris Douglas added a comment -

          I committed this. Thanks, Chris!

          Show
          Chris Douglas added a comment - I committed this. Thanks, Chris!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #286 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/286/)
          . Pass configuration through mocked contexts in MRUnit. Contributed by Chris White

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #286 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/286/ ) . Pass configuration through mocked contexts in MRUnit. Contributed by Chris White
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #264 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/264/)
          . Pass configuration through mocked contexts in MRUnit. Contributed by Chris White

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #264 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/264/ ) . Pass configuration through mocked contexts in MRUnit. Contributed by Chris White

            People

            • Assignee:
              Chris White
              Reporter:
              Chris White
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development