Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Tags:
      ygridqa

      Description

      None of the test program seems to be supporting queue's concept. These programs looks for the default queue only even if some other queue is specified to run these programs.

      1. mapreduce-945-internal-3.8.patch.txt
        8 kB
        Sreekanth Ramakrishnan
      2. mapreduce-945-2.patch
        7 kB
        Sreekanth Ramakrishnan
      3. mapreduce-945-1.patch
        7 kB
        Sreekanth Ramakrishnan

        Activity

        Hide
        Suman Sehgal added a comment -

        Test programs mentioned above are the programs/jobs which can be run using test jar. Following are a few valid test programs:
        DFSCIOTest
        DistributedFSCheck
        MRReliabilityTest
        TestDFSIO
        Filebench
        Loadgen
        Mapredtest
        Mrbench
        Nnbench

        This issue pops-up while executing NNBench on multi-queue environment. Not able to run it on any of the queue viz. <abc>, <xyz> (any queue name except "default"). Every time error message is displayed "default queue doesn't exist" though examples programs (viz. sleep, sort etc.) are running fine on multi-queue envionnment. Queue name "default" seems to be hardcoded for these test programs.

        Show
        Suman Sehgal added a comment - Test programs mentioned above are the programs/jobs which can be run using test jar. Following are a few valid test programs: DFSCIOTest DistributedFSCheck MRReliabilityTest TestDFSIO Filebench Loadgen Mapredtest Mrbench Nnbench This issue pops-up while executing NNBench on multi-queue environment. Not able to run it on any of the queue viz. <abc>, <xyz> (any queue name except "default"). Every time error message is displayed "default queue doesn't exist" though examples programs (viz. sleep, sort etc.) are running fine on multi-queue envionnment. Queue name "default" seems to be hardcoded for these test programs.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Same's the case with grep-sort part of the Grep example. And the common problem is that rest of the arguments are not parsed with a GenericOptionsParser and so explicit configuration properties set on the command line are not passed onto the job.

        Show
        Vinod Kumar Vavilapalli added a comment - Same's the case with grep-sort part of the Grep example. And the common problem is that rest of the arguments are not parsed with a GenericOptionsParser and so explicit configuration properties set on the command line are not passed onto the job.
        Hide
        Sreekanth Ramakrishnan added a comment -

        After the project split the issue has to be split into two parts.

        Created a new HDFS JIRA: HDFS-587

        Show
        Sreekanth Ramakrishnan added a comment - After the project split the issue has to be split into two parts. Created a new HDFS JIRA: HDFS-587
        Hide
        rahul k singh added a comment -

        Iam fine with the patch except 1 small change,
        dont think this was introduced in this patch.
        variable " Path intrimData = null; " is not used in method runJobInSequence in MRBench.java

        Show
        rahul k singh added a comment - Iam fine with the patch except 1 small change, dont think this was introduced in this patch. variable " Path intrimData = null; " is not used in method runJobInSequence in MRBench.java
        Hide
        Sreekanth Ramakrishnan added a comment -

        Attaching patch addressing Rahuls comment.

        Show
        Sreekanth Ramakrishnan added a comment - Attaching patch addressing Rahuls comment.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12418797/mapreduce-945-2.patch
        against trunk revision 812209.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/12/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/12/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/12/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/12/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12418797/mapreduce-945-2.patch against trunk revision 812209. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/12/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/12/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/12/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/12/console This message is automatically generated.
        Hide
        Hemanth Yamijala added a comment -

        This seems fine to me.

        Show
        Hemanth Yamijala added a comment - This seems fine to me.
        Hide
        Sreekanth Ramakrishnan added a comment -

        The testcases modified were following:

        TestMapreduce
        MrBench

        The MRReliabilityTest and MapredTest tests were already modified before this patch.

        Show
        Sreekanth Ramakrishnan added a comment - The testcases modified were following: TestMapreduce MrBench The MRReliabilityTest and MapredTest tests were already modified before this patch.
        Hide
        Sreekanth Ramakrishnan added a comment -

        Attaching Y! distribution patch.

        Show
        Sreekanth Ramakrishnan added a comment - Attaching Y! distribution patch.
        Hide
        Sreekanth Ramakrishnan added a comment -

        All tests contrib and core passed locally.

        output from ant test-patch:

             [exec] +1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     +1 tests included.  The patch appears to include 6 new or modified tests.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
             [exec] 
        
        Show
        Sreekanth Ramakrishnan added a comment - All tests contrib and core passed locally. output from ant test-patch: [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 6 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12418910/mapreduce-945-internal-3.8.patch.txt
        against trunk revision 812486.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/46/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12418910/mapreduce-945-internal-3.8.patch.txt against trunk revision 812486. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/46/console This message is automatically generated.
        Hide
        Hemanth Yamijala added a comment -

        I ran the modified test case and verified it passed. Based on that, committed this to trunk. Thanks, Sreekanth !

        Show
        Hemanth Yamijala added a comment - I ran the modified test case and verified it passed. Based on that, committed this to trunk. Thanks, Sreekanth !
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #24 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/24/)
        . Modifies MRBench and TestMapRed to use ToolRunner so that options such as queue name can be passed via command line. Contributed by Sreekanth Ramakrishnan.

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #24 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/24/ ) . Modifies MRBench and TestMapRed to use ToolRunner so that options such as queue name can be passed via command line. Contributed by Sreekanth Ramakrishnan.

          People

          • Assignee:
            Sreekanth Ramakrishnan
            Reporter:
            Suman Sehgal
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development