Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.3
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: contrib/gridmix
    • Labels:
      None

      Description

      when trying to run gridmix its actually trying to run org.apache.hadoop.tools.HadoopArchives.

      the pom.xml needs to be fixed to have correct mainClass: org.apache.hadoop.mapred.gridmix.Gridmix

      1. MAPREDUCE-4320.patch
        0.5 kB
        Thomas Graves
      2. MAPREDUCE-4320.patch
        0.5 kB
        Thomas Graves
      3. MAPREDUCE-4320.patch
        0.5 kB
        Thomas Graves

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1117 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1117/)
        MAPREDUCE-4320. gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1117 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1117/ ) MAPREDUCE-4320 . gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #294 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/294/)
        merge -r 1352550:1352551 from branch-2. FIXES: MAPREDUCE-4320 (Revision 1352552)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352552
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-gridmix/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #294 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/294/ ) merge -r 1352550:1352551 from branch-2. FIXES: MAPREDUCE-4320 (Revision 1352552) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352552 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-gridmix/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1084 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1084/)
        MAPREDUCE-4320. gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1084 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1084/ ) MAPREDUCE-4320 . gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2395 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2395/)
        MAPREDUCE-4320. gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2395 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2395/ ) MAPREDUCE-4320 . gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2446 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2446/)
        MAPREDUCE-4320. gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2446 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2446/ ) MAPREDUCE-4320 . gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2376 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2376/)
        MAPREDUCE-4320. gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2376 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2376/ ) MAPREDUCE-4320 . gridmix mainClass wrong in pom.xml (tgraves) (Revision 1352550) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1352550 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-gridmix/pom.xml
        Hide
        Thomas Graves added a comment -

        thanks for the review Bobby. I've committed this to trunk, branch-2, and branch-0.23.

        Show
        Thomas Graves added a comment - thanks for the review Bobby. I've committed this to trunk, branch-2, and branch-0.23.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12532757/MAPREDUCE-4320.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 javadoc. The javadoc tool appears to have generated 13 warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-gridmix.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2487//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2487//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2487//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532757/MAPREDUCE-4320.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 javadoc. The javadoc tool appears to have generated 13 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-gridmix. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2487//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2487//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2487//console This message is automatically generated.
        Hide
        Robert Joseph Evans added a comment -

        The change looks good to me +1. I see no way that TestResourceUsageEmulators could fail because of this change.

        Show
        Robert Joseph Evans added a comment - The change looks good to me +1. I see no way that TestResourceUsageEmulators could fail because of this change.
        Hide
        Thomas Graves added a comment -

        I wasn't able to reproduce the test failure. Trying again.

        Show
        Thomas Graves added a comment - I wasn't able to reproduce the test failure. Trying again.
        Hide
        Thomas Graves added a comment -

        investigating test failure

        Show
        Thomas Graves added a comment - investigating test failure
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12532717/MAPREDUCE-4320.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 javadoc. The javadoc tool appears to have generated 13 warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-tools/hadoop-gridmix:

        org.apache.hadoop.mapred.gridmix.TestResourceUsageEmulators

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2482//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2482//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2482//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532717/MAPREDUCE-4320.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 javadoc. The javadoc tool appears to have generated 13 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-tools/hadoop-gridmix: org.apache.hadoop.mapred.gridmix.TestResourceUsageEmulators +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2482//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2482//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2482//console This message is automatically generated.
        Hide
        Thomas Graves added a comment -

        rekicking jenkins.

        Show
        Thomas Graves added a comment - rekicking jenkins.
        Hide
        Ravi Prakash added a comment -

        +1 lgtm

        Show
        Ravi Prakash added a comment - +1 lgtm
        Hide
        Thomas Graves added a comment -

        The findbugs are known issue - see MAPREDUCE-4239

        Show
        Thomas Graves added a comment - The findbugs are known issue - see MAPREDUCE-4239
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12531151/MAPREDUCE-4320.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-gridmix.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2441//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2441//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2441//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531151/MAPREDUCE-4320.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-gridmix. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2441//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2441//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2441//console This message is automatically generated.
        Hide
        Thomas Graves added a comment -

        this is what you get right now:

        hadoop jar ./hadoop-0.23.3-SNAPSHOT/share/hadoop/tools/lib/hadoop-gridmix-0.23.3-SNAPSHOT.jar
        12/06/06 21:09:12 WARN mapred.JobConf: The variable mapred.child.ulimit is no longer used.
        12/06/06 21:09:12 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
        archive -archiveName NAME -p <parent path> <src>* <dest>

        Invalid usage.

        with fix it outputs gridmix usage:
        hadoop jar ./hadoop-0.23.3-SNAPSHOT/share/hadoop/tools/lib/hadoop-gridmix-0.23.3-SNAPSHOT.jar
        Generic options supported are
        -conf <configuration file> specify an application configuration file
        -D <property=value> use value for given property
        -fs <local|namenode:port> specify a namenode
        -jt <local|jobtracker:port> specify a job tracker
        -files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster
        -libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
        -archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.

        The general command line syntax is
        bin/hadoop command [genericOptions] [commandOptions]

        Usage: gridmix [-generate <MiB>] [-users URI] [-Dname=value ...] <iopath> <trace>
        e.g. gridmix -generate 100m foo -
        Options:
        -generate <MiB> : Generate input data of size MiB under <iopath>/input/ and generate
        distributed cache data under <iopath>/distributedCache/.
        -users <usersResourceURI> : URI that contains the users list.
        Configuration parameters:
        General parameters:
        gridmix.output.directory : Output directory
        gridmix.client.submit.threads : Submitting threads
        gridmix.client.pending.queue.depth : Queued job desc
        gridmix.user.resolve.class : User resolution class
        gridmix.job.type : Job types (LOADJOB|SLEEPJOB)
        Parameters related to job submission:
        gridmix.job-submission.default-queue : Default queue
        gridmix.job-submission.use-queue-in-trace : Enable/disable using queues in trace
        gridmix.job-submission.policy : Job submission policy (REPLAY|STRESS|SERIAL)
        Parameters specific for LOADJOB:
        gridmix.key.fraction : Key fraction of rec
        Parameters specific for SLEEPJOB:
        gridmix.sleep.maptask-only : Whether to ignore reduce tasks
        gridmix.sleep.fake-locations : Number of fake locations for map tasks
        gridmix.sleep.max-map-time : Maximum map task runtime in mili-sec
        gridmix.sleep.max-reduce-time : Maximum reduce task runtime in mili-sec (merge+reduce)
        Parameters specific for STRESS submission throttling policy:
        gridmix.throttle.jobs-to-tracker-ratio : jobs vs task-tracker ratio
        gridmix.throttle.maps.task-to-slot-ratio : maps vs map-slot ratio
        gridmix.throttle.reduces.task-to-slot-ratio : reduces vs reduce-slot ratio
        gridmix.throttle.maps.max-slot-share-per-job : map-slot share per job
        gridmix.throttle.reducess.max-slot-share-per-job : reduce-slot share per job

        Show
        Thomas Graves added a comment - this is what you get right now: hadoop jar ./hadoop-0.23.3-SNAPSHOT/share/hadoop/tools/lib/hadoop-gridmix-0.23.3-SNAPSHOT.jar 12/06/06 21:09:12 WARN mapred.JobConf: The variable mapred.child.ulimit is no longer used. 12/06/06 21:09:12 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used archive -archiveName NAME -p <parent path> <src>* <dest> Invalid usage. with fix it outputs gridmix usage: hadoop jar ./hadoop-0.23.3-SNAPSHOT/share/hadoop/tools/lib/hadoop-gridmix-0.23.3-SNAPSHOT.jar Generic options supported are -conf <configuration file> specify an application configuration file -D <property=value> use value for given property -fs <local|namenode:port> specify a namenode -jt <local|jobtracker:port> specify a job tracker -files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster -libjars <comma separated list of jars> specify comma separated jar files to include in the classpath. -archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines. The general command line syntax is bin/hadoop command [genericOptions] [commandOptions] Usage: gridmix [-generate <MiB>] [-users URI] [-Dname=value ...] <iopath> <trace> e.g. gridmix -generate 100m foo - Options: -generate <MiB> : Generate input data of size MiB under <iopath>/input/ and generate distributed cache data under <iopath>/distributedCache/. -users <usersResourceURI> : URI that contains the users list. Configuration parameters: General parameters: gridmix.output.directory : Output directory gridmix.client.submit.threads : Submitting threads gridmix.client.pending.queue.depth : Queued job desc gridmix.user.resolve.class : User resolution class gridmix.job.type : Job types (LOADJOB|SLEEPJOB) Parameters related to job submission: gridmix.job-submission.default-queue : Default queue gridmix.job-submission.use-queue-in-trace : Enable/disable using queues in trace gridmix.job-submission.policy : Job submission policy (REPLAY|STRESS|SERIAL) Parameters specific for LOADJOB: gridmix.key.fraction : Key fraction of rec Parameters specific for SLEEPJOB: gridmix.sleep.maptask-only : Whether to ignore reduce tasks gridmix.sleep.fake-locations : Number of fake locations for map tasks gridmix.sleep.max-map-time : Maximum map task runtime in mili-sec gridmix.sleep.max-reduce-time : Maximum reduce task runtime in mili-sec (merge+reduce) Parameters specific for STRESS submission throttling policy: gridmix.throttle.jobs-to-tracker-ratio : jobs vs task-tracker ratio gridmix.throttle.maps.task-to-slot-ratio : maps vs map-slot ratio gridmix.throttle.reduces.task-to-slot-ratio : reduces vs reduce-slot ratio gridmix.throttle.maps.max-slot-share-per-job : map-slot share per job gridmix.throttle.reducess.max-slot-share-per-job : reduce-slot share per job

          People

          • Assignee:
            Thomas Graves
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development