[#HADOOP-3770] improve composition, submission and result collection of gridmix

Hadoop Common

improve composition, submission and result collection of gridmix

Created: 16/Jul/08 01:29 AM Updated: 23/Apr/09 07:17 PM

Return to search

Component/s:

benchmarks

Affects Version/s:

None

Fix Version/s:

0.20.0

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
patch-3770.txt	2008-07-29 06:45 PM	Lingyun Yang	178 kB
patch-3770.txt	2008-07-16 01:59 AM	Lingyun Yang	175 kB
patch-3770.v2.txt	2008-11-26 09:04 PM	Runping Qi	104 kB

Issue Links:

Blocker

This issue is blocked by:

~~HADOOP-3769~~ expose static SampleMapper and SampleReducer classes of GenericMRLoadGenerator class for gridmix reuse

Duplicate

This issue is duplicated by:

~~HADOOP-4627~~ gridmix version 2

Hadoop Flags:	Reviewed
Resolution Date:	02/Dec/08 07:03 AM

Description

« Hide

Current gridmix submits jobs using a set of scripts, which is inconvenient and the results are difficult to collect. To improve the gridmix submission and results collection, we implemented a new program using JobControl to submit and collect the results of jobs
Also the new gridmix allows to have more different types of jobs such as, pig jobs, jobs with combiners etc.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Repository	Revision	Date	User	Message
ASF	#722391	Tue Dec 02 07:03:09 UTC 2008	cdouglas	~~HADOOP-3770~~. Add gridmix2, an iteration on the gridmix benchmark. Contributed by Runping Qi.
				Files Changed
				ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java ADD /hadoop/core/trunk/src/benchmarks/gridmix2/build.xml ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred/GridMixRunner.java ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java/org/apache ADD /hadoop/core/trunk/src/benchmarks/gridmix2/gridmix-env-2 ADD /hadoop/core/trunk/src/benchmarks/gridmix2/README.gridmix2 ADD /hadoop/core/trunk/src/benchmarks/gridmix2/gridmix_config.xml ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred/SortJobCreator.java ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java/org/apache/hadoop ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred/GridMixConfig.java ADD /hadoop/core/trunk/src/benchmarks/gridmix2/generateGridmix2data.sh ADD /hadoop/core/trunk/src/benchmarks/gridmix2/rungridmix_2 ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred/CombinerJobCreator.java MODIFY /hadoop/core/trunk/CHANGES.txt ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java/org ADD /hadoop/core/trunk/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapred/GenericMRLoadJobCreator.java MODIFY /hadoop/core/trunk/src/contrib/streaming/src/java/org/apache/hadoop/streaming/StreamJob.java ADD /hadoop/core/trunk/src/benchmarks/gridmix2