[MAPREDUCE-2425] Distributed simulator for stressing JobTracker and NameNode - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Resolved
Priority: Major
Resolution: Won't Fix
Affects Version/s: None
Fix Version/s: None
Component/s: contrib/mumak
Labels:
- benchmark
- hadoop

Description

Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a simulated JobTracker, whose behavior doesn't exactly like that of the real JobTracker. Even more, mumak can't simulate a large cluster with quite a lot of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical nodes to replay job stories.

You can think this tool a complementation of mumak and gridmix v3. We successfully used this tool to simulate a 12000 nodes cluster through 4 real machines.
I've talk to Hong Tang and Scott Chen offline, they suggested me contributing this tool to the hadoop community.