[HADOOP-193] DFS i/o benchmark.

• Type: Test
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.2.0
• Component/s: fs
• Assignee: Konstantin Shvachko
• Reporter: Konstantin Shvachko
• Created: 04/May/06 08:57
• Updated: 10/Aug/17 18:20
• Resolved: 04/May/06 11:39

Description

DFS i/o benchmark is a map-reduce based test that measures performance of the cluster for reads and writes.
This is an evolved version of HADOOP-72, and HADOOP-95 test.

This test writes into or reads from a specified number of files.
File size is specified as a parameter to the test.
Each file is processed in a separate map task.
The unique reducer then collects stats.
Finally, the following information is displayed

1. read or write test
2. date and time the test finished
3. number of files processed
4. total number of bytes processed
5. throughput in mb/sec (total number of bytes / sum of processing times)
6. average i/o rate in mb/sec per file
7. standard i/o rate deviation

I included the test into the AllTestDriver.

Issue Links

• is related to: HDFS-708 A stress-test tool for HDFS.