DFS i/o benchmark is a map-reduce based test that measures performance of the cluster for reads and writes.
This is an evolved version of
HADOOP-72, and HADOOP-95 test.
This test writes into or reads from a specified number of files.
File size is specified as a parameter to the test.
Each file is processed in a separate map task.
The unique reducer then collects stats.
Finally, the following information is displayed
- read or write test
- date and time the test finished
- number of files processed
- total number of bytes processed
- throughput in mb/sec (total number of bytes / sum of processing times)
- average i/o rate in mb/sec per file
- standard i/o rate deviation
I included the test into the AllTestDriver.