Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
0.20-append
-
None
-
None
-
An HDFS cluster of 9 nodes (same rack; 1 rack = 40 nodes to one rack switch) was deployed on Yahoo!'s R&D grid cluster. It comprised of 1 Namenode, 1 JobTracker and 7 Datanodes. client assigned to a separate compute node not part of this HDFS. The characteristics of the testbed are as follows:
• Hadoop 0.20.1xx - latest Yahoo! Hadoop Security version
• OS - GNU/Linux x86_64 kernel version 2.6.18
• Java 1.6
• Processor - 2 SMP Quad-core Intel Xeon @ 2.5Ghz per node
• Memory - 16GB RAM per nodeAn HDFS cluster of 9 nodes (same rack; 1 rack = 40 nodes to one rack switch) was deployed on Yahoo!'s R&D grid cluster. It comprised of 1 Namenode, 1 JobTracker and 7 Datanodes. client assigned to a separate compute node not part of this HDFS. The characteristics of the testbed are as follows: • Hadoop 0.20.1xx - latest Yahoo! Hadoop Security version • OS - GNU/Linux x86_64 kernel version 2.6.18 • Java 1.6 • Processor - 2 SMP Quad-core Intel Xeon @ 2.5Ghz per node • Memory - 16GB RAM per node
-
HDFS client, latency, benchmark, performance, statistics
Description
JAR to measure HDFS client latencies. Runs a process comprising of HDFS operations with helper scripts to specify number of iterations, number of concurrent instances and base directory. Compiles statistics in 2 text files inside base directory which can be read/manipulated using data charts applications (e.g. MS Excel).
