Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 0.23.4
    • Component/s: benchmarks, test
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      TestDFSIO measures throughput of HDFS write, read, and append operations. It will be useful to have an option to use it for benchmarking random reads.

      1. randomDFSIO.patch
        32 kB
        Konstantin Shvachko
      2. randomDFSIO.patch
        32 kB
        Konstantin Shvachko
      3. randomDFSIO.patch
        32 kB
        Konstantin Shvachko

        Activity

        Konstantin Shvachko created issue -
        Konstantin Shvachko made changes -
        Field Original Value New Value
        Assignee Konstantin Shvachko [ shv ]
        Hide
        Konstantin Shvachko added a comment -

        The idea is to utilize HDFS positional read, which is defined by PositionedReadable and allows to read a segment of data from a given position.
        I propose three variants of such benchmarks:

        1. Random read. Randomly choose an offset in the range [0, fileSize] and read one buffer of data from that random position. Repeat operation until a specified number of bytes is read.
          Random read can occasionally read the same bytes twice.
        2. Backward read reads file in reverse order.
          This is intended to read all bytes of the given file, but avoid reading any of them twice.
        3. Skip read. Starting from the beginning read one buffer of data, then jump ahead, and read again. Repeat until either the specified number of bytes is read or the end of file is reached.
          Skip read allows to avoid read-ahead. With sequential read data mostly comes from the system block cache. Jumping ahead far enough will ensure that bytes are actually read from the storage device.
        Show
        Konstantin Shvachko added a comment - The idea is to utilize HDFS positional read, which is defined by PositionedReadable and allows to read a segment of data from a given position. I propose three variants of such benchmarks: Random read . Randomly choose an offset in the range [0, fileSize] and read one buffer of data from that random position. Repeat operation until a specified number of bytes is read. Random read can occasionally read the same bytes twice. Backward read reads file in reverse order. This is intended to read all bytes of the given file, but avoid reading any of them twice. Skip read . Starting from the beginning read one buffer of data, then jump ahead, and read again. Repeat until either the specified number of bytes is read or the end of file is reached. Skip read allows to avoid read-ahead. With sequential read data mostly comes from the system block cache. Jumping ahead far enough will ensure that bytes are actually read from the storage device.
        Hide
        Konstantin Shvachko added a comment -

        The patch

        1. Introduces the three types of random reads.
        2. It also adds getIOStream() method, which excludes stream construction from the timed part of the execution. This is important for small writes and reads.
          And this let me move all compression functionality from IOMapperBase to TestDFSIO, where it truly belongs.
        3. Converted to JUnit 4 format. Finally. And added test cases for new benchmarks.
        4. Fixed couple of warnings and removed unnecessary generic parameters.
        Show
        Konstantin Shvachko added a comment - The patch Introduces the three types of random reads. It also adds getIOStream() method, which excludes stream construction from the timed part of the execution. This is important for small writes and reads. And this let me move all compression functionality from IOMapperBase to TestDFSIO, where it truly belongs. Converted to JUnit 4 format. Finally. And added test cases for new benchmarks. Fixed couple of warnings and removed unnecessary generic parameters.
        Konstantin Shvachko made changes -
        Attachment randomDFSIO.patch [ 12544790 ]
        Hide
        Ravi Prakash added a comment -

        Hi Konstantin,

        Thanks for this initiative. I like the idea of benchmarking random reads. Some comments:
        1. Why not label IOMapperBase.getIOStream() abstract rather than return null?
        2. Some extra whitespaces.
        3. TestDFSIO:doIO, @Override //IOMapperBase
        4. In doIO(), would it make sense to do?
        if( this.stream instanceof InputStream) InputStream in = (InputStream)this.stream;
        Similarly for PositionedReadable
        5. public RandomReadMapper() you can use new Random(), to seed it with a distinct seed. You don't need a call to System.nanoTime().

        Oh, and could you please review MAPREDUCE-4645? =D

        Show
        Ravi Prakash added a comment - Hi Konstantin, Thanks for this initiative. I like the idea of benchmarking random reads. Some comments: 1. Why not label IOMapperBase.getIOStream() abstract rather than return null? 2. Some extra whitespaces. 3. TestDFSIO:doIO, @Override //IOMapperBase 4. In doIO(), would it make sense to do? if( this.stream instanceof InputStream) InputStream in = (InputStream)this.stream; Similarly for PositionedReadable 5. public RandomReadMapper() you can use new Random(), to seed it with a distinct seed. You don't need a call to System.nanoTime(). Oh, and could you please review MAPREDUCE-4645 ? =D
        Hide
        Konstantin Shvachko added a comment -

        (1) I did want to make IOMapperBase.getIOStream() abstract, but there are other tests that are based on IOMapperBase. A dummy implementation of getIOStream() will avoid changing them.
        (2) Do you want me to add spaces or you see unnecessary spaces in my patch?
        (3) So you suggest to add comments to @Override specifying the base class, right? Agreed, should have done that for the new method, will also do it for doIO().
        (4) Sure you can check the instance before casting, but what would you do in the else clause - throw exception. So one way or another an exception will be thrown saying the type of stream is not right. And this is sort of an assert, because it means there is a bug in DFSIO, not user's fault.
        (5) Yep, you are right nanoTime() is already in the default constructor. I believe it wasn't there before.

        Thanks for the review. I'll check SLive jira.

        Show
        Konstantin Shvachko added a comment - (1) I did want to make IOMapperBase.getIOStream() abstract, but there are other tests that are based on IOMapperBase. A dummy implementation of getIOStream() will avoid changing them. (2) Do you want me to add spaces or you see unnecessary spaces in my patch? (3) So you suggest to add comments to @Override specifying the base class, right? Agreed, should have done that for the new method, will also do it for doIO(). (4) Sure you can check the instance before casting, but what would you do in the else clause - throw exception. So one way or another an exception will be thrown saying the type of stream is not right. And this is sort of an assert, because it means there is a bug in DFSIO, not user's fault. (5) Yep, you are right nanoTime() is already in the default constructor. I believe it wasn't there before. Thanks for the review. I'll check SLive jira.
        Hide
        Konstantin Shvachko added a comment -

        Removed nanoTime(), added comments to @Override.

        Show
        Konstantin Shvachko added a comment - Removed nanoTime(), added comments to @Override.
        Konstantin Shvachko made changes -
        Attachment randomDFSIO.patch [ 12545095 ]
        Hide
        Ravi Prakash added a comment -

        Thanks Konstantin! I applied the patch and ran the random and backward read tests on my single node dev box.

        $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random -fileSize 10MB 
        Average IO rate mb/sec: 134.43310546875
        IO rate std deviation: 0.00896365222201456
        
        $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward -fileSize 10MB
        Average IO rate mb/sec: 134.49253845214844
        IO rate std deviation: 0.026679629420752023
        
        $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random -fileSize 1GB
        Average IO rate mb/sec: 249.47183227539062
        IO rate std deviation: 0.014617091655162118
        
        $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward -fileSize 1GB
        Average IO rate mb/sec: 295.8538818359375
        IO rate std deviation: 0.061419808441541615
        
        $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random -fileSize 10GB
        Average IO rate mb/sec: 320.3417663574219
        IO rate std deviation: 0.05935480659067817
        
        $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward -fileSize 10GB
        Average IO rate mb/sec: 323.28045654296875
        IO rate std deviation: 0.0598550775330073
        
        $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward -fileSize 30GB
        Average IO rate mb/sec: 390.9880065917969
        IO rate std deviation: 0.06083891027478396
        
        $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random -fileSize 30GB
        Average IO rate mb/sec: 369.2136535644531
        IO rate std deviation: 0.056819116587427144
        

        Could you please post recommended usage? And at what sizes do we expect to achieve stable IO rates?

        Show
        Ravi Prakash added a comment - Thanks Konstantin! I applied the patch and ran the random and backward read tests on my single node dev box. $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random -fileSize 10MB Average IO rate mb/sec: 134.43310546875 IO rate std deviation: 0.00896365222201456 $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward -fileSize 10MB Average IO rate mb/sec: 134.49253845214844 IO rate std deviation: 0.026679629420752023 $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random -fileSize 1GB Average IO rate mb/sec: 249.47183227539062 IO rate std deviation: 0.014617091655162118 $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward -fileSize 1GB Average IO rate mb/sec: 295.8538818359375 IO rate std deviation: 0.061419808441541615 $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random -fileSize 10GB Average IO rate mb/sec: 320.3417663574219 IO rate std deviation: 0.05935480659067817 $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward -fileSize 10GB Average IO rate mb/sec: 323.28045654296875 IO rate std deviation: 0.0598550775330073 $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward -fileSize 30GB Average IO rate mb/sec: 390.9880065917969 IO rate std deviation: 0.06083891027478396 $HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random -fileSize 30GB Average IO rate mb/sec: 369.2136535644531 IO rate std deviation: 0.056819116587427144 Could you please post recommended usage? And at what sizes do we expect to achieve stable IO rates?
        Hide
        Jakob Homan added a comment -

        +1. Good refactoring as well. nit: bit of wonky spacing in TestDFSIO::AppendMapper::getIOStream.

        Show
        Jakob Homan added a comment - +1. Good refactoring as well. nit: bit of wonky spacing in TestDFSIO::AppendMapper::getIOStream.
        Hide
        Konstantin Shvachko added a comment -

        Fixed wonky space in AppendMapper.getIOStream()

        Show
        Konstantin Shvachko added a comment - Fixed wonky space in AppendMapper.getIOStream()
        Konstantin Shvachko made changes -
        Attachment randomDFSIO.patch [ 12546568 ]
        Konstantin Shvachko made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Konstantin Shvachko added a comment -

        Ravi,

        • even with single node you can specify more -nrFiles if you have multiple drives on the node. I usually setup number of map slots equal to the number of drives on a node.
        • I don't know how big was the file that you created with -write prior to reads. If it was 10 MB than the actual size of reads was not more than that. Check the DFSIO summary it prints how much data was read.
        • You probably ran reads right after creating the file. So the the data was in buffer cache. I usually clean the cache before each test run. (On linux 'echo 1 > /proc/sys/vm/drop_caches')
        • Also -fileSize is replaced by -size in my patch. It says how much data you want to read/write/append, rather than specifying the size of a file. Initially (read/write) it was the same.
        Show
        Konstantin Shvachko added a comment - Ravi, even with single node you can specify more -nrFiles if you have multiple drives on the node. I usually setup number of map slots equal to the number of drives on a node. I don't know how big was the file that you created with -write prior to reads. If it was 10 MB than the actual size of reads was not more than that. Check the DFSIO summary it prints how much data was read. You probably ran reads right after creating the file. So the the data was in buffer cache. I usually clean the cache before each test run. (On linux 'echo 1 > /proc/sys/vm/drop_caches') Also -fileSize is replaced by -size in my patch. It says how much data you want to read/write/append, rather than specifying the size of a file. Initially (read/write) it was the same.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12546568/randomDFSIO.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2878//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2878//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12546568/randomDFSIO.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2878//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2878//console This message is automatically generated.
        Hide
        Konstantin Shvachko added a comment -

        I just committed this to trunk, branch-2, and branch 0.23.4

        Show
        Konstantin Shvachko added a comment - I just committed this to trunk, branch-2, and branch 0.23.4
        Konstantin Shvachko made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 0.23.4 [ 12323264 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2835 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2835/)
        MAPREDUCE-4651. Benchmarking random reads with DFSIO. (Revision 1390159)

        Result = SUCCESS
        shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2835 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2835/ ) MAPREDUCE-4651 . Benchmarking random reads with DFSIO. (Revision 1390159) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2772 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2772/)
        MAPREDUCE-4651. Benchmarking random reads with DFSIO. (Revision 1390159)

        Result = SUCCESS
        shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2772 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2772/ ) MAPREDUCE-4651 . Benchmarking random reads with DFSIO. (Revision 1390159) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2793 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2793/)
        MAPREDUCE-4651. Benchmarking random reads with DFSIO. (Revision 1390159)

        Result = FAILURE
        shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2793 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2793/ ) MAPREDUCE-4651 . Benchmarking random reads with DFSIO. (Revision 1390159) Result = FAILURE shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #386 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/386/)
        MAPREDUCE-4651. Benchmarking random reads with DFSIO. Contributed by Konstantin Shvachko. (Revision 1390164)

        Result = UNSTABLE
        shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390164
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #386 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/386/ ) MAPREDUCE-4651 . Benchmarking random reads with DFSIO. Contributed by Konstantin Shvachko. (Revision 1390164) Result = UNSTABLE shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390164 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1177 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1177/)
        MAPREDUCE-4651. Benchmarking random reads with DFSIO. (Revision 1390159)

        Result = SUCCESS
        shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1177 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1177/ ) MAPREDUCE-4651 . Benchmarking random reads with DFSIO. (Revision 1390159) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1208 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1208/)
        MAPREDUCE-4651. Benchmarking random reads with DFSIO. (Revision 1390159)

        Result = SUCCESS
        shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1208 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1208/ ) MAPREDUCE-4651 . Benchmarking random reads with DFSIO. (Revision 1390159) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1390159 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/IOMapperBase.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Hi Konstantin, the patch no longer applies to trunk. Could you update it?

        Show
        Tsz Wo Nicholas Sze added a comment - Hi Konstantin, the patch no longer applies to trunk. Could you update it?
        Hide
        Konstantin Shvachko added a comment -

        That is because I committed it. Jakob reviewed.

        Show
        Konstantin Shvachko added a comment - That is because I committed it. Jakob reviewed.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Oops, I forgot to reload the page. It is great that Jakob has reviewed it.

        Show
        Tsz Wo Nicholas Sze added a comment - Oops, I forgot to reload the page. It is great that Jakob has reviewed it.

          People

          • Assignee:
            Konstantin Shvachko
            Reporter:
            Konstantin Shvachko
          • Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development