Seems like this test was lengthened by HDFS-2834. I'm not sure yet whether it represents a performance regression, or if the test itself was just changed in such a way that it runs much longer.

On branch-2:

  <testcase time="9.15" classname="org.apache.hadoop.hdfs.TestParallelRead" name="testParallelRead"/>

On trunk:

  <testcase time="23.397" classname="org.apache.hadoop.hdfs.TestParallelRead" name="testParallelReadCopying"/>
  <testcase time="133.218" classname="org.apache.hadoop.hdfs.TestParallelRead" name="testParallelReadByteBuffer"/>
  <testcase time="61.364" classname="org.apache.hadoop.hdfs.TestParallelRead" name="testParallelReadMixed"/>

I also see a lot of blocked threads in the jstack on trunk. I asked Henry to take a look at this.

fwiw I tried copying the old TestParallelRead from branch-2 into trunk, and it runs just as fast there as it does in branch-2. So this seems like an issue with the new test code, rather than a regression in the read performance of the existing path.

Todd mentions that the test does eventually finish, so I don't think this is a correctness bug (thankfully). I'm looking to see how the test changes could have made this much slower.

Turns out I left N_ITERATIONS as 4096 in HDFS-2834, not 1024 as it used to be. The tests run much more quickly for me now, as you might expect, and directly comparable with the old test code run against trunk.

This does shine a light on a separate issue, which is that ByteBufferReadable lacks a lockless pread, which is faked out in this test by seeking-then-reading under a lock, with a bit of unnecessary serialisation as a result. I've opened HDSF-3246 to track this.

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522151/HDFS-3243.0.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in .

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2243//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2243//console

This message is automatically generated.

+1, verified in the test results above that TestParallelRead passed relatively quickly.

Committed to trunk, thanks

Integrated in Hadoop-Hdfs-trunk-Commit #2120 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2120/)
HDFS-3243. TestParallelRead timing out on jenkins. Contributed by Henry Robinson. (Revision 1312029)

Result = SUCCESS
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1312029
Files :

• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestParallelReadUtil.java

Integrated in Hadoop-Common-trunk-Commit #2046 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2046/)
HDFS-3243. TestParallelRead timing out on jenkins. Contributed by Henry Robinson. (Revision 1312029)

Result = SUCCESS
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1312029
Files :

• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestParallelReadUtil.java

Integrated in Hadoop-Mapreduce-trunk-Commit #2059 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2059/)
HDFS-3243. TestParallelRead timing out on jenkins. Contributed by Henry Robinson. (Revision 1312029)

Result = ABORTED
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1312029
Files :

• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestParallelReadUtil.java

Integrated in Hadoop-Hdfs-trunk #1011 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1011/)
HDFS-3243. TestParallelRead timing out on jenkins. Contributed by Henry Robinson. (Revision 1312029)

Result = FAILURE
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1312029
Files :

• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestParallelReadUtil.java

Integrated in Hadoop-Mapreduce-trunk #1046 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1046/)
HDFS-3243. TestParallelRead timing out on jenkins. Contributed by Henry Robinson. (Revision 1312029)

Result = SUCCESS
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1312029
Files :

• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
• /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestParallelReadUtil.java

Committed to branch-2 as well.