[HDFS-9744] TestDirectoryScanner#testThrottling occasionally time out after 300 seconds

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: datanode
• Labels: test

Environment

Jenkins

• Hadoop Flags: Reviewed
• Assignee: Yiqun Lin
• Reporter: Wei-Chiu Chuang
• Created: 02/Feb/16 21:02
• Updated: 02/Oct/19 17:14
• Resolved: 19/Apr/16 11:00

Description

I have seen quite a few test failures in TestDirectoryScanner#testThrottling.
https://builds.apache.org/job/Hadoop-Hdfs-trunk/2793/testReport/org.apache.hadoop.hdfs.server.datanode/TestDirectoryScanner/testThrottling/

Looking at the log, it does not look like the test got stucked. On my local machine, this test took 219 seconds. It is likely that this test takes more than 300 seconds to complete on a busy jenkins slave. I think it is reasonable to set a longer time out value, or reduce the number of blocks to reduce the duration of the test.

Error Message

test timed out after 300000 milliseconds

Stacktrace

java.lang.Exception: test timed out after 300000 milliseconds
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:503)
	at org.apache.hadoop.hdfs.DataStreamer.waitAndQueuePacket(DataStreamer.java:804)
	at org.apache.hadoop.hdfs.DFSOutputStream.enqueueCurrentPacket(DFSOutputStream.java:423)
	at org.apache.hadoop.hdfs.DFSOutputStream.enqueueCurrentPacketFull(DFSOutputStream.java:432)
	at org.apache.hadoop.hdfs.DFSOutputStream.writeChunk(DFSOutputStream.java:418)
	at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:217)
	at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:125)
	at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:111)
	at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:57)
	at java.io.DataOutputStream.write(DataOutputStream.java:107)
	at org.apache.hadoop.hdfs.DFSTestUtil.createFile(DFSTestUtil.java:418)
	at org.apache.hadoop.hdfs.DFSTestUtil.createFile(DFSTestUtil.java:376)
	at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.createFile(TestDirectoryScanner.java:108)
	at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:584)