[HDFS-6543] TestWebHDFS#testLargeFile fails intermittently

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.4.0
• Component/s: webhdfs
• Assignee: Yongjun Zhang
• Reporter: Yongjun Zhang
• Created: 16/Jun/14 16:45
• Updated: 27/Oct/20 16:03

Description

Running latest trunk locally, I'm seeing the following failure. Running it several times, some fail, most are successful.

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.hdfs.web.TestWebHDFS
Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 119.42 sec <<< FAILURE! - in org.apache.hadoop.hdfs.web.TestWebHDFS
testLargeFile(org.apache.hadoop.hdfs.web.TestWebHDFS)  Time elapsed: 26.415 sec  <<< ERROR!
java.io.IOException: File /test/largeFile/file could only be replicated to 0 nodes instead of minReplication (=1).  There are 3 datanode(s) running and no node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1468)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2725)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:611)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:455)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:605)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)

        at org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:163)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:312)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:86)
        at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$FsPathOutputStreamRunner$1.close(WebHdfsFileSystem.java:708)
        at org.apache.hadoop.hdfs.web.TestWebHDFS.largeFileTest(TestWebHDFS.java:134)
        at org.apache.hadoop.hdfs.web.TestWebHDFS.testLargeFile(TestWebHDFS.java:97)