Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15646 Track failing tests in HDFS
  3. HDFS-6543

TestWebHDFS#testLargeFile fails intermittently

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.4.0
    • None
    • webhdfs
    • None

    Description

      Running latest trunk locally, I'm seeing the following failure. Running it several times, some fail, most are successful.

      -------------------------------------------------------
       T E S T S
      -------------------------------------------------------
      Running org.apache.hadoop.hdfs.web.TestWebHDFS
      Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 119.42 sec <<< FAILURE! - in org.apache.hadoop.hdfs.web.TestWebHDFS
      testLargeFile(org.apache.hadoop.hdfs.web.TestWebHDFS)  Time elapsed: 26.415 sec  <<< ERROR!
      java.io.IOException: File /test/largeFile/file could only be replicated to 0 nodes instead of minReplication (=1).  There are 3 datanode(s) running and no node(s) are excluded in this operation.
              at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1468)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2725)
              at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:611)
              at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:455)
              at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
              at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:605)
              at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2099)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2095)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:415)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
              at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2093)
      
              at org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:163)
              at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:312)
              at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:86)
              at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$FsPathOutputStreamRunner$1.close(WebHdfsFileSystem.java:708)
              at org.apache.hadoop.hdfs.web.TestWebHDFS.largeFileTest(TestWebHDFS.java:134)
              at org.apache.hadoop.hdfs.web.TestWebHDFS.testLargeFile(TestWebHDFS.java:97)
      

      Attachments

        Activity

          People

            yzhangal Yongjun Zhang
            yzhangal Yongjun Zhang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: