Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13065

TestErasureCodingMultipleRacks#testSkewedRack3 is failing

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.1.0, 3.0.1
    • Component/s: hdfs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Test fails intermittently.

      java.lang.AssertionError: expected:<9> but was:<7>
      at org.apache.hadoop.hdfs.TestErasureCodingMultipleRacks.testSkewedRack3(TestErasureCodingMultipleRacks.java:178)

      2017-12-12 20:25:46,751 [IPC Server handler 6 on 47378] DEBUG net.NetworkTopology (DFSNetworkTopology.java:chooseRandomWithStorageTypeTwoTrial(119)) - No node to choose.
      2017-12-12 20:25:46,751 [IPC Server handler 6 on 47378] WARN blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyRackFaultTolerant.java:chooseTargetInOrder(142)) - Only able to place 7 of total expected 9 (maxNodesPerRack=2, numOfReplicas=4) nodes evenly across racks, falling back to evenly place on the remaining racks. This may not guarantee rack-level fault tolerance. Please check if the racks are configured properly.
      2017-12-12 20:25:46,751 [IPC Server handler 6 on 47378] DEBUG blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyRackFaultTolerant.java:chooseTargetInOrder(148)) - Caught exception was:
      org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy$NotEnoughReplicasException: 
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseRandom(BlockPlacementPolicyDefault.java:792)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyRackFaultTolerant.chooseOnce(BlockPlacementPolicyRackFaultTolerant.java:224)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyRackFaultTolerant.chooseTargetInOrder(BlockPlacementPolicyRackFaultTolerant.java:139)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:392)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:268)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:121)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:137)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:2091)
      at org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.chooseTargetForNewBlock(FSDirWriteFileOp.java:287)
      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2602)
      at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:864)
      at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:549)
      at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
      at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
      at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:422)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)

        Attachments

        1. HDFS-13065.003.patch
          2 kB
          Gabor Bota
        2. HDFS-13065.002.patch
          2 kB
          Gabor Bota
        3. HDFS-13065.001.patch
          2 kB
          Gabor Bota

          Activity

            People

            • Assignee:
              gabor.bota Gabor Bota
              Reporter:
              gabor.bota Gabor Bota
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved: