Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6599

2.4 addBlock is 10 to 20 times slower compared to 0.23

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.3.0, 2.4.0
    • Fix Version/s: 2.5.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      From one of our busiest 0.23 clusters:

      "AddBlockAvgTime" : 0.9514711501719515
      "CreateAvgTime" : 1.7564162389174
      "CompleteAvgTime" : 1.3310406035056548
      "BlockReceivedAndDeletedAvgTime" : 0.661210005151392

      From a not-so-busy 2.4 cluster:

      "AddBlockAvgTime" : 10.083333333333334
      "CreateAvgTime" : 1.0
      "CompleteAvgTime" : 1.1111111111111112
      "BlockReceivedAndDeletedAvgTime" : 0.07692307692307694

      When the 2.4 cluster gets a moderate amount of write requests, the latency is terrible. E.g. addBlock goes upward of 60ms.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                daryn Daryn Sharp
                Reporter:
                kihwal Kihwal Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                22 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: