Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4181

LeaseManager tries to double remove and prints extra messages

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.4, 2.0.2-alpha
    • Fix Version/s: 2.0.3-alpha, 0.23.5
    • Component/s: namenode
    • Labels:
      None

      Description

      When checkLeases() runs, internalReleaseLease() is called on the expired ones. When it returns true, the lease is already removed, yet it is tried again in checkLease(). This causes unnecessary ERROR messages to be logged. The line doing removing.add(p) should be removed.

      The internalReleaseLease() method logs a detailed message per call, so the extra INFO log message from checkLease() is redundant.

      The error message from removeLease() can be very big and needs to be cut down. When the namenode itself is holding a lot of leases for block recovery, hitting this error is very expensive. In one instance, slow block recovery caused the namenode to hold more than 42K leases. The one log line in this case was over 4 MB. The dump of data structure should be only enabled in debug mode.

        Attachments

        1. hdfs-4181-trunk.patch
          2 kB
          Kihwal Lee
        2. hdfs-4181-trunk.patch
          2 kB
          Kihwal Lee
        3. hdfs-4181-branch-0.23.patch
          2 kB
          Kihwal Lee
        4. hdfs-4181-branch-0.23.patch
          2 kB
          Kihwal Lee

          Activity

            People

            • Assignee:
              kihwal Kihwal Lee
              Reporter:
              kihwal Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: