Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3032

Lease renewer tries forever even if renewal is not possible

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.23.0, 0.23.1, 2.0.0-alpha
    • 0.23.2
    • hdfs-client
    • None
    • Reviewed

    Description

      When LeaseRenewer gets an IOException while attempting to renew for a client, it retries after sleeping 500ms. If the exception is caused by a condition that will never change, it keeps talking to the name node until the DFSClient object is closed or aborted. With the FileSystem cache, a DFSClient can stay alive for very long time. We've seen the cases in which node managers and long living jobs flooding name node with this type of calls.

      The current proposal is to abort the client when RemoteException is caught during renewal. LeaseRenewer already does abort on all clients when it sees a SocketTimeoutException.

      Attachments

        1. h3032_20120205.patch
          13 kB
          Tsz-wo Sze
        2. hdfs-3032.patch.txt
          13 kB
          Kihwal Lee
        3. hdfs-3032.patch.txt
          7 kB
          Kihwal Lee
        4. hdfs-3032.patch.txt
          8 kB
          Kihwal Lee
        5. hdfs-3032.patch.txt
          6 kB
          Kihwal Lee

        Activity

          People

            kihwal Kihwal Lee
            kihwal Kihwal Lee
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: