Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-925

Make it harder to accidentally close a shared DFSClient

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: 0.21.0
    • Fix Version/s: None
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      Every so often I get stack traces telling me that DFSClient is closed, usually in {{org.apache.hadoop.hdfs.DFSClient.checkOpen() }} . The root cause of this is usually that one thread has closed a shared fsclient while another thread still has a reference to it. If the other thread then asks for a new client it will get one and the cache repopulated but if has one already, then I get to see a stack trace.

      It's effectively a race condition between clients in different threads.

        Attachments

        1. HADOOP-5933.patch
          2 kB
          Steve Loughran
        2. HADOOP-5933.patch
          1 kB
          Steve Loughran
        3. HDFS-925.patch
          5 kB
          Steve Loughran
        4. HDFS-925.patch
          4 kB
          Steve Loughran
        5. HDFS-925.patch
          2 kB
          Steve Loughran
        6. HDFS-925.patch
          2 kB
          Steve Loughran

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: