Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-925

Make it harder to accidentally close a shared DFSClient

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Won't Fix
    • Affects Version/s: 0.21.0
    • Fix Version/s: None
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      Every so often I get stack traces telling me that DFSClient is closed, usually in {{org.apache.hadoop.hdfs.DFSClient.checkOpen() }} . The root cause of this is usually that one thread has closed a shared fsclient while another thread still has a reference to it. If the other thread then asks for a new client it will get one and the cache repopulated but if has one already, then I get to see a stack trace.

      It's effectively a race condition between clients in different threads.

      1. HDFS-925.patch
        5 kB
        Steve Loughran
      2. HDFS-925.patch
        4 kB
        Steve Loughran
      3. HDFS-925.patch
        2 kB
        Steve Loughran
      4. HDFS-925.patch
        2 kB
        steve_l
      5. HADOOP-5933.patch
        2 kB
        steve_l
      6. HADOOP-5933.patch
        1 kB
        steve_l

        Issue Links

          Activity

            People

            • Assignee:
              Steve Loughran
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development