Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4161

HDFS keeps a thread open for every file writer

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.1
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In 1.0 release DFSClient uses a thread per file writer. In some use cases (dynamic partions in hive) that use a large number of file writers a large number of threads are created. The file writer thread has the following stack:

      at java.lang.Thread.sleep(Native Method)
      at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.run(DFSClient.java:1462)
      at java.lang.Thread.run(Thread.java:662)
      

      This problem has been fixed in later releases. This jira will post a consolidated patch from various jiras that addresses the issue.

      1. h4161_20121108_b-1.patch
        41 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Suresh Srinivas
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development