Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4161

HDFS keeps a thread open for every file writer

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.1
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In 1.0 release DFSClient uses a thread per file writer. In some use cases (dynamic partions in hive) that use a large number of file writers a large number of threads are created. The file writer thread has the following stack:

      at java.lang.Thread.sleep(Native Method)
      at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.run(DFSClient.java:1462)
      at java.lang.Thread.run(Thread.java:662)
      

      This problem has been fixed in later releases. This jira will post a consolidated patch from various jiras that addresses the issue.

        Attachments

        1. h4161_20121108_b-1.patch
          41 kB
          Tsz Wo Nicholas Sze

          Issue Links

            Activity

              People

              • Assignee:
                szetszwo Tsz Wo Nicholas Sze
                Reporter:
                sureshms Suresh Srinivas
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: