Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: nfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      NFS gateway can run out of memory when the stream timeout is set to a relatively long period(e.g., >1 minute) and user uploads thousands of files in parallel. Each stream DFSClient creates a DataStreamer thread, and will eventually run out of memory by creating too many threads.

      NFS gateway should have a OpenFileCtx cache to limit the total opened files.

        Attachments

        1. HDFS-5364.001.patch
          19 kB
          Brandon Li
        2. HDFS-5364.002.patch
          25 kB
          Brandon Li
        3. HDFS-5364.003.patch
          30 kB
          Brandon Li
        4. HDFS-5364.004.patch
          37 kB
          Brandon Li
        5. HDFS-5364.005.patch
          37 kB
          Brandon Li
        6. HDFS-5364.006.patch
          38 kB
          Brandon Li
        7. HDFS-5364.007.patch
          41 kB
          Brandon Li
        8. HDFS-5364.008.patch
          43 kB
          Brandon Li
        9. HDFS-5364.009.patch
          43 kB
          Brandon Li

          Issue Links

            Activity

              People

              • Assignee:
                brandonli Brandon Li
                Reporter:
                brandonli Brandon Li
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: