[HDFS-5364] Add OpenFileCtx cache - ASF JIRA

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.3.0
Component/s: nfs
Labels:
None

Hadoop Flags:

Reviewed

Description

NFS gateway can run out of memory when the stream timeout is set to a relatively long period(e.g., >1 minute) and user uploads thousands of files in parallel. Each stream DFSClient creates a DataStreamer thread, and will eventually run out of memory by creating too many threads.

NFS gateway should have a OpenFileCtx cache to limit the total opened files.

Attachments

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HDFS-5364.009.patch
07/Nov/13 20:59
43 kB
Brandon Li
HDFS-5364.008.patch
07/Nov/13 20:03
43 kB
Brandon Li
HDFS-5364.007.patch
06/Nov/13 00:29
41 kB
Brandon Li
HDFS-5364.006.patch
05/Nov/13 22:24
38 kB
Brandon Li
HDFS-5364.005.patch
05/Nov/13 01:08
37 kB
Brandon Li
HDFS-5364.004.patch
04/Nov/13 23:47
37 kB
Brandon Li
HDFS-5364.003.patch
26/Oct/13 00:06
30 kB
Brandon Li
HDFS-5364.002.patch
24/Oct/13 00:29
25 kB
Brandon Li
HDFS-5364.001.patch
22/Oct/13 17:50
19 kB
Brandon Li

Issue Links

Add Link

is duplicated by

HDFS-5172 Handle race condition for writes

Resolved

Delete this link

Activity

Comment

This comment will be Viewable by All Users Viewable by All Users

Cancel

People

Assignee:: Brandon Li

Reporter:: Brandon Li

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 15/Oct/13 05:46

Updated:: 24/Feb/14 20:57

Resolved:: 08/Nov/13 01:51

Agile

Slack

Issue deployment