[HADOOP-7754] Expose file descriptors from Hadoop-wrapped local FileSystems - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 0.23.0
Fix Version/s: 1.2.0, 2.0.2-alpha
Component/s: native, performance
Labels:
None

Target Version/s:

0.23.0
Hadoop Flags:

Reviewed

Description

In HADOOP-7714, we determined that using fadvise inside of the MapReduce shuffle can yield very good performance improvements. But many parts of the shuffle are FileSystem-agnostic and thus operate on FSDataInputStreams and RawLocalFileSystems. This JIRA is to figure out how to allow RawLocalFileSystem to expose its FileDescriptor object without unnecessarily polluting the public APIs.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

hasfd.txt
18/Oct/11 02:44
4 kB
Todd Lipcon
hadoop-7754-0.23.0-hasfd.txt
20/Jun/12 16:52
4 kB
Cristina L. Abad
HADOOP-7754_trunk.patch
06/Aug/12 08:48
5 kB
Ahmed Radwan
HADOOP-7754_trunk_rev2.patch
07/Aug/12 23:36
6 kB
Ahmed Radwan
HADOOP-7754_branch-1_rev2.patch
07/Aug/12 23:36
5 kB
Ahmed Radwan
HADOOP-7754_trunk_rev2.patch
08/Aug/12 20:50
6 kB
Ahmed Radwan
HADOOP-7754_branch-1_rev3.patch
08/Aug/12 21:38
5 kB
Ahmed Radwan
HADOOP-7754_trunk_rev3.patch
08/Aug/12 21:39
6 kB
Ahmed Radwan
HADOOP-7754_branch-1_rev4.patch
13/Aug/12 19:27
6 kB
Ahmed Radwan
HADOOP-7754_trunk_rev4.patch
13/Aug/12 19:29
7 kB
Ahmed Radwan
HADOOP-7754_trunk_rev4.patch
13/Aug/12 23:42
7 kB
Ahmed Radwan
test-patch-hadoop-7754.txt
14/Aug/12 18:38
168 kB
Ahmed Radwan
HADOOP-7754_trunk_rev4.patch
15/Aug/12 00:21
7 kB
Ahmed Radwan

Issue Links

is depended upon by

MAPREDUCE-4511 Add IFile readahead

Closed

Activity

People

Assignee:: Todd Lipcon

Reporter:: Todd Lipcon

Votes:: 0 Vote for this issue

Watchers:: 16 Start watching this issue

Dates

Created:: 18/Oct/11 02:39

Updated:: 02/May/13 02:29

Resolved:: 15/Aug/12 06:43