[MAPREDUCE-7016] Avoid making separate RPC calls for FileStatus and block locations in FileInputFormat - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: None
Labels:
None

Description

FileInputFormat::getSplits uses FileSystem::globStatus to determine its inputs. When the glob returns directories, each is traversed and LocatedFileStatus instances are returned with the block locations. However, when the glob returns files, this is a FileStatus that requires a second RPC to obtain its locations.

Attachments

Activity

People

Assignee:: Unassigned

Reporter:: Christopher Douglas

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 28/Nov/17 23:54

Updated:: 29/Nov/17 06:07