[SPARK-5232] CombineFileInputFormatShim#getDirIndices is expensive

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Assignee: Unassigned
• Reporter: Jimmy Xiang
• Created: 14/Jan/15 00:26
• Updated: 14/Jan/15 00:29
• Resolved: 14/Jan/15 00:29

Description

lirui found out that we spent quite some time on CombineFileInputFormatShim#getDirIndices. Looked into it and it seems to me we should be able to get rid of this method completely if we can enhance CombineFileInputFormatShim a little.