[SPARK-23958] HadoopRdd filters empty files to avoid generating empty tasks that affect the performance of the Spark computing performance.

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 2.4.0
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: guoxiaolong
• Created: 11/Apr/18 02:40
• Updated: 20/Sep/18 17:51
• Resolved: 12/Apr/18 02:41

Description

HadoopRdd filter empty files to avoid generating empty tasks that affect the performance of the Spark computing performance.

Empty file's length is zero.

Issue Links

• links to: [Github] Pull Request #21036 (guoxiaolongzte)