[SPARK-25595] Ignore corrupt Avro file if flag IGNORE_CORRUPT_FILES enabled

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.4.6, 3.0.0
• Component/s: SQL
• Assignee: Gengliang Wang
• Reporter: Gengliang Wang
• Created: 02/Oct/18 08:32
• Updated: 12/Dec/22 18:10
• Resolved: 03/Oct/18 09:09

Description

With flag IGNORE_CORRUPT_FILES enabled, schema inference should ignore corrupt Avro files, which is consistent with Parquet and Orc data source.

Issue Links

• is depended upon by: SPARK-31546 Backport SPARK-25595 Ignore corrupt Avro file if flag IGNORE_CORRUPT_FILES enabled
• links to: [Github] Pull Request #22611 (gengliangwang)
• links to: [Github] Pull Request #22611 (gengliangwang)