[HIVE-24310] Allow specified number of deserialize errors to be ignored

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Operators
• Labels: pull-request-available
• Assignee: Zhihua Deng
• Reporter: Zhihua Deng
• Created: 24/Oct/20 02:38
• Updated: 15/Jun/21 00:10

Description

Sometimes we see some corrupted records in user's raw data,  like one corrupted in a file which contains over thousands of records, user has to either give up all records or replay the whole data in order to run successfully on hive, we should provide a way to ignore such corrupted records. 
 

Issue Links

• relates to: HIVE-733 allow specifying maximum number of serde exceptions to ignore
• links to: GitHub Pull Request #1607