[IMPALA-3764] Fuzz-test HDFS scanners

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.7.0
• Fix Version/s: Impala 2.7.0
• Component/s: Backend
• Labels: test
• Target Version: Impala 2.7.0
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 20/Jun/16 20:35
• Updated: 19/Aug/16 23:50
• Resolved: 11/Aug/16 14:08

Description

We've hit a couple of recent bugs where scanners crashed on data that was corrupted in relatively minor or predictable ways: IMPALA-3441, IMPALA-3745, IMPALA-3646

These could have all been reproduced with a truncated file or a single byte being changed.

We should do some randomised testing of HDFS scanners with simple errors like that, in order to find such bugs more proactively.

Issue Links

• is duplicated by: IMPALA-3544 crash in DiskIoMgr::GetFreeBuffer
• is related to: IMPALA-2561 Functional test coverage for parquet files with invalid data
• relates to: IMPALA-2230 Create test parquet files with bad metadata, invalid data, etc.

Sub-Tasks

• Fix invalid data handling in Parquet files: Resolved