[DRILL-7305] Multiple operators do not handle empty batches

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.16.0
• Assignee: Unassigned
• Reporter: Paul Rogers
• Created: 24/Jun/19 02:09
• Updated: 30/Nov/19 03:09
• Resolved: 30/Nov/19 03:09

Description

While testing the new "EVF" framework, it was found that multiple operators incorrectly handle empty batches. The EVF framework is set up to return a "fast schema" empty batch with only schema as its first batch. It turns out that many operators fail with problems such as:

• Failure to set the value counts in the output container
• Fail to initialize the offset vector position 0 to 0 for variable-width or repeated vectors

And so on.

Partial fixes are in the JSON reader PR.

For now, the easiest work-around is to disable the "fast schema" path in the EVF: DRILL-7306.

To discover the remaining issues, enable the ScanOrchestratorBuilder.enableSchemaBatch option and run unit tests. You can use the VectorChecker and VectorAccessorUtilities.verify() methods to check state. Insert a call to verify() in each "next" method: verify the incoming and outgoing batches. The checker only verifies a few vector types; but these are enough to show many problems.

Issue Links

• duplicates: DRILL-7324 Many vector-validity errors from unit tests