Affects Version/s: 1.16.0
Fix Version/s: None
While testing the new "EVF" framework, it was found that multiple operators incorrectly handle empty batches. The EVF framework is set up to return a "fast schema" empty batch with only schema as its first batch. It turns out that many operators fail with problems such as:
- Failure to set the value counts in the output container
- Fail to initialize the offset vector position 0 to 0 for variable-width or repeated vectors
And so on.
Partial fixes are in the JSON reader PR.
For now, the easiest work-around is to disable the "fast schema" path in the EVF:
To discover the remaining issues, enable the ScanOrchestratorBuilder.enableSchemaBatch option and run unit tests. You can use the VectorChecker and VectorAccessorUtilities.verify() methods to check state. Insert a call to verify() in each "next" method: verify the incoming and outgoing batches. The checker only verifies a few vector types; but these are enough to show many problems.