[DRILL-7441] Fix issues with fillEmpties, offset vectors

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.17.0
• Labels: ready-to-commit
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Reviewer: Vova Vysotskyi
• Created: 10/Nov/19 19:55
• Updated: 20/Nov/19 11:44
• Resolved: 20/Nov/19 11:44

Description

Enable the vector validator with full testing of offset vectors. A number of operators trigger errors. Tracking down the issues, and adding detailed tests, it turns out that:

• Drill has an informal standard that zero-length batches should have zero-length offset vectors, while a batch of size 1 will have offset vectors of size 2. Thus, zero-length is a special case.
• Nullable, repeated and variable-width vectors have "fill empties" logic that is used in two places: when setting the value count and when preparing to write a new value. The current logic is not quite right for either case.

Detailed vector checks fail due to inconsistencies in how the above works. This PR fixes those issues.

Issue Links

• links to: GitHub Pull Request #1896