[DRILL-4001] Empty vectors from previous batch left by MapVector.load(...)/RecordBatchLoader.load(...)

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Execution - Data Types
• Assignee: Paul Rogers
• Reporter: Daniel Barclay
• Created: 01/Nov/15 04:04
• Updated: 19/Jun/17 04:54

Description

In certain cases, MapVector.load(...) (called by RecordBatchLoader.load(...)) returns with some map child vectors having a length of zero instead of having a length matching the length of sibling vectors and the number of records in the batch. This causes MapVector.getObject(int) to fail, saying "java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))" (one of the errors seen in fixing DRILL-2288).

The condition seems to be that a child field (e.g., an HBase column in a HBase column family) appears in an earlier batch and does not appear in a later batch.

(The HBase column's child vector gets created (in the MapVector for the HBase column family) during loading of the earlier batch. During loading of the later batch, all vectors get reset to zero length, and then only vectors for fields appearing in the batch message being loaded get loaded and set to the length of the batch--other vectors created from earlier messages/load calls are left with a length of zero (instead of, say, being filled with nulls to the length of their siblings and the current record batch).)

See the TODO(DRILL-4001) mark and workaround in MapVector.getObject(int).

Issue Links

• relates to: DRILL-2288 ScanBatch violates IterOutcome protocol for zero-row sources [was: missing JDBC metadata (schema) for 0-row results...]