[DRILL-7303] Filter record batch does not handle zero-length batches

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.16.0
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Created: 22/Jun/19 05:58
• Updated: 30/Nov/19 03:10
• Resolved: 30/Nov/19 03:10

Description

Testing of the row-set-based JSON reader revealed a limitation of the Filter record batch: if an incoming batch has zero records, the length of the associated SV2 is left at -1. In particular:

public class SelectionVector2 implements AutoCloseable {
  // Indicates actual number of rows in the RecordBatch
  // container which owns this SV2 instance
  private int batchActualRecordCount = -1;

Then:

public abstract class FilterTemplate2 implements Filterer {
  @Override
  public void filterBatch(int recordCount) throws SchemaChangeException{
    if (recordCount == 0) {
      outgoingSelectionVector.setRecordCount(0);
      return;
    }

Notice there is no call to set the actual record count. The solution is to insert one line of code:

    if (recordCount == 0) {
      outgoingSelectionVector.setRecordCount(0);
      outgoingSelectionVector.setBatchActualRecordCount(0); // <-- Add this
      return;
    }

Without this, the query fails with an error due to an invalid index of -1.

Issue Links

• duplicates: DRILL-7324 Many vector-validity errors from unit tests