[DRILL-5344] External sort priority queue copier fails with an empty batch

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.11.0
• Labels: ready-to-commit
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Reviewer: Boaz Ben-Zvi
• Created: 10/Mar/17 05:32
• Updated: 05/May/17 13:32
• Resolved: 05/May/17 13:32

Description

The external sort uses a "priority queue copier" to merge batches when spilling or when merging spilled batches.

The code will fail with an IndexOutOfBoundsException if any record batch is empty. The reason is a faulty assumption in generated code:

  public void setup(...) {
    ...
      vector4.set(i, i, batchGroups.get(i).getNextIndex());
    ...
  }

  public int getNextIndex() {
    if (pointer == getRecordCount()) {
      return -1;
    }
    ...
  }

The code to get the next index returns -1 when the "position" in a record batch is zero. The -1 position translates (when truncated) into 65535 which produces the index exception.

The workaround has been to special case empty batches elsewhere in the code, apparently to avoid hitting this error.

Issue Links

• links to: GitHub Pull Request #778