[DRILL-1960] Automatically realloc buffers when vector runs out of space

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Unassigned
• Reporter: Steven Phillips
• Created: 08/Jan/15 20:29
• Updated: 22/Apr/15 17:32
• Resolved: 23/Jan/15 01:57

Description

When reaching the end of a buffer, the current way drill handles this is for setSafe() to return false, and then whatever operator is doing the write will send the batch, and the redo the last record. This creates a lot of difficulty, because it sometimes requires being able to "rewind" the input stream to replay the last record.

The proposal is to move the handling of buffer sizing and allocation into the value vectors themselves, making it transparent to the user of the value vector. The operators will now no longer have to worry about the possibility that writing into a vector may fail due to lack of space.

Issue Links

• duplicates: DRILL-1829 Move to a reallocation rather than rewind strategy for vectors
• is related to: DRILL-2739 Drill stops reading columns > 4095
• is related to: DRILL-1958 select * fails on a text file with 3500 columns or more