[IMPALA-1382] Wasted space in buffered-tuple-stream in presence of many NULL tuples

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.0
• Fix Version/s: Impala 2.10.0
• Component/s: Backend
• Target Version: Product Backlog
• Assignee: Tim Armstrong
• Reporter: Ippokratis Pandis
• Created: 10/Oct/14 18:29
• Updated: 05/Aug/17 03:21
• Resolved: 05/Aug/17 03:21

Description

We currently preallocate the null tuple indicator bitstring in each tuple stream assuming that very few tuples will be NULL. That can lead to lots of wasted space in the buffer if there are many NULL tuples in the stream.

A possible solution is to use a slotted page (buffer) with NULL indicators growing from the end of the page (buffer).

Issue Links

• depends upon: IMPALA-3200 Replace BufferedBlockMgr with new buffer pool
• relates to: IMPALA-2138 Get rid of unused columns by upstream operators at points of materialization