[IMPALA-5085] Backend support for large rows in BufferedTupleStream

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.6.0
• Fix Version/s: Impala 2.10.0
• Component/s: Backend
• Labels: resource-management
• Target Version: Product Backlog
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 16/Mar/17 22:50
• Updated: 19/Jun/17 15:32
• Resolved: 19/Jun/17 15:32

Description

See IMPALA-3208 for the context.

BufferedTupleStream changes:

• Rows <= the default page size are written as before
• Rows that don't fit in the default page size get written into a larger page, with one row per page.
• Upon writing a large row to an unpinned stream, the page is immediately unpinned and we immediately advance to the next write page, so that the large page is not kept pinned outside of the AddRow() call.
• We should only be reading from one unpinned stream at a time, so only one large page is required there.

Issue Links

• blocks: IMPALA-3208 Backend support for large rows