[TAJO-1041] Tuple Block Iteration Model

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: block_iteration
• Labels: block_iteration
• Assignee: Unassigned
• Reporter: Hyunsik Choi
• Created: 16/Sep/14 09:11
• Updated: 21/Oct/15 04:39

Description

Currently, our physical executor is based on a tuple-at-a-time model. It is a traditional model and is well known that it is not efficient in terms of CPU costs.

Also, I think that we can design integrated interface for row block with considering row block and vectorized row block. I'll add some possibility to extend row block iteration model to be vectorized.

In addition, in order to achieve more CPU efficient implementations, we will make good use of off heap data structure (TAJO-907) and code generation (TAJO-906).

This is an umbrella issue, which is to change current execution model to block iteration model.

Issue Links

• depends upon: TAJO-907 Implement off-heap tuple block and zero-copy tuple
• depends upon: TAJO-906 Runtime code generation for evaluating expression trees
• is related to: TAJO-631 Zero copy physical layer processing
• is related to: TAJO-1148 Support for offheap-based scanner
• relates to: TAJO-1058 Implement off-heap hash table for hash aggregation

Sub-Tasks

• Implement block iteration interfaces for executors and scanners: Resolved
• Implement nextFetch(RowBlock) of CSVScanner: Resolved
• Implement nextFetch(RowBlock) of Parquer scanner: Resolved
• Implement nextFetch(RowBlock) of RCFile scanner: Resolved
• Function and UDF mechanism should not use Datum as parameter and return values: In Progress
• HashShuffleFileWriteExec should be block iterative.: Open
• Cleanup BooleanDatum and Inet4Datum: Resolved
• Implement nextFetch(RowBlock) of RawFileScanner: Resolved
• StoreTableExec should be block iterative.: Resolved
• Generated classes should access directly UnSafeTuple and RowWriter.: Resolved
• Implement nextFetch(RowBlock) of SequenceFile scanner: Open
• Implement off-heap hash table for hash aggregation: In Progress
• LimitExec should be block iterative.: Open
• SelectionExec should be block iterative.: Open
• RangeShuffleFileWriteExec should be block iterative.: Open
• ProjectionExec should be block iterative.: Open
• MergeJoinExec should be block iterative.: Open