[IMPALA-3481] Kudu Connector: use new Scan Token API

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Kudu_Impala
• Fix Version/s: Kudu_Impala, Impala 2.8.0
• Component/s: Backend, Frontend
• Labels: kudu
• Target Version: Kudu_Impala
• Assignee: Matthew Jacobs
• Reporter: Dan Burkert
• Created: 05/May/16 20:08
• Updated: 06/Jul/17 17:35
• Resolved: 08/Sep/16 14:32

Description

Kudu 0.9 will include a new feature called Scan Tokens which will simplify planning on the Java side and creating scanners on the C++ side. The API is designed with Impala in mind, so it has a built in serialization method so that scan tokens can be 'created' on the Java side, and then serialized and rehydrated into a scanner. See the design doc for a high level intro to the API, and the MR input format and Spark RDD for usage examples.

Issue Links

• blocks: IMPALA-3874 Predicates are not always pushed to Kudu
• blocks: IMPALA-4016 Avoid materializing slots only referenced by Kudu conjuncts