[IMPALA-6725] Impala 2.12 & 3.0 Docs: KRPC Milestone 1

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: Impala 3.0, Impala 2.12.0
• Component/s: Docs
• Epic Color: ghx-label-3
• Assignee: Alexandra Rodoni
• Reporter: Alexandra Rodoni
• Created: 22/Mar/18 23:08
• Updated: 27/Mar/18 22:38
• Resolved: 27/Mar/18 22:38

Description

IMPALA-2567 replaces the RPC system for transmitting data between Impala backends with Kudu RPC which provides asynchronous RPC and connection multiplexing. Previously, Impala uses Thrift RPC and Impala will create a connection and a thread between each fragment instance pair. This leads to excessive number of connections and threads created when running complex queries on a moderate size cluster, causing crashes of Impala and inefficient CPU scheduling. After switching to KRPC, the number of threads responsible for RPC transportation will be fixed. There is only one pair of connection between any two hosts, one for each direction. Note that we still use the legacy thrift RPC system for some other RPCs between Impala backends so we may still exert some pressure on KDC in a cold cluster. We plan to eventually replace all RPCs between Impala backends with KRPC.

This KRPC feature is enabled by default and it can be disabled by using the start-up flag --use_krpc=false.