[IMPALA-1722] Clients that don't Fetch can cause Impala to busy spin in ScannerThreads

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.1
• Fix Version/s: Impala 2.2
• Assignee: Ippokratis Pandis
• Reporter: Nong Li
• Created: 30/Jan/15 22:50
• Updated: 02/Feb/16 02:54
• Resolved: 06/Feb/15 16:33

Description

The scanner thread registers with the token mgr in Prepare() (Execute from the client) and doesn't issue work to the io_mgr until GetNext() (Fetch from the client). After registering with the token mgr, the token mgr will offer tokens (HdfsScanNode::ThreadAvailableCb) which spins up scanner threads. Since there are no ranges issued, the scanner threads busy spin keeping a core busy per thread.

This is bad anytime there is a large gap between Prepare/Open() and GetNext(). This can happen because the client went away (spinning until query timeout) or other parts of the plan take a long time (i.e. processing the build side of a join).

Issue Links

• is duplicated by: IMPALA-2901 Impala query using 100% CPU system state