[HIVE-10103] LLAP: Cancelling tasks fails to stop cache filling threads

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: llap
• Assignee: Sergey Shelukhin
• Reporter: Gopal Vijayaraghavan
• Created: 26/Mar/15 21:04
• Updated: 17/Feb/16 01:32
• Resolved: 01/Apr/15 21:46

Description

Running a bad query (~1Tb scan on a 1Gb cache) and killing the tasks via the container launcher fails to free up the cache filler threads.

The cache filler threads with no consumers get stuck into a loop

2015-03-26 14:02:47,335 [pool-2-thread-2(container_222212222_1659_01_000074_gopal_20150326135614_2bb61f02-3c2b-4512-a34e-81803cd13fb6:1_Map 1_73_0)] WARN org.apache.hadoop.hive.llap.io.api.impl.LlapIoImpl: Cannot evict blocks for 262144 calls; cache full?
2015-03-26 14:02:48,018 [pool-2-thread-7(container_222212222_1659_01_000076_gopal_20150326135614_2bb61f02-3c2b-4512-a34e-81803cd13fb6:1_Map 1_75_0)] WARN org.apache.hadoop.hive.llap.io.api.impl.LlapIoImpl: Cannot evict blocks for 262144 calls; cache full?
2015-03-26 14:02:51,658 [pool-2-thread-1(container_222212222_1659_01_000073_gopal_20150326135614_2bb61f02-3c2b-4512-a34e-81803cd13fb6:1_Map 1_72_0)] WARN org.apache.hadoop.hive.llap.io.api.impl.LlapIoImpl: Cannot evict blocks for 262144 calls; cache full?

Needs to kill a daemon to get back to normal operations.