[HIVE-11263] LLAP: TaskExecutorService state is not cleaned up

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: llap
• Assignee: Siddharth Seth
• Reporter: Sergey Shelukhin
• Created: 14/Jul/15 22:44
• Updated: 17/Feb/16 01:33
• Resolved: 17/Jul/15 15:47

Description

See TaskExecutorService::getExecutorsStatus, this is used to report on queue/etc. status in JMX.
Currently, it reports 100s of bogus tasks in queue:

"ExecutorsStatus" : [ "attempt_1435700346116_1888_1_04_000205_22 (sershe_20150714174105_0d013941-1f0e-4f74-9387-a2f29279a185:3/Map 1, in queue)", "attempt_1435700346116_1889_1_05_000101_1 (sershe_20150714174104_b0b9f300-667e-4370-bb01-f9cb7da331e0:4/Map 1, in queue)", "attempt_1435700346116_1889_1_05_000191_3 (sershe_20150714174104_b0b9f300-667e-4370-bb01-f9cb7da331e0:4/Map 1, in queue)", "attempt_1435700346116_1887_7_00_000202_3 (sershe_20150714174737_bea682d1-fa0f-4281-a1cb-439d85bb2016:22/Map 5, in queue)", "attempt_1435700346116_1886_1_04_000073_15 (sershe_20150714174108_f9483d76-8fd9-4f82-96ee-17231b6f9b2c:1/Reducer 2, in queue)", "attempt_1435700346116_1887_8_04_000166_15 (sershe_20150714174900_710d7d69-3d66-45e9-865b-cd0f87bb0d98:27/Map 1, in queue)", "attempt_1435700346116_1888_11_05_000140_3 (sershe_20150714174903_50359459-5342-4d1b-852c-622a3fa92a27:28/Map 3, in queue)", "attempt_1435700346116_1886_1_04_000042_29 (sershe_20150714174108_f9483d76-8fd9-4f82-96ee-17231b6f9b2c:1/Reducer 2, in queue)", "attempt_1435700346116_1888_2_03_000169_12 (sershe_20150714174310_97ce1d4b-8029-4ef6-a823-46e29f09718a:5/Map 1, in queue)", "attempt_1435700346116_1887_1_04_000197_18 (sershe_20150714174107_8fcfe954-4eeb-46e5-bad5-42a47327b26c:2/Map 1, in queue)", "attempt_1435700346116_1887_1_04_000218_21 (sershe_20150714174107_8fcfe954-4eeb-46e5-bad5-42a47327b26c:2/Map 1, in queue)", "attempt_1435700346116_1886_7_09_000084_1 (sershe_20150714174841_462b9bdb-c017-47c2-9fa7-7edfbfc09e60:24/Map 1, in queue)", "attempt_1435700346116_1887_5_04_000078_0 (sershe_20150714174509_9a5cd476-b3c8-4679-af8e-1188922713a2:14/Map 3, in queue)", "attempt_1435700346116_1887_7_04_000162_6 (sershe_20150714174737_bea682d1-fa0f-4281-a1cb-439d85bb2016:22/Map 3, in queue)", "attempt_1435700346116_1887_7_04_000180_0 (sershe_20150714174737_bea682d1-fa0f-4281-a1cb-439d85bb2016:22/Map 3, in queue)", "attempt_1435700346116_1886_3_04_000144_0 (sershe_20150714174435_fe3077dd-a97f-4582-995b-5f723170b02f:12/Reducer 2, in queue)", "attempt_1435700346116_1887_5_00_000153_1 (sershe_20150714174509_9a5cd476-b3c8-4679-af8e-1188922713a2:14/Map 5, in queue)", "attempt_1435700346116_1887_7_04_000141_7 (sershe_20150714174737_bea682d1-fa0f-4281-a1cb-439d85bb2016:22/Map 3, in queue)", "attempt_1435700346116_1887_1_04_000024_7 (sershe_20150714174107_8fcfe954-4eeb-46e5-bad5-42a47327b26c:2/Map 1, in queue)", "attempt_1435700346116_1887_5_04_000130_1 (sershe_20150714174509_9a5cd476-b3c8-4679-af8e-1188922713a2:14/Map 3, in queue)", "attempt_1435700346116_1888_1_04_000200_1 (sershe_20150714174105_0d013941-1f0e-4f74-9387-a2f29279a185:3/Map 1, in queue)", "attempt_1435700346116_1886_15_04_000180_0 (sershe_20150714175411_bda950b7-8aa5-417f-84f6-dd646247dca8:43/Map 1, in queue)", "attempt_1435700346116_1887_7_00_000205_1 (sershe_20150714174737_bea682d1-fa0f-4281-a1cb-439d85bb2016:22/Map 5, in queue)", "attempt_1435700346116_1888_4_04_000183_4 (sershe_20150714174407_f0924540-f69f-45c2-831a-9d2d1f66a124:10/Map 1, in queue)", "attempt_1435700346116_1887_1_04_000081_6 (sershe_20150714174107_8fcfe954-4eeb-46e5-bad5-42a47327b26c:2/Map 1, in queue)", "attempt_1435700346116_1888_1_04_000080_4 (sershe_20150714174105_0d013941-1f0e-4f74-9387-a2f29279a185:3/Map 1, in queue)", "attempt_1435700346116_1887_7_04_000005_3 (sershe_20150714174737_bea682d1-fa0f-4281-a1cb-439d85bb2016:22/Map 3, in queue)", "attempt_1435700346116_1887_7_00_000169_2 (sershe_20150714174737_bea682d1-fa0f-4281-a1cb-439d85bb2016:22/Map 5, in queue)", "attempt_1435700346116_1888_8_04_000037_2 (sershe_20150714174731_261f2d52-8c47-4db6-8f17-8098efe144a2:20/Reducer 3, in queue)", "attempt_1435700346116_1887_9_00_000096_6 (sershe_20150714175015_cc1b6647-8479-4c5f-918c-00935bff7232:30/Map 5, in queue)", "attempt_1435700346116_1888_11_01_000001_2 (sershe_20150714174903_50359459-5342-4d1b-852c-622a3fa92a27:28/Map 7, in queue)", "attempt_1435700346116_1889_1_05_000206_8 (sershe_20150714174104_b0b9f300-667e-4370-bb01-f9cb7da331e0:4/Map 1, in queue)", "attempt_1435700346116_1887_5_04_000054_0 (sershe_20150714174509_9a5cd476-b3c8-4679-af8e-1188922713a2:14/Map 3, in queue)", "attempt_1435700346116_1889_1_05_000168_21 (sershe_20150714174104_b0b9f300-667e-4370-bb01-f9cb7da331e0:4/Map 1, in queue)", "attempt_1435700346116_1889_1_05_000078_16 (sershe_20150714174104_b0b9f300-667e-4370-bb01-f9cb7da331e0:4/Map 1, in queue)", "attempt_1435700346116_1888_1_04_000091_7 (sershe_20150714174105_0d013941-1f0e-4f74-9387-a2f29279a185:3/Map 1, in queue)", "attempt_1435700346116_1888_4_04_000148_1 (sershe_20150714174407_f0924540-f69f-45c2-831a-9d2d1f66a124:10/Map 1, in queue)", "attempt_1435700346116_1886_3_03_000184_3 (sershe_20150714174435_fe3077dd-a97f-4582-995b-5f723170b02f:12/Map 1, in queue)", "attempt_1435700346116_1887_1_04_000175_1 (sershe_20150714174107_8fcfe954-4eeb-46e5-bad5-42a47327b26c:2/Map 1, in queue)", "attempt_1435700346116_1887_8_04_000074_3 (sershe_20150714174900_710d7d69-3d66-45e9-865b-cd0f87bb0d98:27/Map 1, in queue)", "attempt_1435700346116_1888_7_03_000169_8 (sershe_20150714174629_c40b5b79-39d0-4b72-aee1-1015f2f8ec0e:17/Map 1, in queue)", "attempt_1435700346116_1888_1_04_000124_9 (sershe_20150714174105_0d013941-1f0e-4f74-9387-a2f29279a185:3/Map 1, in queue)", "attempt_1435700346116_1889_1_05_000130_2 (sershe_20150714174104_b0b9f300-667e-4370-bb01-f9cb7da331e0:4/Map 1, in queue)", "attempt_1435700346116_1889_1_05_000130_4 (sershe_20150714174104_b0b9f300-667e-4370-bb01-f9cb7da331e0:4/Map 1, in queue)", "attempt_1435700346116_1888_7_03_000137_7 (sershe_20150714174629_c40b5b79-39d0-4b72-aee1-1015f2f8ec0e:17/Map 1, in queue)", "attempt_1435700346116_1887_8_04_000125_19 (sershe_20150714174900_710d7d69-3d66-45e9-865b-cd0f87bb0d98:27/Map 1, in queue)", "attempt_1435700346116_1888_2_03_000090_2 (sershe_20150714174310_97ce1d4b-8029-4ef6-a823-46e29f09718a:5/Map 1, in queue)", "attempt_1435700346116_1887_7_04_000044_3 (sershe_20150714174737_bea682d1-fa0f-4281-a1cb-439d85bb2016:22/Map 3, in queue)", "attempt_1435700346116_1888_1_04_000014_6 (sershe_20150714174105_0d013941-1f0e-4f74-9387-a2f29279a185:3/Map 1, in queue)", "attempt_1435700346116_1887_1_04_000175_9 (sershe_20150714174107_8fcfe954-4eeb-46e5-bad5-42a47327b26c:2/Map 1, in queue)", "attempt_1435700346116_1887_8_04_000183_1 (sershe_20150714174900_710d7d69-3d66-45e9-865b-cd0f87bb0d98:27/Map 1, in queue)", "attempt_1435700346116_1887_1_04_000111_1 (sershe_20150714174107_8fcfe954-4eeb-46e5-bad5-42a47327b26c:2/Map 1, in queue)", "attempt_1435700346116_1888_1_04_000046_7 (sershe_20150714174105_0d013941-1f0e-4f74-9387-a2f29279a185:3/Map 1, in queue)", "attempt_1435700346116_1889_1_05_000041_22 (sershe_20150714174104_b0b9f300-667e-4370-bb01-f9cb7da331e0:4/Map 1, in queue)", "attempt_1435700346116_1888_1_04_000158_11 (sershe_20150714174105_0d013941-1f0e-4f74-9387-a2f29279a185:3/Map 1, in queue)", "attempt_1435700346116_1889_1_05_000013_11 (sershe_20150714174104_ ... a lot more

for queries and vertices that have long finished.
Looks like knownTasks is not cleaned properly.