Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
ghx-label-11
Description
We found in a recent s3 build that the test of test_sequential_startup_wait failed due to admission for query exceeding timeout 60000ms.
The error message in the console output of the Jenkins job is the following.
Error Message: E Query aborted:Admission for query exceeded timeout 60000ms in pool default-pool. Queued reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run. Additional Details: Not Applicable
Before the timeout was reached we saw the following entries in the corresponding log file produced by an impalad.
W1119 20:11:45.731312 20667 executor-group.cc:164] e64b11989b018148:20eb7ce500000000] Executor group default-pool-group1 is unhealthy: 1 out of 3 are available. W1119 20:11:45.731338 20667 admission-controller.cc:1558] e64b11989b018148:20eb7ce500000000] Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run. I1119 20:11:45.731348 20667 admission-controller.cc:1210] e64b11989b018148:20eb7ce500000000] Queuing, query id=e64b11989b018148:20eb7ce500000000 reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run. I1119 20:11:45.773303 20040 admission-controller.cc:1876] Could not dequeue query id=e64b11989b018148:20eb7ce500000000 reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
The corresponding log files are also provided.
The test was recently revised in IMPALA-8830, maybe bikramjeet.vig could provide some insight into it. Thanks!