Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
Impala 3.4.0
-
None
-
None
-
ghx-label-1
Description
This looks like a bug to me but I'm not entirely sure. I'm trying to run our tests over hs2-http (IMPALA-8863) and after the change for IMPALA-7312 to introduce a non-blocking mode for FetchResults() it looks like we sometimes return an empty result way before fetch_rows_timeout_ms has elapsed. This triggers a bug in Impyla (#369), but it also seems like something we should investigate and fix in Impala.
I1007 22:10:10.697760 56550 impala-hs2-server.cc:821] FetchResults(): query_id=764d4313dbc64e20:2831560c00000000 fetch_size=1024I1007 22:10:10.697760 56550 impala-hs2-server.cc:821] FetchResults(): query_id=764d4313dbc64e20:2831560c00000000 fetch_size=1024I1007 22:10:10.697988 56527 scheduler.cc:468] 6d4cba4d2e8ccc42:66ce26a800000000] Exec at coord is falseI1007 22:10:10.698014 54090 impala-hs2-server.cc:663] GetOperationStatus(): query_id=0d43fd73ce4403fd:da25dde900000000I1007 22:10:10.698173 127 control-service.cc:142] 0646e91fd6a0a953:02949ff300000000] ExecQueryFInstances(): query_id=0646e91fd6a0a953:02949ff300000000 coord=b04a12d76e27:22000 #instances=1I1007 22:10:10.698356 56527 admission-controller.cc:1270] 6d4cba4d2e8ccc42:66ce26a800000000] Trying to admit id=6d4cba4d2e8ccc42:66ce26a800000000 in pool_name=root.default executor_group_name=default per_host_mem_estimate=52.02 MB dedicated_coord_mem_estimate=110.02 MB max_requests=-1 (configured statically) max_queued=200 (configured statically) max_mem=29.30 GB (configured statically)I1007 22:10:10.698386 56527 admission-controller.cc:1282] 6d4cba4d2e8ccc42:66ce26a800000000] Stats: agg_num_running=9, agg_num_queued=0, agg_mem_reserved=8.34 GB, local_host(local_mem_admitted=9.09 GB, num_admitted_running=9, num_queued=0, backend_mem_reserved=6.70 GB)I1007 22:10:10.698415 56527 admission-controller.cc:871] 6d4cba4d2e8ccc42:66ce26a800000000] Admitting query id=6d4cba4d2e8ccc42:66ce26a800000000I1007 22:10:10.698479 56527 impala-server.cc:1713] 6d4cba4d2e8ccc42:66ce26a800000000] Registering query locationsI1007 22:10:10.698529 56527 coordinator.cc:97] 6d4cba4d2e8ccc42:66ce26a800000000] Exec() query_id=6d4cba4d2e8ccc42:66ce26a800000000 stmt=select count(*) from alltypes where month=1I1007 22:10:10.698992 56527 coordinator.cc:361] 6d4cba4d2e8ccc42:66ce26a800000000] starting execution on 3 backends for query_id=6d4cba4d2e8ccc42:66ce26a800000000I1007 22:10:10.699383 56523 coordinator.cc:375] 0646e91fd6a0a953:02949ff300000000] started execution on 1 backends for query_id=0646e91fd6a0a953:02949ff300000000I1007 22:10:10.699409 56534 scheduler.cc:468] e1495f928c2cd4f6:eeda82aa00000000] Exec at coord is falseI1007 22:10:10.700017 127 control-service.cc:142] 6d4cba4d2e8ccc42:66ce26a800000000] ExecQueryFInstances(): query_id=6d4cba4d2e8ccc42:66ce26a800000000 coord=b04a12d76e27:22000 #instances=1I1007 22:10:10.700147 56534 scheduler.cc:468] e1495f928c2cd4f6:eeda82aa00000000] Exec at coord is falseI1007 22:10:10.700234 325 TAcceptQueueServer.cpp:340] New connection to server hiveserver2-http-frontend from client <Host: 172.18.0.1 Port: 51580>I1007 22:10:10.700286 329 TAcceptQueueServer.cpp:227] TAcceptQueueServer: hiveserver2-http-frontend started connection setup for client <Host: 172.18.0.1 Port: 51580>I1007 22:10:10.700314 329 TAcceptQueueServer.cpp:245] TAcceptQueueServer: hiveserver2-http-frontend finished connection setup for client <Host: 172.18.0.1 Port: 51580>I1007 22:10:10.700371 56550 impala-hs2-server.cc:844] FetchResults(): query_id=764d4313dbc64e20:2831560c00000000 #results=1 has_more=trueI1007 22:10:10.700508 56551 impala-server.cc:1969] Connection 8249c7defcb10124:1bc65ed9ea562aab from client 172.18.0.1:51576 to server hiveserver2-http-frontend closed. The connection had 1 associated session(s).I1007 22:10:10.700688 53748 impala-beeswax-server.cc:260] close(): query_id=e9473ff80c5d4afe:733cefe000000000I1007 22:10:10.700711 53748 impala-server.cc:1129] UnregisterQuery(): query_id=e9473ff80c5d4afe:733cefe000000000I1007 22:10:10.700721 53748 impala-server.cc:1234] Cancel(): query_id=e9473ff80c5d4afe:733cefe000000000I1007 22:10:10.700742 53748 coordinator.cc:716] CancelBackends() query_id=e9473ff80c5d4afe:733cefe000000000, tried to cancel 0 backendsI1007 22:10:10.700690 56534 scheduler.cc:468] e1495f928c2cd4f6:eeda82aa00000000] Exec at coord is falseI1007 22:10:10.701387 56534 admission-controller.cc:1270] e1495f928c2cd4f6:eeda82aa00000000] Trying to admit id=e1495f928c2cd4f6:eeda82aa00000000 in pool_name=root.default executor_group_name=default per_host_mem_estimate=231.95 MB dedicated_coord_mem_estimate=109.92 MB max_requests=-1 (configured statically) max_queued=200 (configured statically) max_mem=29.30 GB (configured statically)I1007 22:10:10.701417 56534 admission-controller.cc:1282] e1495f928c2cd4f6:eeda82aa00000000] Stats: agg_num_running=10, agg_num_queued=0, agg_mem_reserved=8.34 GB, local_host(local_mem_admitted=9.84 GB, num_admitted_running=10, num_queued=0, backend_mem_reserved=6.70 GB)I1007 22:10:10.701447 56534 admission-controller.cc:871] e1495f928c2cd4f6:eeda82aa00000000] Admitting query id=e1495f928c2cd4f6:eeda82aa00000000I1007 22:10:10.701503 56534 impala-server.cc:1713] e1495f928c2cd4f6:eeda82aa00000000] Registering query locationsI1007 22:10:10.701534 56534 coordinator.cc:97] e1495f928c2cd4f6:eeda82aa00000000] Exec() query_id=e1495f928c2cd4f6:eeda82aa00000000 stmt=with t1 as (select int_col x, bigint_col y from functional.alltypes limit 2), t2 as (select int_col x, bigint_col y from functional.alltypestiny limit 2), t3 as (select int_col x, bigint_col y from functional.alltypessmall limit 2) select * from t1, t2, t3 where t1.x = t2.x and t2.x = t3.xI1007 22:10:10.702641 56534 coordinator.cc:361] e1495f928c2cd4f6:eeda82aa00000000] starting execution on 3 backends for query_id=e1495f928c2cd4f6:eeda82aa00000000I1007 22:10:10.703843 56556 query-state.cc:643] 6d4cba4d2e8ccc42:66ce26a800000000] Executing instance. instance_id=6d4cba4d2e8ccc42:66ce26a800000000 fragment_idx=0 per_fragment_instance_idx=0 coord_state_idx=2 #in-flight=18I1007 22:10:10.704418 56550 impala-server.cc:1969] Connection 41442926b263eea1:b922dd2b3a2513bd from client 172.18.0.1:51572 to server hiveserver2-http-frontend closed. The connection had 1 associated session(s).I1007 22:10:10.704545 325 TAcceptQueueServer.cpp:340] New connection to server hiveserver2-http-frontend from client <Host: 172.18.0.1 Port: 51584>I1007 22:10:10.704847 56553 query-state.cc:643] 0646e91fd6a0a953:02949ff300000000] Executing instance. instance_id=0646e91fd6a0a953:02949ff300000000 fragment_idx=0 per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=19I1007 22:10:10.705631 327 TAcceptQueueServer.cpp:227] TAcceptQueueServer: hiveserver2-http-frontend started connection setup for client <Host: 172.18.0.1 Port: 51584>I1007 22:10:10.705659 327 TAcceptQueueServer.cpp:245] TAcceptQueueServer: hiveserver2-http-frontend finished connection setup for client <Host: 172.18.0.1 Port: 51584>I1007 22:10:10.705934 56555 impala-server.cc:1969] Connection 0b41c31be5df55d6:c2d9fcea092c26b2 from client 172.18.0.1:51580 to server hiveserver2-http-frontend closed. The connection had 1 associated session(s).I1007 22:10:10.706935 56557 impala-hs2-server.cc:821] FetchResults(): query_id=764d4313dbc64e20:2831560c00000000 fetch_size=1024I1007 22:10:10.707320 325 TAcceptQueueServer.cpp:340] New connection to server hiveserver2-http-frontend from client <Host: 172.18.0.1 Port: 51588>I1007 22:10:10.707453 56535 query-state.cc:652] 764d4313dbc64e20:2831560c00000001] Instance completed. instance_id=764d4313dbc64e20:2831560c00000001 #in-flight=18 status=OKI1007 22:10:10.713026 329 TAcceptQueueServer.cpp:227] TAcceptQueueServer: hiveserver2-http-frontend started connection setup for client <Host: 172.18.0.1 Port: 51588>I1007 22:10:10.713054 329 TAcceptQueueServer.cpp:245] TAcceptQueueServer: hiveserver2-http-frontend finished connection setup for client <Host: 172.18.0.1 Port: 51588>I1007 22:10:10.714377 127 control-service.cc:142] e1495f928c2cd4f6:eeda82aa00000000] ExecQueryFInstances(): query_id=e1495f928c2cd4f6:eeda82aa00000000 coord=b04a12d76e27:22000 #instances=6I1007 22:10:10.715759 56527 coordinator.cc:375] 6d4cba4d2e8ccc42:66ce26a800000000] started execution on 3 backends for query_id=6d4cba4d2e8ccc42:66ce26a800000000I1007 22:10:10.717445 56560 query-state.cc:643] e1495f928c2cd4f6:eeda82aa00000000] Executing instance. instance_id=e1495f928c2cd4f6:eeda82aa00000000 fragment_idx=0 per_fragment_instance_idx=0 coord_state_idx=2 #in-flight=19I1007 22:10:10.718142 127 coordinator.cc:745] Backend completed: host=1161aa24dbe8:22000 remaining=3 query_id=99454b2820335de0:0a0a549e00000000I1007 22:10:10.718163 127 coordinator-backend-state.cc:338] query_id=99454b2820335de0:0a0a549e00000000: first in-progress backend: 38e8ee0f58f6:22000I1007 22:10:10.720214 56557 impala-hs2-server.cc:844] FetchResults(): query_id=764d4313dbc64e20:2831560c00000000 #results=0 has_more=true
Attachments
Issue Links
- blocks
-
IMPALA-8863 Add support to run tests over HS2-HTTP
- Resolved
- duplicates
-
IMPALA-8962 FETCH_ROWS_TIMEOUT_MS should apply before rows are available
- Resolved
- is caused by
-
IMPALA-7312 Non-blocking mode for Fetch() RPC
- Resolved
- is related to
-
IMPALA-9098 TestQueries.test_union failed
- Resolved