[IMPALA-8420] Support HTTP based HS2/beeswax endpoints on coordinators

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 3.3.0
• Fix Version/s: Impala 3.3.0
• Component/s: Backend, Clients
• Epic Color: ghx-label-1
• Assignee: Thomas Tauber-Marshall
• Reporter: Bharath Vissapragada
• Created: 16/Apr/19 01:08
• Updated: 14/Aug/19 21:21
• Resolved: 30/May/19 17:43

Description

The ask is to support HTTP based endpoints for client <-> coordinator communication.
Currently they are implemented as thrift over binary. With this, the coordinator can support client reconnects (incase of flaky clients) and also opens up the possibility to use smarter L7 load balancers instead of relying on L4 load balancers using IP hashing.

Some notes (based on my research so far)

• Thrift supports THttpTransport but cookie support is unclear
• We need to rethink the server side session management a bit, since the current session handling is tied to the connection. If a connection is closed, server side session is wiped off. We need to decouple both to support reconnects.
• Current query lifecycle is tied to the client since the coordinator expects the client to drain the current result set row batch before producing the next one. If reconnects are supported, coordinator needs to aggressively materialize result set so that the resources are not held for longer while waiting for the client to reconnect.

Issue Links

• is duplicated by: IMPALA-8538 Support hiveserver2 over HTTP
• is related to: IMPALA-8863 Add support to run tests over HS2-HTTP

Sub-Tasks

• Impala Doc: Document HTTP based HS2/beeswax endpoints on coordinators: Closed
• impala-shell support for HiveServer2 HTTP endpoint: Resolved