Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
Impala 2.2.4
Description
Impala relies on Thrift 0.9.X for its RPC implementation. This is a venerable release and is showing its age. Problems include:
- Low-quality SASL support (that we implemented ourselves)
- Lack of high-quality nonblocking server (TNonBlockingServer is ok, but doesn't work with SASL, making it a non-starter for us. It is also very hard to provide support for sessions with TNonBlockingServer).
- Lack of 0-copy native bytes type, meaning that large data structures are inefficient to send
- Lack of support for async. server implementation, so expensive RPCs can consume threads that could have been used by cheap ones.
Both Kudu and fbthrift have RPC implementations that address some or all of these shortcomings. We should evaluate both and commit to moving our RPC stack to one or the other.
Attachments
Issue Links
- contains
-
IMPALA-5577 Memory leak when looping a select, CTAS, and daemon crashes
- Resolved
- duplicates
-
IMPALA-1717 Reduce backend TCP connections per-node to be independent of the number of backends
- Resolved
- is blocked by
-
IMPALA-4905 Fragments always report insert status, even if not insert query
- Resolved
-
IMPALA-5131 Known issues with KRPC (IMPALA-2567)
- Resolved
-
KUDU-2305 Local variables can overflow when serializing a 2GB message
- Resolved
-
KUDU-2228 Make Messenger options configurable
- Resolved
- is depended upon by
-
IMPALA-5441 Send larger row batches over the wire
- Open
-
IMPALA-5473 Make diagnosing network issues easier
- Open
- is duplicated by
-
IMPALA-4127 different fragment should not authenticate to the same node again in a short period of time
- Resolved
- is related to
-
IMPALA-9919 Bad Impala Performance after a period of time
- Open
-
IMPALA-4317 Single Overloaded Impalad Causes Entire Cluster to Hang
- Resolved
-
IMPALA-6720 impala deamon server 22000 connect refuse in cluster running time
- Resolved
-
IMPALA-4069 Introduce startup option to create and cache backend connections on startup
- Resolved
-
IMPALA-4127 different fragment should not authenticate to the same node again in a short period of time
- Resolved
-
IMPALA-4952 Differentiate between resolved and unresolved IP addresses
- Resolved
- relates to
-
IMPALA-6238 Add source and destination hosts to TErrorCode::DATASTREAM_SENDER_TIMEOUT
- Resolved
-
IMPALA-5865 Improve Impala execution scalability
- Open
-
IMPALA-6025 Improve hang diagnostics
- Open
-
KUDU-2011 Request-side sidecars cannot be safely destroyed on timeout
- Reopened
-
KUDU-2110 RPC footer may be appended more than once
- Resolved
-
IMPALA-2990 Coordinator should timeout and cancel queries with unresponsive / stuck executors
- Resolved
-
KUDU-2086 Uneven assignment of connections to Reactor threads creates skew and limits transfer throughput
- Open
-
KUDU-2192 KRPC should have a timer to close stuck connections
- Open
-
IMPALA-5481 RowDescriptors should be shared, rather than copied
- Resolved
-
IMPALA-6095 Microbenchmark for debugging KRPC issues
- Resolved
- supercedes
-
IMPALA-5199 Impala may hang on empty row batch exchange
- Resolved