[HBASE-11052] Sending random data crashes thrift service - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 0.98.1, 1.0.0, 0.94.18
Fix Version/s: 0.99.0, 0.94.21, 0.98.4
Component/s: Thrift
Labels:
None

Hadoop Flags:

Reviewed
Release Note:

Hide
Thrift servers should use framed/compact protocol to protect against buffer overflow (default disabled as they are breaking old clients)
- hbase.regionserver.thrift.framed = true
- hbase.regionserver.thrift.compact = true

Show
Thrift servers should use framed/compact protocol to protect against buffer overflow (default disabled as they are breaking old clients) - hbase.regionserver.thrift.framed = true - hbase.regionserver.thrift.compact = true

Description

Upstream thrift library has a know issue (~~THRIFT-601~~) causing the thrift server to crash with an Out-of-Memory Error when bogus requests are sent.

This reproduces when a very large request size is sent in the request header, making the thrift server to allocate a large memory segment leading to OOM.

LoadBalancer health checks are the first "candidate" for bogus requests
Thrift developers admit this is a known issue with TBinaryProtocol and their recommandation is to use TCompactProtocol/TFramedTransport but this requires all thrift clients to be updated (might not be feasible atm)

So we need a fix similar to ~~CASSANDRA-475~~.