Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-11052

Sending random data crashes thrift service

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.98.1, 1.0.0, 0.94.18
    • Fix Version/s: 0.99.0, 0.94.21, 0.98.4
    • Component/s: Thrift
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Thrift servers should use framed/compact protocol to protect against buffer overflow (default disabled as they are breaking old clients)
      - hbase.regionserver.thrift.framed = true
      - hbase.regionserver.thrift.compact = true
      Show
      Thrift servers should use framed/compact protocol to protect against buffer overflow (default disabled as they are breaking old clients) - hbase.regionserver.thrift.framed = true - hbase.regionserver.thrift.compact = true

      Description

      Upstream thrift library has a know issue (THRIFT-601) causing the thrift server to crash with an Out-of-Memory Error when bogus requests are sent.

      This reproduces when a very large request size is sent in the request header, making the thrift server to allocate a large memory segment leading to OOM.

      LoadBalancer health checks are the first "candidate" for bogus requests
      Thrift developers admit this is a known issue with TBinaryProtocol and their recommandation is to use TCompactProtocol/TFramedTransport but this requires all thrift clients to be updated (might not be feasible atm)

      So we need a fix similar to CASSANDRA-475.

        Attachments

        1. HBASE-11052_trunk_v4.patch
          4 kB
          Adrian Muraru
        2. HBASE-11052_trunk_v3.patch
          4 kB
          Adrian Muraru
        3. HBASE-11052_0.94_v4.patch
          6 kB
          Adrian Muraru
        4. HBASE-11052_trunk_v1.patch
          4 kB
          Adrian Muraru
        5. HBASE-11052_0.94_v2.patch
          5 kB
          Adrian Muraru

          Issue Links

            Activity

              People

              • Assignee:
                amuraru Adrian Muraru
                Reporter:
                amuraru Adrian Muraru
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: