[THRIFT-5322] Go compact_protocol allocating unreasonable buffer size - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 0.13.0
Fix Version/s: 0.14.0
Component/s: Go - Library
Labels:
None

Description

I don't yet know all the pieces to this puzzle, and it's quite possible that the problem is on our side, but we use the Thrift Go library in the Jaeger Agent and we are seeing a case where the memory consumption for a payload of 4k bytes to result in a buffer allocation in the compact_protocol.go with unreasonable sizes. I found buffers of 1.4GiB while debugging the issue.

This is the code that we are seeing this memory usage:

https://github.com/apache/thrift/blob/b75e88a33d67ae05ef9b5fa001d2a63a2effe377/lib/go/thrift/compact_protocol.go#L556-L577

Here's more information about this, including a reproducer and initial diagnostics:

https://github.com/jaegertracing/jaeger/issues/2638#issuecomment-741848201

As mentioned above, I'm still getting all the pieces together, but perhaps you've seen this before or know what might be going on. What I know for sure at the moment is that this happens on malformed payloads, but I would expect the library to have an upper limit on the buffer size.