[FLUME-683] Add input/output buffering to ThriftEventSource/Sink

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.4
• Fix Version/s: 0.9.5
• Component/s: Sinks+Sources
• Assignee: Disabled imported user
• Reporter: Disabled imported user
• Created: 24/Jun/11 20:50
• Updated: 29/Jun/11 00:10
• Resolved: 29/Jun/11 00:10

Description

Some simple buffering should improve network performance quite a bit. I've so far implemented this only on the sending side (ThriftEventSink) and performance seems to have improved nicely, though I don't have good numbers for this yet. There are two possible solutions:

1) Use the native Thrift TFastFramedTransport (buffered version of TFramedTransport); lazy socket changes will need to be ported over (for TSaneServerSocket)
2) Wrap the existing code with a buffered version (see TBufferedTransport in patch)

I used the latter method for a simple test implementation only on the sending (sink) side.