-
Type:
Bug
-
Status: Patch Available
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 1.6.0, 1.7.0
-
Fix Version/s: None
-
Component/s: Sinks+Sources
-
Labels:None
Our usecase has multi agent flow and we are achieving this with thrift source and thrift sink combination. When there is a surge in incoming traffic downstream agent having thrift source is running out of memory. On further investigation of thrift transport/server code and scribe source found that we may need to fix the thrift transport's maxReadBufferBytes. Similar issue has been fixed in ScribeSource as part of FLUME-2404 by making this buffer configurable and capping it to a 16MB if not configured.
thrift code (org.apache.thrift.server.AbstractNonblockingServer)
public abstract static class AbstractNonblockingServerArgs<T extends AbstractNonblockingServer.AbstractNonblockingServerArgs<T>> extends AbstractServerArgs<T> { public long maxReadBufferBytes = 9223372036854775807L; public AbstractNonblockingServerArgs(TNonblockingServerTransport transport) { super(transport); this.transportFactory(new Factory()); } }
stacktrace
31 Aug 2016 12:52:24 ERROR [Thread-12] - run() exiting due to uncaught error java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1357) at org.apache.thrift.server.TThreadedSelectorServer.requestInvoke(TThreadedSelectorServer.java:310) at org.apache.thrift.server.AbstractNonblockingServer$AbstractSelectThread.handleRead(AbstractNonblockingServer.java:209) at org.apache.thrift.server.TThreadedSelectorServer$SelectorThread.select(TThreadedSelectorServer.java:576) at org.apache.thrift.server.TThreadedSelectorServer$SelectorThread.run(TThreadedSelectorServer.java:536)
- is related to
-
FLUME-2405 Flume stops working on arbitrary Frame Size more than 16 MB on ScribeSource
-
- Resolved
-
-
FLUME-2404 Default maxReadBufferBytes might cause OOM and cause scribe source exit
-
- Resolved
-
- links to