[FLUME-3365] flume high rss memeory occupied

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.6.0
• Component/s: Sinks+Sources

Environment

UbuntuDescription: Ubuntu 16.04.6 LTSRelease: 16.04Codename: xenial

java_env:1.8.0_171

 

jvm options:

/usr/bin/java -Xms200m -Xmx200m -Dflume.root.logger=info,console -Dflume.monitoring.type=http -Dflume.monitoring.port=1235 -XX:NativeMemoryTracking=detail

• Assignee: Unassigned
• Reporter: yuqibin
• Created: 09/May/20 06:15
• Updated: 09/May/20 06:15

Description

We use flume-client (taildir-source and avro-sink) to collect machine's application logs to another flume B (all collection),

Flume B send events to kafka.

With a long runtime . we found  flume-client occupied rss memory large than memory by used -Xmx 

we think its may cause System OOMKiller to kill another application (which had occupied the largest memory)

 

 Why flume can occupied high rss memory with a long runtime?

We use pmap and NativeMemoryTracking. but the amount of them not equal. emmm