[FLUME-1484] Flume support throughput in Agent, Source, Sink level at JMX

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.2.0
• Component/s: Node, Sinks+Sources
• Assignee: Theodore michael Malaska
• Reporter: Denny Ye
• Created: 15/Aug/12 03:30
• Updated: 16/Sep/12 01:22

Description

From user's view of point, we would like to know the current throughput from one of monitoring tools. WebUI is best, of course. JMX is simple way to implement throughput monitoring.

Agent should have input and output throughput based on several Sources and Sinks.

Here is just simple code in my environment to monitoring throughput of Source.

ThroughputCounter.java

import java.util.concurrent.atomic.AtomicInteger;

import org.apache.flume.instrumentation.SourceCounter;

public class ThroughputCounter {
  private volatile boolean isRunning;	  
  private AtomicInteger cache = new AtomicInteger();
	  
  SourceCounter sourceCounter;
  public ThroughputCounter(SourceCounter sourceCounter) {
    this.sourceCounter = sourceCounter;
  }
	  
  public void start() {
    isRunning = true;
	    
    Counter counter = new Counter();
    counter.start();
  }
	  
  public void stop() {
    isRunning = false;
  }
	  
	  
  public void addWriteBytes(int bytes) {
    cache.getAndAdd(bytes);
  }
	  
  private class Counter extends Thread {
	    
    Counter() {
      super("ThroughputCounterThread");	
    }
		
    public void run() {
      while (isRunning) {
	try {
	  Thread.sleep(1000);
	  sourceCounter.incrementSourceThroughput(
cache.getAndSet(0));
	} catch (Exception e) {
	  e.printStackTrace();
	}
      }
    }
  }
	  
}

Issue Links

• is related to: FLUME-1411 Add average events per second params to MBeans
• links to: review board