Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1484

Flume support throughput in Agent, Source, Sink level at JMX

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.2.0
    • None
    • Node, Sinks+Sources
    • None

    Description

      From user's view of point, we would like to know the current throughput from one of monitoring tools. WebUI is best, of course. JMX is simple way to implement throughput monitoring.

      Agent should have input and output throughput based on several Sources and Sinks.

      Here is just simple code in my environment to monitoring throughput of Source.

      ThroughputCounter.java
      import java.util.concurrent.atomic.AtomicInteger;
      
      import org.apache.flume.instrumentation.SourceCounter;
      
      public class ThroughputCounter {
        private volatile boolean isRunning;	  
        private AtomicInteger cache = new AtomicInteger();
      	  
        SourceCounter sourceCounter;
        public ThroughputCounter(SourceCounter sourceCounter) {
          this.sourceCounter = sourceCounter;
        }
      	  
        public void start() {
          isRunning = true;
      	    
          Counter counter = new Counter();
          counter.start();
        }
      	  
        public void stop() {
          isRunning = false;
        }
      	  
      	  
        public void addWriteBytes(int bytes) {
          cache.getAndAdd(bytes);
        }
      	  
        private class Counter extends Thread {
      	    
          Counter() {
            super("ThroughputCounterThread");	
          }
      		
          public void run() {
            while (isRunning) {
      	try {
      	  Thread.sleep(1000);
      	  sourceCounter.incrementSourceThroughput(
      cache.getAndSet(0));
      	} catch (Exception e) {
      	  e.printStackTrace();
      	}
            }
          }
        }
      	  
      }
      

      Attachments

        1. FLUME-1411-IDEA.patch
          9 kB
          Theodore michael Malaska

        Issue Links

          Activity

            People

              ted.m Theodore michael Malaska
              dennyy Denny Ye
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: