Flume
  1. Flume
  2. FLUME-1957

FileChannel should print channel size when it stops

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: v1.4.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    1. FLUME-1957.patch
      1 kB
      Israel Ekpo
    2. FLUME-1957.2.patch
      2 kB
      Israel Ekpo

      Issue Links

        Activity

        Hide
        Roshan Naik added a comment -

        This should be the case with all channels actually. So its better to implement it in the base class.

        We should expose the channel size as part of the channel interface itself. Max channel capacity is another one that is worth exposing.

        Show
        Roshan Naik added a comment - This should be the case with all channels actually. So its better to implement it in the base class. We should expose the channel size as part of the channel interface itself. Max channel capacity is another one that is worth exposing.
        Hide
        Hari Shreedharan added a comment -

        I agree, we should do it in the ChannelCounter class.

        Show
        Hari Shreedharan added a comment - I agree, we should do it in the ChannelCounter class.
        Hide
        Israel Ekpo added a comment -

        I have a patch that overrides MonitoredCounterGroup.stop() as ChannelCounter.stop() and prints out the channel size and capacity.

        This method will be called in FileChannel, MemoryChannel etc.

        I will attach it shortly.

        Feedbacks are welcome.

        Show
        Israel Ekpo added a comment - I have a patch that overrides MonitoredCounterGroup.stop() as ChannelCounter.stop() and prints out the channel size and capacity. This method will be called in FileChannel, MemoryChannel etc. I will attach it shortly. Feedbacks are welcome.
        Hide
        Israel Ekpo added a comment -

        These changes were made against flume-1.4.0

        Show
        Israel Ekpo added a comment - These changes were made against flume-1.4.0
        Hide
        Hari Shreedharan added a comment -

        Thanks Israel. I will assign this jira to you, since you have already worked on this.

        Show
        Hari Shreedharan added a comment - Thanks Israel. I will assign this jira to you, since you have already worked on this.
        Hide
        Hari Shreedharan added a comment -

        Israel,

        Thanks for the patch! Generally looks good. I have a couple minor suggestions:

        • Could you please add an @Override to the stop() method?
        • In the logs printed out on stop (the channel size message), could you also add the name of the component. The name field is private in MonitoredCounterGroup - you can either make it protected or add a getter.
        Show
        Hari Shreedharan added a comment - Israel, Thanks for the patch! Generally looks good. I have a couple minor suggestions: Could you please add an @Override to the stop() method? In the logs printed out on stop (the channel size message), could you also add the name of the component. The name field is private in MonitoredCounterGroup - you can either make it protected or add a getter.
        Hide
        Mike Percy added a comment -

        Hey guys. Can we just dump the full metrics on shutdown? That would be much more useful.

        See also FLUME-1940

        Show
        Mike Percy added a comment - Hey guys. Can we just dump the full metrics on shutdown? That would be much more useful. See also FLUME-1940
        Hide
        Israel Ekpo added a comment -

        Mike,

        By full metrics are you referring to dumping all the key-value pairs available in the map org.apache.flume.instrumentation.MonitoredCounterGroup.counterMap?

        I can provide a patch for that if that is what you meant.

        The patch will do a loop through all the keys in that map and then print out the component type and name alongside the key and value for each entry in the map.

        In the time being, I will provide a patch based on Hari's recommendations.

        Show
        Israel Ekpo added a comment - Mike, By full metrics are you referring to dumping all the key-value pairs available in the map org.apache.flume.instrumentation.MonitoredCounterGroup.counterMap? I can provide a patch for that if that is what you meant. The patch will do a loop through all the keys in that map and then print out the component type and name alongside the key and value for each entry in the map. In the time being, I will provide a patch based on Hari's recommendations.
        Hide
        Hari Shreedharan added a comment -

        Israel, Mike,

        I don't mind having all the metrics dumped, but I filed this ticket only for this one specific metric. So I am ok with a patch that just dumps the channel size.

        Show
        Hari Shreedharan added a comment - Israel, Mike, I don't mind having all the metrics dumped, but I filed this ticket only for this one specific metric. So I am ok with a patch that just dumps the channel size.
        Hide
        Israel Ekpo added a comment -

        Mike, Hari:

        I have attached a patch for FLUME-1957 that addresses just Hari's original metrics.

        I attached a separate patch for FLUME-1940 on that JIRA issue.

        The patch contains logic that dumps all the metrics available from org.apache.flume.instrumentation.MonitoredCounterGroup.counterMap as well as the start and stop times registered in the class org.apache.flume.instrumentation.MonitoredCounterGroup.

        Show
        Israel Ekpo added a comment - Mike, Hari: I have attached a patch for FLUME-1957 that addresses just Hari's original metrics. I attached a separate patch for FLUME-1940 on that JIRA issue. The patch contains logic that dumps all the metrics available from org.apache.flume.instrumentation.MonitoredCounterGroup.counterMap as well as the start and stop times registered in the class org.apache.flume.instrumentation.MonitoredCounterGroup.
        Hide
        Israel Ekpo added a comment -

        This is been superceded by FLUME-1940.

        The patch has been uploaded for review

        https://reviews.apache.org/r/10416/

        Show
        Israel Ekpo added a comment - This is been superceded by FLUME-1940 . The patch has been uploaded for review https://reviews.apache.org/r/10416/
        Hide
        Mike Percy added a comment -

        Marking as dup of FLUME-1940

        Show
        Mike Percy added a comment - Marking as dup of FLUME-1940

          People

          • Assignee:
            Israel Ekpo
            Reporter:
            Hari Shreedharan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Due:
              Created:
              Updated:
              Resolved:

              Development