Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2275

Improve scalability of MorphlineInterceptor under contention

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.4.1, 1.5.0
    • Component/s: Sinks+Sources
    • Labels:
      None
    1. FLUME-2275-v3.patch
      2 kB
      wolfgang hoschek
    2. FLUME-2275-v4.patch
      2 kB
      wolfgang hoschek

      Activity

      Hide
      whoschek wolfgang hoschek added a comment -

      Patch is attached.

      Show
      whoschek wolfgang hoschek added a comment - Patch is attached.
      Hide
      hshreedharan Hari Shreedharan added a comment -

      This line causes a compiler warning that can be fixed pretty easily by using new ConcurrentLinkedQueue<LocalMorphlineInterceptor>:

      private final ConcurrentLinkedQueue<LocalMorphlineInterceptor> pool = new ConcurrentLinkedQueue();
      

      Also, we should uses interfaces for instances than concrete implementations - so we should use Queue<LocalMorphlineInterceptor> pool = ...

      Show
      hshreedharan Hari Shreedharan added a comment - This line causes a compiler warning that can be fixed pretty easily by using new ConcurrentLinkedQueue<LocalMorphlineInterceptor>: private final ConcurrentLinkedQueue<LocalMorphlineInterceptor> pool = new ConcurrentLinkedQueue(); Also, we should uses interfaces for instances than concrete implementations - so we should use Queue<LocalMorphlineInterceptor> pool = ...
      Hide
      whoschek wolfgang hoschek added a comment -

      Sure, +1 for adding those suggestions, or do you want me send another corresponding patch?

      Show
      whoschek wolfgang hoschek added a comment - Sure, +1 for adding those suggestions, or do you want me send another corresponding patch?
      Hide
      hshreedharan Hari Shreedharan added a comment -

      Please go ahead and submit the patch

      Show
      hshreedharan Hari Shreedharan added a comment - Please go ahead and submit the patch
      Hide
      whoschek wolfgang hoschek added a comment -

      revised patch according to Hari's suggestions

      Show
      whoschek wolfgang hoschek added a comment - revised patch according to Hari's suggestions
      Hide
      hshreedharan Hari Shreedharan added a comment -

      +1. Committing this.

      Show
      hshreedharan Hari Shreedharan added a comment - +1. Committing this.
      Hide
      hshreedharan Hari Shreedharan added a comment -

      Committed, rev: c3a9c80ab431f6ba670142c7ce6813692422f764. Thanks Wolfgang!

      Show
      hshreedharan Hari Shreedharan added a comment - Committed, rev: c3a9c80ab431f6ba670142c7ce6813692422f764. Thanks Wolfgang!
      Hide
      hudson Hudson added a comment -

      SUCCESS: Integrated in flume-trunk #535 (See https://builds.apache.org/job/flume-trunk/535/)
      FLUME-2275. Improve scalability of MorphlineInterceptor under contention (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c3a9c80ab431f6ba670142c7ce6813692422f764)

      • flume-ng-sinks/flume-ng-morphline-solr-sink/src/main/java/org/apache/flume/sink/solr/morphline/MorphlineInterceptor.java
      Show
      hudson Hudson added a comment - SUCCESS: Integrated in flume-trunk #535 (See https://builds.apache.org/job/flume-trunk/535/ ) FLUME-2275 . Improve scalability of MorphlineInterceptor under contention (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c3a9c80ab431f6ba670142c7ce6813692422f764 ) flume-ng-sinks/flume-ng-morphline-solr-sink/src/main/java/org/apache/flume/sink/solr/morphline/MorphlineInterceptor.java

        People

        • Assignee:
          whoschek wolfgang hoschek
          Reporter:
          whoschek wolfgang hoschek
        • Votes:
          0 Vote for this issue
          Watchers:
          4 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development