Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2171

Add Interceptor to remove headers from event

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.8.0
    • Component/s: Easy

      Description

      I found Flume OG's decorators to handle event headers useful and some to be missing from Flume NG. More specifically, we could have an interceptor to remove headers from an event.

      1. FLUME-2171-0.patch
        20 kB
        Gabriel Commeau

        Issue Links

          Activity

          Hide
          jholoman Jeff Holoman added a comment -

          Committed to trunk.

          Thank you for your contributions Gabriel Commeau and Bessenyei Balázs Donát.

          Thanks Denes Arvay and Attila Simon for the reviews.

          Show
          jholoman Jeff Holoman added a comment - Committed to trunk. Thank you for your contributions Gabriel Commeau and Bessenyei Balázs Donát . Thanks Denes Arvay and Attila Simon for the reviews.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Flume-trunk-hbase-1 #229 (See https://builds.apache.org/job/Flume-trunk-hbase-1/229/)
          FLUME-2171. Add Interceptor to remove headers from event (jholoman: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=de941e7b85ed55bfbd804880f421be3889187d31)

          • (edit) flume-ng-doc/sphinx/FlumeUserGuide.rst
          • (edit) flume-ng-core/src/main/java/org/apache/flume/interceptor/InterceptorType.java
          • (add) flume-ng-core/src/main/java/org/apache/flume/interceptor/RemoveHeaderInterceptor.java
          • (add) flume-ng-core/src/test/java/org/apache/flume/interceptor/RemoveHeaderInterceptorTest.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Flume-trunk-hbase-1 #229 (See https://builds.apache.org/job/Flume-trunk-hbase-1/229/ ) FLUME-2171 . Add Interceptor to remove headers from event (jholoman: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=de941e7b85ed55bfbd804880f421be3889187d31 ) (edit) flume-ng-doc/sphinx/FlumeUserGuide.rst (edit) flume-ng-core/src/main/java/org/apache/flume/interceptor/InterceptorType.java (add) flume-ng-core/src/main/java/org/apache/flume/interceptor/RemoveHeaderInterceptor.java (add) flume-ng-core/src/test/java/org/apache/flume/interceptor/RemoveHeaderInterceptorTest.java
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit de941e7b85ed55bfbd804880f421be3889187d31 in flume's branch refs/heads/trunk from Gabriel Commeau
          [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=de941e7 ]

          FLUME-2171. Add Interceptor to remove headers from event

          Similar to Flume OG decorators, this introduces an Interceptor to remove headers from an event.

          Reviewers: Bessenyei Balázs Donát, Jeff Holoman, Denes Arvay, Attila Simon

          (Gabriel Commeau and Bessenyei Balázs Donát via Jeff Holoman)

          Show
          jira-bot ASF subversion and git services added a comment - Commit de941e7b85ed55bfbd804880f421be3889187d31 in flume's branch refs/heads/trunk from Gabriel Commeau [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=de941e7 ] FLUME-2171 . Add Interceptor to remove headers from event Similar to Flume OG decorators, this introduces an Interceptor to remove headers from an event. Reviewers: Bessenyei Balázs Donát, Jeff Holoman, Denes Arvay, Attila Simon (Gabriel Commeau and Bessenyei Balázs Donát via Jeff Holoman)
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          Jeff Holoman: thank you for your comments!

          I have updated the change. (I could not resolve both issues you have opened.)

          Anyways, if you think it is to be committed, I can also do that.

          Show
          bessbd Bessenyei Balázs Donát added a comment - Jeff Holoman : thank you for your comments! I have updated the change. (I could not resolve both issues you have opened.) Anyways, if you think it is to be committed, I can also do that.
          Hide
          jholoman Jeff Holoman added a comment -

          I added some minor comments. After you've had a chance to take a look I can commit this either today or tomorrow.

          Thanks

          Jeff

          Show
          jholoman Jeff Holoman added a comment - I added some minor comments. After you've had a chance to take a look I can commit this either today or tomorrow. Thanks Jeff
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          Jeff Holoman, Denes Arvay: I've updated reviewboard. Please, review and let me know what you think.

          Show
          bessbd Bessenyei Balázs Donát added a comment - Jeff Holoman , Denes Arvay : I've updated reviewboard. Please, review and let me know what you think.
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          Jeff Holoman: thank you for that!

          Denes Arvay: I'll fix those two open issues in the next few days. When Jeff Holoman agrees, I'll commit this change.
          Is that okay for you?

          Show
          bessbd Bessenyei Balázs Donát added a comment - Jeff Holoman : thank you for that! Denes Arvay : I'll fix those two open issues in the next few days. When Jeff Holoman agrees, I'll commit this change. Is that okay for you?
          Hide
          jholoman Jeff Holoman added a comment -

          I will do a review of this over the next couple of days.

          Thanks

          Jeff

          Show
          jholoman Jeff Holoman added a comment - I will do a review of this over the next couple of days. Thanks Jeff
          Hide
          denes Denes Arvay added a comment -

          Bessenyei Balázs Donát, could you please have a look on this ticket? There's only one minor comment on the reviewboard, could you please fix and commit it? Thanks in advance.

          Show
          denes Denes Arvay added a comment - Bessenyei Balázs Donát , could you please have a look on this ticket? There's only one minor comment on the reviewboard, could you please fix and commit it? Thanks in advance.
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          Thank you for the review, Denes Arvay.

          I'm looking forward to further suggestions if anyone has some.

          Show
          bessbd Bessenyei Balázs Donát added a comment - Thank you for the review, Denes Arvay . I'm looking forward to further suggestions if anyone has some.
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          I've created a new reviewboard entry at https://reviews.apache.org/r/51244/diff/1/#index_header

          Please, review

          Show
          bessbd Bessenyei Balázs Donát added a comment - I've created a new reviewboard entry at https://reviews.apache.org/r/51244/diff/1/#index_header Please, review
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          As there has been no response, I'll reassign this ticket to myself, rebase it on trunk, then create a reviewboard request.

          Show
          bessbd Bessenyei Balázs Donát added a comment - As there has been no response, I'll reassign this ticket to myself, rebase it on trunk, then create a reviewboard request.
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          Gabriel Commeau: can you please create a reviewboard request for this patch?

          Show
          bessbd Bessenyei Balázs Donát added a comment - Gabriel Commeau : can you please create a reviewboard request for this patch?
          Hide
          CHorrocks Chris Horrocks added a comment -

          This would be incredibly helpful considering the lack of flexibility in some of the sources with adding their own headers which may conflict with others further downstream. I'll try the patch out in our test environment later today and push Cloudera to backport it.

          Show
          CHorrocks Chris Horrocks added a comment - This would be incredibly helpful considering the lack of flexibility in some of the sources with adding their own headers which may conflict with others further downstream. I'll try the patch out in our test environment later today and push Cloudera to backport it.
          Hide
          jmspaggi Jean-Marc Spaggiari added a comment -

          This will have helped me a lot today if this will have been commited in 2013

          Show
          jmspaggi Jean-Marc Spaggiari added a comment - This will have helped me a lot today if this will have been commited in 2013
          Hide
          gcommeau Gabriel Commeau added a comment -

          The patch contains the interceptor, the corresponding JUnit test case and the update to the user guide.

          Show
          gcommeau Gabriel Commeau added a comment - The patch contains the interceptor, the corresponding JUnit test case and the update to the user guide.
          Hide
          mpercy Mike Percy added a comment -

          To submit the patch, go to More Actions > Attach Files. Once you're ready for a review, click Submit Patch.

          Show
          mpercy Mike Percy added a comment - To submit the patch, go to More Actions > Attach Files. Once you're ready for a review, click Submit Patch.
          Hide
          mpercy Mike Percy added a comment -

          Thanks for the patch Gabriel. I went ahead and added you to the Contributors list in JIRA so you can submit the patch and assign to yourself.

          Show
          mpercy Mike Percy added a comment - Thanks for the patch Gabriel. I went ahead and added you to the Contributors list in JIRA so you can submit the patch and assign to yourself.
          Hide
          gcommeau Gabriel Commeau added a comment -

          I've got a patch ready for this, but I think I don't have the permissions to submit it (I can't assign this ticket to myself).

          Show
          gcommeau Gabriel Commeau added a comment - I've got a patch ready for this, but I think I don't have the permissions to submit it (I can't assign this ticket to myself).

            People

            • Assignee:
              bessbd Bessenyei Balázs Donát
              Reporter:
              gcommeau Gabriel Commeau
            • Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development