Flume
  1. Flume
  2. FLUME-1770

Flume should have serializer which supports serializer the headers to a simple string

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: v1.3.0
    • Fix Version/s: v1.4.0
    • Component/s: None
    • Labels:

      Description

      BodyTextserializer ignores the headers where as AvroEventserializer writes out an Avro file. It would be nice if we had a simple one which write out the headers prefixed the line of text.

        Activity

        Hide
        Thom DeCarlo added a comment -

        This new file lives in flume-parent/flume-ng-core/src/main/java/org/apache/flume/serialization.
        It provides a serializer which outputs the contents of the headers as well as the body of the message.

        Show
        Thom DeCarlo added a comment - This new file lives in flume-parent/flume-ng-core/src/main/java/org/apache/flume/serialization. It provides a serializer which outputs the contents of the headers as well as the body of the message.
        Hide
        Thom DeCarlo added a comment -

        This patch adds a serializer that output the contents of the event headers as well as the body of the message.

        Show
        Thom DeCarlo added a comment - This patch adds a serializer that output the contents of the event headers as well as the body of the message.
        Hide
        Brock Noland added a comment -

        Nice work! I'll take a look at this today.

        Show
        Brock Noland added a comment - Nice work! I'll take a look at this today.
        Hide
        Brock Noland added a comment -

        Hi,

        Looks like the portion to TestBodyTextEventSerializer doesn't apply? Perhaps you could rebase on trunk?

        patching file flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializerType.java
        Hunk #1 succeeded at 25 with fuzz 1 (offset 5 lines).
        patching file flume-ng-core/src/main/java/org/apache/flume/serialization/HeaderAndBodyTextEventSerializer.java
        patching file flume-ng-core/src/test/java/org/apache/flume/serialization/TestBodyTextEventSerializer.java
        Hunk #1 FAILED at 57.
        1 out of 1 hunk FAILED -- saving rejects to file flume-ng-core/src/test/java/org/apache/flume/serialization/TestBodyTextEventSerializer.java.rej
        patching file flume-ng-core/src/test/java/org/apache/flume/serialization/TestHeaderAndBodyTextEventSerializer.java
        
        Show
        Brock Noland added a comment - Hi, Looks like the portion to TestBodyTextEventSerializer doesn't apply? Perhaps you could rebase on trunk? patching file flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializerType.java Hunk #1 succeeded at 25 with fuzz 1 (offset 5 lines). patching file flume-ng-core/src/main/java/org/apache/flume/serialization/HeaderAndBodyTextEventSerializer.java patching file flume-ng-core/src/test/java/org/apache/flume/serialization/TestBodyTextEventSerializer.java Hunk #1 FAILED at 57. 1 out of 1 hunk FAILED -- saving rejects to file flume-ng-core/src/test/java/org/apache/flume/serialization/TestBodyTextEventSerializer.java.rej patching file flume-ng-core/src/test/java/org/apache/flume/serialization/TestHeaderAndBodyTextEventSerializer.java
        Hide
        Thom DeCarlo added a comment -

        Hmmm... I'm still new to this "git" thingy. How do I rebase? When I try "git rebase remotes/origin/trunk" it asks about deleting everything that I changed. I don't think that's what I want to do.

        Show
        Thom DeCarlo added a comment - Hmmm... I'm still new to this "git" thingy. How do I rebase? When I try "git rebase remotes/origin/trunk" it asks about deleting everything that I changed. I don't think that's what I want to do.
        Hide
        Brock Noland added a comment - - edited

        What I would do is do the work on a seperate branch so:

        git checkout trunk
        git checkout -b FLUME-1770
        # do work
        git commit -m ""
        git diff trunk > FLUME-1770-0.patch
        
        # then assume there is a revsion
        git checkout FLUME-1770
        git fetch origin
        git merge trunk
        # do work
        git commit -m ""
        git diff trunk > FLUME-1770-0.patch
        

        so in your case, I am not sure if you have a local branch or not?

        but what you could do, assuming a clean trunk is:

        git checkout trunk
        git fetch origin
        git merge origin/trunk
        git checkout -b FLUME-1770
        patch -p0 < FLUME-1770.patch (normally you'd use git apply here but the patch doesn't work with it)
        # fix any problems
        git commit -m ""
        git diff trunk > FLUME-1770-1.patch
        
        Show
        Brock Noland added a comment - - edited What I would do is do the work on a seperate branch so: git checkout trunk git checkout -b FLUME-1770 # do work git commit -m "" git diff trunk > FLUME-1770-0.patch # then assume there is a revsion git checkout FLUME-1770 git fetch origin git merge trunk # do work git commit -m "" git diff trunk > FLUME-1770-0.patch so in your case, I am not sure if you have a local branch or not? but what you could do, assuming a clean trunk is: git checkout trunk git fetch origin git merge origin/trunk git checkout -b FLUME-1770 patch -p0 < FLUME-1770.patch (normally you'd use git apply here but the patch doesn't work with it) # fix any problems git commit -m "" git diff trunk > FLUME-1770-1.patch
        Hide
        Thom DeCarlo added a comment -

        Ok, I think I have updated this patch properly.

        Show
        Thom DeCarlo added a comment - Ok, I think I have updated this patch properly.
        Hide
        Brock Noland added a comment -

        Nice work Thom! I have committed this to trunk and 1.4!

        Show
        Brock Noland added a comment - Nice work Thom! I have committed this to trunk and 1.4!
        Hide
        Hudson added a comment -

        Integrated in flume-trunk #345 (See https://builds.apache.org/job/flume-trunk/345/)
        FLUME-1770: Flume should have serializer which supports serializer the headers to a simple string (Revision 1675d49a217a81753c67f48b56de543934ce117b)

        Result = SUCCESS
        brock : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=1675d49a217a81753c67f48b56de543934ce117b
        Files :

        • flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializerType.java
        • flume-ng-core/src/test/java/org/apache/flume/serialization/TestHeaderAndBodyTextEventSerializer.java
        • flume-ng-core/src/main/java/org/apache/flume/serialization/HeaderAndBodyTextEventSerializer.java
        • flume-ng-core/src/test/java/org/apache/flume/serialization/TestBodyTextEventSerializer.java
        Show
        Hudson added a comment - Integrated in flume-trunk #345 (See https://builds.apache.org/job/flume-trunk/345/ ) FLUME-1770 : Flume should have serializer which supports serializer the headers to a simple string (Revision 1675d49a217a81753c67f48b56de543934ce117b) Result = SUCCESS brock : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=1675d49a217a81753c67f48b56de543934ce117b Files : flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializerType.java flume-ng-core/src/test/java/org/apache/flume/serialization/TestHeaderAndBodyTextEventSerializer.java flume-ng-core/src/main/java/org/apache/flume/serialization/HeaderAndBodyTextEventSerializer.java flume-ng-core/src/test/java/org/apache/flume/serialization/TestBodyTextEventSerializer.java

          People

          • Assignee:
            Thom DeCarlo
            Reporter:
            Brock Noland
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development