Flume
  1. Flume
  2. FLUME-2217

Preserve priority, timestamp and hostname fields in MultiportSyslogTcp and Udp sources

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.5.0
    • Fix Version/s: v1.5.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      Flume-1666 added the ability to preserve timestamp and hostname fields of a syslog message. We should also add this property to the MultiportSyslogTcp Source and the SyslogUdp sources.

      1. FLUME-2217.6.patch
        30 kB
        Jeff Lord
      2. FLUME-2217.3.patch
        28 kB
        Jeff Lord
      3. FLUME-2217.2.patch
        23 kB
        Jeff Lord
      4. FLUME-2217.1.patch
        23 kB
        Jeff Lord

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          SUCCESS: Integrated in flume-trunk #525 (See https://builds.apache.org/job/flume-trunk/525/)
          FLUME-2217. Add option to preserve all Syslog headers in syslog sources (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=9790ca7587060285efa4ae64591cea17dd3f00cf)

          • flume-ng-core/src/main/java/org/apache/flume/source/SyslogUtils.java
          • flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java
          • flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogParser.java
          • flume-ng-doc/sphinx/FlumeUserGuide.rst
          • flume-ng-core/src/test/java/org/apache/flume/source/TestMultiportSyslogTCPSource.java
          • flume-ng-core/src/main/java/org/apache/flume/source/SyslogParser.java
          • flume-ng-core/src/main/java/org/apache/flume/source/MultiportSyslogTCPSource.java
          • flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java
          • flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUtils.java
          • flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java
          • flume-ng-core/src/main/java/org/apache/flume/source/SyslogUDPSource.java
          Show
          Hudson added a comment - SUCCESS: Integrated in flume-trunk #525 (See https://builds.apache.org/job/flume-trunk/525/ ) FLUME-2217 . Add option to preserve all Syslog headers in syslog sources (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=9790ca7587060285efa4ae64591cea17dd3f00cf ) flume-ng-core/src/main/java/org/apache/flume/source/SyslogUtils.java flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogParser.java flume-ng-doc/sphinx/FlumeUserGuide.rst flume-ng-core/src/test/java/org/apache/flume/source/TestMultiportSyslogTCPSource.java flume-ng-core/src/main/java/org/apache/flume/source/SyslogParser.java flume-ng-core/src/main/java/org/apache/flume/source/MultiportSyslogTCPSource.java flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUdpSource.java flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogUtils.java flume-ng-core/src/test/java/org/apache/flume/source/TestSyslogTcpSource.java flume-ng-core/src/main/java/org/apache/flume/source/SyslogUDPSource.java
          Hide
          Mike Percy added a comment -

          Pushed to trunk and flume-1.5 branches. Thanks for the patch Jeff!

          Show
          Mike Percy added a comment - Pushed to trunk and flume-1.5 branches. Thanks for the patch Jeff!
          Hide
          Mike Percy added a comment -

          +1 LGTM. I am going to commit this soon.

          Show
          Mike Percy added a comment - +1 LGTM. I am going to commit this soon.
          Hide
          Jeff Lord added a comment -

          Removed baosRaw and extra byte array.
          Modified the initial parsing to include Priority e.g. <10>
          Modified the regex to account for this and we should be good now.
          Was able to build successfully.
          Please let me know if there is anything else I can do to get this committed.

          Show
          Jeff Lord added a comment - Removed baosRaw and extra byte array. Modified the initial parsing to include Priority e.g. <10> Modified the regex to account for this and we should be good now. Was able to build successfully. Please let me know if there is anything else I can do to get this committed.
          Hide
          Jeff Lord added a comment -

          Rev 2 based on feedbak from Mike Percy on reviewboard.

          Show
          Jeff Lord added a comment - Rev 2 based on feedbak from Mike Percy on reviewboard.
          Hide
          Jeff Lord added a comment -

          Here is a first pass at this.
          Please not that the functionality of all 3 sources was modified slightly such that with this patch we will now preserve the syslog priority as well as the timestamp and header. e.g.

          <10>2013-10-31T17:36:27.381-07:00 localhost.localdomain test UDP syslog data

          Show
          Jeff Lord added a comment - Here is a first pass at this. Please not that the functionality of all 3 sources was modified slightly such that with this patch we will now preserve the syslog priority as well as the timestamp and header. e.g. <10>2013-10-31T17:36:27.381-07:00 localhost.localdomain test UDP syslog data

            People

            • Assignee:
              Jeff Lord
              Reporter:
              Jeff Lord
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development