Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1938

Flume UDP Source for Windows Events being trimmed at \n characters by SyslogUtils.extractEvent() function

    Details

    • Type: New Feature
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.3.1
    • Fix Version/s: None
    • Labels:
    • Environment:

      Ubuntu/Linux

    • Release Note:
      Hide
      This feature can be disabled if causes any issues with headers or header extraction process in flume syslog sources (both TCP and UDP).

      To enable multi-line configuration set the 'body.multiline' source context configuration to true e.g.

      agent.sources.syslogudp.body.multiline=true
      Show
      This feature can be disabled if causes any issues with headers or header extraction process in flume syslog sources (both TCP and UDP). To enable multi-line configuration set the 'body.multiline' source context configuration to true e.g. agent.sources.syslogudp.body.multiline=true

      Description

      Flume trims UDP syslog events originating from Kiwi Windows Event Log Forwarder at the first new-line character.

      I would like to contribute with a patch to allow configurable inclusion of new-line characters for SyslogUDP source in Flume

      Background:
      I am using Solarwinds/Kiwi Syslog's Windows Event Log Forwarder. Ref: http://www.solarwinds.com/kiwi-syslog-server.aspx

      The Log Forwarder sends CR+LF & horizontal tab characters in the UDP/TCP syslog events that it sends. Other than Kiwi Syslog this problem exists for syslog-ng on certain linux syslog events where CR+LF characters are sent in syslog events.

      Syslog-ng handles this problem by replacing these characters with \011 and \015\012 but Flume has no such charset customization available for SyslogUDP. SyslogTCP does have charset customization along with character replace functionality for header.

        Attachments

        1. SyslogSourceConfigurationConstants.java
          2 kB
          Ambud Sharma
        2. SyslogUDPSource.java
          5 kB
          Ambud Sharma
        3. SyslogUtils.java
          13 kB
          Ambud Sharma
        4. TestSyslogUtils.java
          16 kB
          Ambud Sharma
        5. FLUME-1938.patch
          8 kB
          Ambud Sharma
        6. FLUME-1938.patch-README.txt
          0.2 kB
          Ambud Sharma

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ashar103 Ambud Sharma
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 120h
                120h
                Remaining:
                Remaining Estimate - 120h
                120h
                Logged:
                Time Spent - Not Specified
                Not Specified