Flume
  1. Flume
  2. FLUME-1666

Syslog source strips timestamp and hostname from log message body

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.2.0, v1.3.0
    • Fix Version/s: v1.5.0
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Environment:

      This occurs with Flume all the way up through 1.3.0.

      Description

      The syslog source parses incoming syslog messages. In the process, it strips the timestamp and hostname from each log message, and places them as Event headers.

      Thus, a syslog message that would normally look like so (when written via rsyslog or syslogd):

      Wed Oct 24 09:18:01 UTC 2012 someserver /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/somescript)
      

      Appears in flume output as:

      /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/somescript)
      
      1. FLUME-1666-4.patch
        14 kB
        Jeff Lord
      2. FLUME-1666-3.patch
        14 kB
        Jeff Lord
      3. FLUME-1666-2.patch
        14 kB
        Jeff Lord
      4. FLUME-1666-1.patch
        13 kB
        Jeff Lord
      5. FLUME-1666-SyslogTextSerializer.patch
        4 kB
        Josh West

        Issue Links

          Activity

          Mike Percy made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s v1.5.0 [ 12324642 ]
          Resolution Fixed [ 1 ]
          Jeff Lord made changes -
          Attachment FLUME-1666-4.patch [ 12607715 ]
          Jeff Lord made changes -
          Attachment FLUME-1666-3.patch [ 12607714 ]
          Jeff Lord made changes -
          Attachment FLUME-1666-2.patch [ 12607098 ]
          Jeff Lord made changes -
          Remote Link This issue links to "ReviewBoard (Web Link)" [ 12801 ]
          Jeff Lord made changes -
          Attachment FLUME-1666-1.patch [ 12606234 ]
          Jeff Lord made changes -
          Assignee Jeff Lord [ jlord ]
          Josh West made changes -
          Fix Version/s v1.3.0 [ 12322140 ]
          Josh West made changes -
          Attachment FLUME-1666-SyslogTextSerializer.patch [ 12550616 ]
          Josh West made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s v1.3.0 [ 12322140 ]
          Josh West made changes -
          Description The syslog source parses incoming syslog messages. In the process, it strips the timestamp and hostname from each log message, and places them as Event headers.

          Thus, a syslog message that would normally look like so (when written via rsyslog or syslogd):

          {noformat}
          Wed Oct 24 09:18:01 UTC 2012 someserver /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/somescript)
          {noformat}

          Appears in flume output as:

          {noformat}
          /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/varnish_log_monitor)
          {noformat}
          The syslog source parses incoming syslog messages. In the process, it strips the timestamp and hostname from each log message, and places them as Event headers.

          Thus, a syslog message that would normally look like so (when written via rsyslog or syslogd):

          {noformat}
          Wed Oct 24 09:18:01 UTC 2012 someserver /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/somescript)
          {noformat}

          Appears in flume output as:

          {noformat}
          /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/somescript)
          {noformat}
          Josh West made changes -
          Field Original Value New Value
          Description The syslog source parses incoming syslog messages. In the process, it strips the timestamp and hostname from each log message, and places them as Event headers.

          Thus, a syslog message that would normally look like so (when written via rsyslog or syslogd):


          Wed Oct 24 09:18:01 UTC 2012 someserver /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/somescript)

          Appears in flume output as:

          /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/varnish_log_monitor)
          The syslog source parses incoming syslog messages. In the process, it strips the timestamp and hostname from each log message, and places them as Event headers.

          Thus, a syslog message that would normally look like so (when written via rsyslog or syslogd):

          {noformat}
          Wed Oct 24 09:18:01 UTC 2012 someserver /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/somescript)
          {noformat}

          Appears in flume output as:

          {noformat}
          /USR/SBIN/CRON[26981]: (root) CMD (/usr/local/sbin/varnish_log_monitor)
          {noformat}
          Josh West created issue -

            People

            • Assignee:
              Jeff Lord
              Reporter:
              Josh West
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development