Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1419

Using system time if 'timestamp' property is absent in event header

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.3.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      If we want to use pattern for generating HDFS path but the 'timestamp' property is absent in event header, it may cause the exception...

      Event may comes from several Sources and doesn't use Interceptor(pupulate 'timestamp' property). Thus, event may be haven't this property normally.
      Use the local system time to replace it.

      java.lang.RuntimeException: Flume wasn't able to parse timestamp header in the event to resolve time based bucketing. Please check that you're correctly populating timestamp header (for example using TimestampInterceptor source interceptor).
      at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:149)
      at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:318)
      at org.apache.flume.formatter.output.TestBucketPath.testDateFormatHours(TestBucketPath.java:46)
      Caused by: java.lang.NumberFormatException: null
      at java.lang.Long.parseLong(Unknown Source)
      at java.lang.Long.valueOf(Unknown Source)
      at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:147)
      ... 26 more

        Attachments

          Activity

            People

            • Assignee:
              dennyy Denny Ye
              Reporter:
              dennyy Denny Ye
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: