Flume
  1. Flume
  2. FLUME-1096

Add support to pass headers through AvroCLIClient

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: v1.2.0
    • Fix Version/s: v1.2.0
    • Component/s: None
    • Labels:

      Description

      Configured HDFSEvent Sink to generate file path's using regex based date format.

      Config for hdfs-sink -> collector1.sinks.hdfs-sink1.hdfs.filePrefix = flume/%D
      Hit NumberFormatException in BucketPath.replaceShorthand() as it reads timestamp from headers

      Date date = new Date(Long.valueOf(headers.get("timestamp")));

      java.lang.NumberFormatException: null
      at java.lang.Long.parseLong(Long.java:375)
      at java.lang.Long.valueOf(Long.java:525)
      at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:185)
      at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:219)
      at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:341)
      at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:65)
      at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:148)
      at java.lang.Thread.run(Thread.java:680)

      Sending the events using - ./flume-ng avro-client --conf ../conf/ -H localhost -p 41415 -F ~/.bash_profile

      Issue - AvroCLiClient isn't adding timestamp in headers

      1. FLUME-1096-2.patch
        3 kB
        Inder SIngh
      2. FLUME-1096-3.patch
        4 kB
        Inder SIngh
      3. FLUME-1096-4.patch
        4 kB
        Inder SIngh
      4. FLUME-1096-7.patch
        4 kB
        Inder SIngh

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Inder SIngh
            Reporter:
            Inder SIngh
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development