Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-3076

property flumeBatchSize in kafka sink confused

    XMLWordPrintableJSON

    Details

    • Type: Documentation
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 1.7.0
    • Fix Version/s: None
    • Component/s: Docs
    • Labels:
    • Environment:

      CentOS release 6.4 (Final)

    • Flags:
      Important

      Description

      REF: https://archive.cloudera.com/cdh5/cdh/5/flume-ng/FlumeUserGuide.html#kafka-sink
      Original in example:

      a1.sinks.k1.channel = c1
      a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink
      a1.sinks.k1.kafka.topic = mytopic
      a1.sinks.k1.kafka.bootstrap.servers = localhost:9092
      a1.sinks.k1.kafka.flumeBatchSize = 20
      a1.sinks.k1.kafka.producer.acks = 1
      a1.sinks.k1.kafka.producer.linger.ms = 1
      a1.sinks.ki.kafka.producer.compression.type = snappy

      a1.sinks.k1.kafka.flumeBatchSize doesn't make any sense, the batch size will be set to default (100), after I change it to:

      a1.sinks.k1.flumeBatchSize = 20

      everything works fine

        Attachments

          Activity

            People

            • Assignee:
              jholoman Jeff Holoman
              Reporter:
              Kamil June Kamil June
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 12h
                12h
                Remaining:
                Remaining Estimate - 12h
                12h
                Logged:
                Time Spent - Not Specified
                Not Specified