[FLUME-3136] Flume on windows with Kafka sink, Kafka server on centos

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.7.0
• Component/s: Sinks+Sources

Environment

Flume agent on Windows 10
Kafka server 0.10 on CENTOS

• Assignee: Unassigned
• Reporter: Tarek Abouzeid
• Created: 27/Jul/17 13:33
• Updated: 27/Jul/17 13:33

Description

i am having an agent on windows 10 with netcat source and kafka sink, when i put data using telnet it got received in Flume but wont be sent to kafka then eventually i got "Batch expired exception", i tried same exercise with same configuration file but agent was on centos and it succeeded , below my conf.

a1.sources = r1
a1.channels = c1

a1.channels = c1
a1.channels.c1.type = memory
a1.channels.c1.capacity = 10000
a1.channels.c1.transactionCapacity = 10000
a1.channels.c1.byteCapacityBufferPercentage = 20
a1.channels.c1.byteCapacity = 800000

a1.sources.r1.type = netcat
a1.sources.r1.bind = 192.168.184.1
a1.sources.r1.port = 44444
a1.sources.r1.channels = c1

a1.sinks = k1
a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k1.channel = c1
a1.sinks.k1.kafka.topic = test
a1.sinks.k1.kafka.bootstrap.servers = 192.168.184.132:9092
a1.sinks.k1.kafka.flumeBatchSize = 20
a1.sinks.k1.kafka.producer.acks = 1
a1.sinks.k1.kafka.producer.linger.ms = 1
a1.sinks.ki.kafka.producer.compression.type = snappy