[FLUME-1798] Avro source can't bind to wildcard address. Problem with ipv6

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.3.0
• Component/s: Sinks+Sources
• Labels: AvroSource avro

Environment

Server: Ubuntu 12.04.1 x86_64 (with IPv6), OpenJDK 64-bit IcedTea7 2.3.3
Client: WinXP (no IPv6), Oracle Java JRE build 1.7.0_05-b05

• Assignee: Unassigned
• Reporter: Greg Bowering
• Created: 19/Dec/12 05:54
• Updated: 05/Feb/17 04:32

Description

Thanks for developing Flume-NG

However requiring bind address for AvroSource seems problematic here!

On WinXP I only have IPv4 available, and bind address same as the hostname works just fine.

When I migrated my Flume agent to Ubuntu, this no longer works since binding to the server hostname by default seems to only accept incoming IPv6 connections. This breaks for me since all my clients only have IPv4.

I found I could force AvroSource to accept IPv4 only by binding using the IPv4 address in the conf rather than the hostname.

This is unsatisfactory since in many cases servers have dynamically assigned IP addresses.

Furthermore, in some cluster deployment scenarios, even hostnames are not always static (e.g. standing up an EC2 spot-instance to process logs). It would be much simpler to configure a cluster of Flume agents without having to insert the hostname or IP address in all the conf files, i.e. allow AvroSource to bind to the wildcard address. This is a simple fix (omit the Configurables.ensureRequiredNonNull on BIND_KEY in AvroSource#configure(Context))

If sysadmins wish to lock-down network interfaces then there are firewalls/iptables for that kind of stuff...