[FLUME-424] Make Multi master startup not depend on special config vars or special command line flags.

• Type: New Feature
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.9.0, 0.9.1, 0.9.1u1, 0.9.2
• Fix Version/s: 0.9.3
• Component/s: Master
• Assignee: Jonathan Hsieh
• Reporter: Jonathan Hsieh
• Created: 23/Dec/10 04:09
• Updated: 05/Feb/11 16:11
• Resolved: 05/Jan/11 21:22

Description

A common problems just getting started with multi master setup is having to specify a special config var in the flume-site.xml file for each master, having to run 'flume master -i #' where # is some numerical index.

This presents an error like this:

---

2010-12-24 01:55:47,187 [main] INFO master.ZKInProcessServer:

{server. 1=hostB:3182:3183, server.0=localhost:3182:3183, initLimit=10, syncLimit=10, maxClientCnxns=0, clientPort=3181, tickTime=2000, electionAlg=3, dataDir=/tmp/flume-hdpadmin-zk/server-0}

2010-12-24 01:55:47,191 [ZooKeeper thread] INFO
master.ZKInProcessServer: Starting ZooKeeper server
2010-12-24 01:55:47,193 [main] INFO master.ZKInProcessServer: server
0.0.0.0:3181 not up yet
2010-12-24 01:56:03,225 [main] ERROR master.FlumeMaster: IO problem:
ZooKeeper server did not come up within 15 seconds

---

Ideally a master peer should be able to figure out what its index is by going through the flume.master.servers and matching it itself up to one of the listed masters. It should probably exit with a useful error message if it cannot figure out what master in the list it is, and there should be some way to override or force a master to think it is one of peer masters in the list.

Issue Links

• is related to: FLUME-181 Allow flume.master.serverid to be a non-integer