[FLUME-240] Flume master status page keeps accumulating information

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.1, 0.9.1u1, 0.9.2
• Fix Version/s: 0.9.3
• Component/s: Master, Shell

Environment

Ubuntu lucid, Java 1.6+

• Assignee: Jonathan Hsieh
• Reporter: Disabled imported user
• Created: 23/Sep/10 18:05
• Updated: 20/Dec/10 22:21
• Resolved: 20/Dec/10 22:21

Description

The flume master keeps accumulating config information. This is especially bad in an auto scaled (EC2) environment where nodes go up and down every day.

We simply want to see all the active nodes on the master status page. We do not want to see anything that is decommissioned or lost.
There are multiple ways to handle this

One way is to have some sort of 'purge' commands that will clear out decommission or lost nodes.We can always set a cron to run the purge command every day. I would further suggest that the purge command should take a status so that you can delete bunch of nodes together. Otherwise in a bigger cluster, it will be a problem.

Another way is to add a filter to the master status page, that will only show all the active nodes. That way people can choose what they want to see.

Issue Links

• blocks: FLUME-417 Add command to purge only DECOMMISSIONED or LOST node status.