[CHUKWA-87] Rewrite all rolling classes

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: MR Data Processors
• Assignee: Unassigned
• Reporter: Jerome Boulon
• Created: 02/Apr/09 18:41
• Updated: 22/Mar/16 03:00

Description

There's a dependency between Hourly and Daily rolling.
This should be simplified to only be a generic compacter.
The compacter should run whenever there's more than xx spills files in the stream directory and this could happen any time during the day.
This may implies flattering the stream hierarchy but the logic for that is only on the 2 rolling classes and the MoveToRepo class.

The compacter should be able to get the list of files, compact them, put the new file in place and removed the one that have been part of the compacter M/R job.
The compacter will be a daemon process that will always run in background.