To solve certain known issues and limitations, Flume requires a refactoring of some core classes and systems. This bug is a parent issue to track the development of a "Flume NG" - a poorly named, but necessary refactoring. Subtasks should be added to track individual systems and components.
The following known issues are specifically to be addressed:
- Code complexity; Flume has evolved over the last few years and has a fair amount of extraneous code.
- Core component lifecycle standardization and control code (e.g. anything that can be start()ed or stop()ed, sources, sinks).
- (Static) Configuration access throughout the code base.
- Drastic simplification of common data paths (e.g. durability as an element of the source rather than a disconnected sink).
- Heartbeat and master rearchitecture.
- Renaming packages to org.apache.flume.
This is a large and far reaching set of tasks. The intent is to perform this work in a branch as to not disrupt immediate releases or short term forthcoming releases while still allowing open development in the community.