Flume
  1. Flume
  2. FLUME-1941

Support defaults or inheritance in configs

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Configuration
    • Labels:
      None

      Description

      Proposal to support defaults or inheritance in configs.

      The idea is to create a "prototypal" component config, such as a source or sink, which is not necessarily instantiated but is used to avoid repetitive configurations when creating multiple components of the same type. A great example of this is users who define 5 HDFS sinks to increase write parallelism, but they each contain many of the same configuration parameters and differ only in their name and path.

      Basic idea:

      agent.sinks.sink-proto-1.type = my-sink
      agent.sinks.sink-proto-1.path = /var/log/foo/bar
      agent.sinks.sink-proto-1.serializer = MySerializer$Builder
      agent.sinks.sink-proto-1.credentials = mpercy
      
      agent.sinks.sink-1.__prototype__ = sink-proto-1
      agent.sinks.sink-1.path = /var/log/baz/blam
      
      agent.sinks.sink-2.__prototype__ = sink-proto-1
      agent.sinks.sink-2.path = /var/log/glerp/bazinga
      

        Issue Links

          Activity

          Mike Percy created issue -
          Mike Percy made changes -
          Field Original Value New Value
          Link This issue is blocked by FLUME-2090 [ FLUME-2090 ]
          Mike Percy made changes -
          Fix Version/s v1.4.0 [ 12323372 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Mike Percy
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development