Flume
  1. Flume
  2. FLUME-299

Add support for subarguments in flume config elements

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v0.9.4
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      It would make sense for certain arguments to have sub-arguments.

      For example, when specifiying the console("avro") sink, it would be nice to give the avro output format arguments. So we could be able to say 'console(avro("json"))' or 'console(avro("bin","gzip"))'

      This would also allow roll sinks to specify different trigger policies:

      'roll( timer(1000))

      { ... }' // roll every second
      'roll( count(100)) { ... }

      ' // roll every 100 events
      'roll( countTimer(100,1000))

      { ... }

      ' // roll every 100 events or every 1000 ms
      'roll( size(1000000) )

      {... }

      ' // roll every 1000000 bytes.

      It would also allow failovers sinks to specify retry policies.

        Issue Links

          Activity

          Jonathan Hsieh made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Mark Thomas made changes -
          Project Import Tue Aug 02 16:57:12 UTC 2011 [ 1312304232406 ]
          Jonathan Hsieh made changes -
          Link This issue blocks FLUME-541 [ FLUME-541 ]
          Jonathan Hsieh made changes -
          Link This issue relates to FLUME-472 [ FLUME-472 ]
          Jonathan Hsieh made changes -
          Status Patch Available [ 10000 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Fix Version/s v0.9.4 [ 10050 ]
          Jonathan Hsieh made changes -
          Jonathan Hsieh made changes -
          Status Open [ 1 ] Patch Available [ 10000 ]
          Jonathan Hsieh made changes -
          Status Patch Available [ 10000 ] Open [ 1 ]
          Jonathan Hsieh made changes -
          Assignee Jonathan Hsieh [ jmhsieh ]
          Jonathan Hsieh made changes -
          Status Open [ 1 ] Patch Available [ 10000 ]
          Jonathan Hsieh made changes -
          Link This issue blocks FLUME-632 [ FLUME-632 ]
          Jonathan Hsieh made changes -
          Summary Add support for arguments flume config elements Add support for subarguments in flume config elements
          Jonathan Hsieh made changes -
          Link This issue relates to FLUME-133 [ FLUME-133 ]
          Jonathan Hsieh made changes -
          Field Original Value New Value
          Link This issue blocks FLUME-273 [ FLUME-273 ]
          Jonathan Hsieh created issue -

            People

            • Assignee:
              Jonathan Hsieh
              Reporter:
              Jonathan Hsieh
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development