Flume
  1. Flume
  2. FLUME-945

Add the ability to specify a default channel for multiplexing channel selector.

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.0.0
    • Fix Version/s: v1.1.0
    • Component/s: None
    • Labels:
      None

      Description

      For events that do not match any mapping specified in the selector configuration, there should be the ability to setup a default channel. This will ensure that no events are getting dropped from the system.

      1. FLUME-945-3.patch
        7 kB
        Hari Shreedharan
      2. FLUME-945-2.patch
        7 kB
        Hari Shreedharan

        Activity

        Arvind Prabhakar created issue -
        Arvind Prabhakar made changes -
        Field Original Value New Value
        Affects Version/s v1.0.0 [ 12318896 ]
        Hide
        Hari Shreedharan added a comment -

        I have added support for this. Now if an event comes in which has header value which does not map to any of the configured mappings, then the event is routed to default channels. It is now mandatory to define mandatory channels in the config. Also if an event comes in with no header value, then it is also sent to the the default channels. Unit tests have been updated to cover these cases.

        Show
        Hari Shreedharan added a comment - I have added support for this. Now if an event comes in which has header value which does not map to any of the configured mappings, then the event is routed to default channels. It is now mandatory to define mandatory channels in the config. Also if an event comes in with no header value, then it is also sent to the the default channels. Unit tests have been updated to cover these cases.
        Hari Shreedharan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Hari Shreedharan [ hshreedharan ]
        Fix Version/s v1.1.0 [ 12319284 ]
        Hari Shreedharan made changes -
        Attachment FLUME-945-2.patch [ 12514425 ]
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3888/
        -----------------------------------------------------------

        Review request for Flume.

        Summary
        -------

        If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels.

        This addresses bug FLUME-945.
        https://issues.apache.org/jira/browse/FLUME-945

        Diffs


        flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2
        flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad

        Diff: https://reviews.apache.org/r/3888/diff

        Testing
        -------

        Added unit tests.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/ ----------------------------------------------------------- Review request for Flume. Summary ------- If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels. This addresses bug FLUME-945 . https://issues.apache.org/jira/browse/FLUME-945 Diffs flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2 flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad Diff: https://reviews.apache.org/r/3888/diff Testing ------- Added unit tests. Thanks, Hari
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3888/
        -----------------------------------------------------------

        (Updated 2012-02-14 00:32:05.297592)

        Review request for Flume.

        Changes
        -------

        Removed a bunch of trailing spaces

        Summary
        -------

        If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels.

        This addresses bug FLUME-945.
        https://issues.apache.org/jira/browse/FLUME-945

        Diffs (updated)


        flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2
        flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad

        Diff: https://reviews.apache.org/r/3888/diff

        Testing
        -------

        Added unit tests.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/ ----------------------------------------------------------- (Updated 2012-02-14 00:32:05.297592) Review request for Flume. Changes ------- Removed a bunch of trailing spaces Summary ------- If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels. This addresses bug FLUME-945 . https://issues.apache.org/jira/browse/FLUME-945 Diffs (updated) flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2 flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad Diff: https://reviews.apache.org/r/3888/diff Testing ------- Added unit tests. Thanks, Hari
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3888/
        -----------------------------------------------------------

        (Updated 2012-02-14 17:24:36.789367)

        Review request for Flume.

        Changes
        -------

        Moved channelMap creation into getChannelListFromNames function.

        Summary
        -------

        If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels.

        This addresses bug FLUME-945.
        https://issues.apache.org/jira/browse/FLUME-945

        Diffs (updated)


        flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2
        flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad

        Diff: https://reviews.apache.org/r/3888/diff

        Testing
        -------

        Added unit tests.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/ ----------------------------------------------------------- (Updated 2012-02-14 17:24:36.789367) Review request for Flume. Changes ------- Moved channelMap creation into getChannelListFromNames function. Summary ------- If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels. This addresses bug FLUME-945 . https://issues.apache.org/jira/browse/FLUME-945 Diffs (updated) flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2 flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad Diff: https://reviews.apache.org/r/3888/diff Testing ------- Added unit tests. Thanks, Hari
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3888/#review5115
        -----------------------------------------------------------

        Ship it!

        +1

        • Arvind

        On 2012-02-14 17:24:36, Hari Shreedharan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/3888/

        -----------------------------------------------------------

        (Updated 2012-02-14 17:24:36)

        Review request for Flume.

        Summary

        -------

        If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels.

        This addresses bug FLUME-945.

        https://issues.apache.org/jira/browse/FLUME-945

        Diffs

        -----

        flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2

        flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad

        Diff: https://reviews.apache.org/r/3888/diff

        Testing

        -------

        Added unit tests.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/#review5115 ----------------------------------------------------------- Ship it! +1 Arvind On 2012-02-14 17:24:36, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/ ----------------------------------------------------------- (Updated 2012-02-14 17:24:36) Review request for Flume. Summary ------- If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels. This addresses bug FLUME-945 . https://issues.apache.org/jira/browse/FLUME-945 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2 flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad Diff: https://reviews.apache.org/r/3888/diff Testing ------- Added unit tests. Thanks, Hari
        Hide
        Arvind Prabhakar added a comment -

        Patch committed to flume-728 branch. Thanks Hari!

        Show
        Arvind Prabhakar added a comment - Patch committed to flume-728 branch. Thanks Hari!
        Arvind Prabhakar made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hari Shreedharan added a comment -

        The reviewed and committed patch.

        Show
        Hari Shreedharan added a comment - The reviewed and committed patch.
        Hari Shreedharan made changes -
        Attachment FLUME-945-3.patch [ 12514603 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        12d 21h 36m 1 Hari Shreedharan 14/Feb/12 00:25
        Patch Available Patch Available Resolved Resolved
        1d 5h 24m 1 Arvind Prabhakar 15/Feb/12 05:50

          People

          • Assignee:
            Hari Shreedharan
            Reporter:
            Arvind Prabhakar
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development