Flume
  1. Flume
  2. FLUME-945

Add the ability to specify a default channel for multiplexing channel selector.

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.0.0
    • Fix Version/s: v1.1.0
    • Component/s: None
    • Labels:
      None

      Description

      For events that do not match any mapping specified in the selector configuration, there should be the ability to setup a default channel. This will ensure that no events are getting dropped from the system.

      1. FLUME-945-3.patch
        7 kB
        Hari Shreedharan
      2. FLUME-945-2.patch
        7 kB
        Hari Shreedharan

        Activity

        Hide
        Hari Shreedharan added a comment -

        I have added support for this. Now if an event comes in which has header value which does not map to any of the configured mappings, then the event is routed to default channels. It is now mandatory to define mandatory channels in the config. Also if an event comes in with no header value, then it is also sent to the the default channels. Unit tests have been updated to cover these cases.

        Show
        Hari Shreedharan added a comment - I have added support for this. Now if an event comes in which has header value which does not map to any of the configured mappings, then the event is routed to default channels. It is now mandatory to define mandatory channels in the config. Also if an event comes in with no header value, then it is also sent to the the default channels. Unit tests have been updated to cover these cases.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3888/
        -----------------------------------------------------------

        Review request for Flume.

        Summary
        -------

        If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels.

        This addresses bug FLUME-945.
        https://issues.apache.org/jira/browse/FLUME-945

        Diffs


        flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2
        flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad

        Diff: https://reviews.apache.org/r/3888/diff

        Testing
        -------

        Added unit tests.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/ ----------------------------------------------------------- Review request for Flume. Summary ------- If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels. This addresses bug FLUME-945 . https://issues.apache.org/jira/browse/FLUME-945 Diffs flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2 flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad Diff: https://reviews.apache.org/r/3888/diff Testing ------- Added unit tests. Thanks, Hari
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3888/
        -----------------------------------------------------------

        (Updated 2012-02-14 00:32:05.297592)

        Review request for Flume.

        Changes
        -------

        Removed a bunch of trailing spaces

        Summary
        -------

        If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels.

        This addresses bug FLUME-945.
        https://issues.apache.org/jira/browse/FLUME-945

        Diffs (updated)


        flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2
        flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad

        Diff: https://reviews.apache.org/r/3888/diff

        Testing
        -------

        Added unit tests.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/ ----------------------------------------------------------- (Updated 2012-02-14 00:32:05.297592) Review request for Flume. Changes ------- Removed a bunch of trailing spaces Summary ------- If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels. This addresses bug FLUME-945 . https://issues.apache.org/jira/browse/FLUME-945 Diffs (updated) flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2 flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad Diff: https://reviews.apache.org/r/3888/diff Testing ------- Added unit tests. Thanks, Hari
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3888/
        -----------------------------------------------------------

        (Updated 2012-02-14 17:24:36.789367)

        Review request for Flume.

        Changes
        -------

        Moved channelMap creation into getChannelListFromNames function.

        Summary
        -------

        If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels.

        This addresses bug FLUME-945.
        https://issues.apache.org/jira/browse/FLUME-945

        Diffs (updated)


        flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2
        flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad

        Diff: https://reviews.apache.org/r/3888/diff

        Testing
        -------

        Added unit tests.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/ ----------------------------------------------------------- (Updated 2012-02-14 17:24:36.789367) Review request for Flume. Changes ------- Moved channelMap creation into getChannelListFromNames function. Summary ------- If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels. This addresses bug FLUME-945 . https://issues.apache.org/jira/browse/FLUME-945 Diffs (updated) flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2 flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad Diff: https://reviews.apache.org/r/3888/diff Testing ------- Added unit tests. Thanks, Hari
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/3888/#review5115
        -----------------------------------------------------------

        Ship it!

        +1

        • Arvind

        On 2012-02-14 17:24:36, Hari Shreedharan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/3888/

        -----------------------------------------------------------

        (Updated 2012-02-14 17:24:36)

        Review request for Flume.

        Summary

        -------

        If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels.

        This addresses bug FLUME-945.

        https://issues.apache.org/jira/browse/FLUME-945

        Diffs

        -----

        flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2

        flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad

        Diff: https://reviews.apache.org/r/3888/diff

        Testing

        -------

        Added unit tests.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/#review5115 ----------------------------------------------------------- Ship it! +1 Arvind On 2012-02-14 17:24:36, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3888/ ----------------------------------------------------------- (Updated 2012-02-14 17:24:36) Review request for Flume. Summary ------- If an event comes in with header that does not exits in the config, then route it to the default channels. Also, if the header does not exist in the event, route it to default channels. This addresses bug FLUME-945 . https://issues.apache.org/jira/browse/FLUME-945 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/channel/MultiplexingChannelSelector.java e070cf2 flume-ng-core/src/test/java/org/apache/flume/channel/TestMultiplexingChannelSelector.java 4e585ad Diff: https://reviews.apache.org/r/3888/diff Testing ------- Added unit tests. Thanks, Hari
        Hide
        Arvind Prabhakar added a comment -

        Patch committed to flume-728 branch. Thanks Hari!

        Show
        Arvind Prabhakar added a comment - Patch committed to flume-728 branch. Thanks Hari!
        Hide
        Hari Shreedharan added a comment -

        The reviewed and committed patch.

        Show
        Hari Shreedharan added a comment - The reviewed and committed patch.

          People

          • Assignee:
            Hari Shreedharan
            Reporter:
            Arvind Prabhakar
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development