Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.1.0
    • Fix Version/s: v1.2.0
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Environment:

      RHEL 6.2 64-bit

      Description

      Currently there doesn't seem to be a way for a user to specify a custom processor... for example, a replicating processor that replicates a channel's output to all sinks. If this is implemented in a way to how custom sources and sinks are implemented, then we'd need a SinkProcessor.OTHER type, as a start.

      1. FLUME-1001.5.patch
        5 kB
        Juhani Connolly

        Activity

        Hide
        Juhani Connolly added a comment -

        Seeing as no-one else has expressed interest in this, I'll put it together, following the same style as was used for sinks and sources

        Show
        Juhani Connolly added a comment - Seeing as no-one else has expressed interest in this, I'll put it together, following the same style as was used for sinks and sources
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4152/
        -----------------------------------------------------------

        Review request for Flume.

        Summary
        -------

        Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.

        This addresses bug FLUME-1001.
        https://issues.apache.org/jira/browse/FLUME-1001

        Diffs


        flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b
        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e
        flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION

        Diff: https://reviews.apache.org/r/4152/diff

        Testing
        -------

        Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

        Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works

        Thanks,

        Juhani

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/ ----------------------------------------------------------- Review request for Flume. Summary ------- Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef. This addresses bug FLUME-1001 . https://issues.apache.org/jira/browse/FLUME-1001 Diffs flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION Diff: https://reviews.apache.org/r/4152/diff Testing ------- Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works Thanks, Juhani
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4152/#review5557
        -----------------------------------------------------------

        flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java
        <https://reviews.apache.org/r/4152/#comment12021>

        For consistency with the other types (SinkType, SourceType, ...), can we please put OTHER at the top of the enum listing?

        Also for same consistency, can we have the comment read:

        /**

        • Place holder for custom sink processors not part of this enumeration.
          */

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
        <https://reviews.apache.org/r/4152/#comment12035>

        Before this line:
        Preconditions.checkNotNull(context);
        Preconditions.checkNotNull(sinks);
        logger.debug("Creating instance of sink processor name {}, type {}", name, type);

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
        <https://reviews.apache.org/r/4152/#comment12032>

        => "Sink processor type {} is a custom type"

        (this string output matches the convention used by DefaultSourceFactory)

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
        <https://reviews.apache.org/r/4152/#comment12041>

        add "sink", remove comma:

        =>"Unable to create sink processor type: "

        flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java
        <https://reviews.apache.org/r/4152/#comment12028>

        Thank you for this test. It looks like it confirms that the FQDN and the enum name of the sink processor each return return the same SinkProcessor instance. I see that the intent of using the FQDN is to test the "OTHER" functionality.

        • Will

        On 2012-03-02 07:51:49, Juhani Connolly wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4152/

        -----------------------------------------------------------

        (Updated 2012-03-02 07:51:49)

        Review request for Flume.

        Summary

        -------

        Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.

        This addresses bug FLUME-1001.

        https://issues.apache.org/jira/browse/FLUME-1001

        Diffs

        -----

        flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e

        flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION

        Diff: https://reviews.apache.org/r/4152/diff

        Testing

        -------

        Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

        Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works

        Thanks,

        Juhani

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/#review5557 ----------------------------------------------------------- flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java < https://reviews.apache.org/r/4152/#comment12021 > For consistency with the other types (SinkType, SourceType, ...), can we please put OTHER at the top of the enum listing? Also for same consistency, can we have the comment read: /** Place holder for custom sink processors not part of this enumeration. */ flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java < https://reviews.apache.org/r/4152/#comment12035 > Before this line: Preconditions.checkNotNull(context); Preconditions.checkNotNull(sinks); logger.debug("Creating instance of sink processor name {}, type {}", name, type); flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java < https://reviews.apache.org/r/4152/#comment12032 > => "Sink processor type {} is a custom type" (this string output matches the convention used by DefaultSourceFactory) flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java < https://reviews.apache.org/r/4152/#comment12041 > add "sink", remove comma: =>"Unable to create sink processor type: " flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java < https://reviews.apache.org/r/4152/#comment12028 > Thank you for this test. It looks like it confirms that the FQDN and the enum name of the sink processor each return return the same SinkProcessor instance. I see that the intent of using the FQDN is to test the "OTHER" functionality. Will On 2012-03-02 07:51:49, Juhani Connolly wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/ ----------------------------------------------------------- (Updated 2012-03-02 07:51:49) Review request for Flume. Summary ------- Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef. This addresses bug FLUME-1001 . https://issues.apache.org/jira/browse/FLUME-1001 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION Diff: https://reviews.apache.org/r/4152/diff Testing ------- Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works Thanks, Juhani
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-03-02 16:47:32, Will McQueen wrote:

        > flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java, line 40

        > <https://reviews.apache.org/r/4152/diff/2/?file=87333#file87333line40>

        >

        > For consistency with the other types (SinkType, SourceType, ...), can we please put OTHER at the top of the enum listing?

        >

        > Also for same consistency, can we have the comment read:

        >

        > /**

        > * Place holder for custom sink processors not part of this enumeration.

        > */

        Makes sense, done

        On 2012-03-02 16:47:32, Will McQueen wrote:

        > flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java, line 40

        > <https://reviews.apache.org/r/4152/diff/2/?file=87334#file87334line40>

        >

        > Before this line:

        > Preconditions.checkNotNull(context);

        > Preconditions.checkNotNull(sinks);

        > logger.debug("Creating instance of sink processor name {}, type {}", name, type);

        Added precondition checks, changed the debug message a bit because processors belong to a sink group and thus themselves are not named.

        On 2012-03-02 16:47:32, Will McQueen wrote:

        > flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java, line 48

        > <https://reviews.apache.org/r/4152/diff/2/?file=87334#file87334line48>

        >

        > => "Sink processor type {} is a custom type"

        >

        > (this string output matches the convention used by DefaultSourceFactory)

        ok

        On 2012-03-02 16:47:32, Will McQueen wrote:

        > flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java, line 66

        > <https://reviews.apache.org/r/4152/diff/2/?file=87334#file87334line66>

        >

        > add "sink", remove comma:

        >

        > =>"Unable to create sink processor type: "

        >

        done

        • Juhani

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4152/#review5557
        -----------------------------------------------------------

        On 2012-03-02 07:51:49, Juhani Connolly wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4152/

        -----------------------------------------------------------

        (Updated 2012-03-02 07:51:49)

        Review request for Flume.

        Summary

        -------

        Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.

        This addresses bug FLUME-1001.

        https://issues.apache.org/jira/browse/FLUME-1001

        Diffs

        -----

        flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e

        flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION

        Diff: https://reviews.apache.org/r/4152/diff

        Testing

        -------

        Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

        Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works

        Thanks,

        Juhani

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-03-02 16:47:32, Will McQueen wrote: > flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java, line 40 > < https://reviews.apache.org/r/4152/diff/2/?file=87333#file87333line40 > > > For consistency with the other types (SinkType, SourceType, ...), can we please put OTHER at the top of the enum listing? > > Also for same consistency, can we have the comment read: > > /** > * Place holder for custom sink processors not part of this enumeration. > */ Makes sense, done On 2012-03-02 16:47:32, Will McQueen wrote: > flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java, line 40 > < https://reviews.apache.org/r/4152/diff/2/?file=87334#file87334line40 > > > Before this line: > Preconditions.checkNotNull(context); > Preconditions.checkNotNull(sinks); > logger.debug("Creating instance of sink processor name {}, type {}", name, type); Added precondition checks, changed the debug message a bit because processors belong to a sink group and thus themselves are not named. On 2012-03-02 16:47:32, Will McQueen wrote: > flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java, line 48 > < https://reviews.apache.org/r/4152/diff/2/?file=87334#file87334line48 > > > => "Sink processor type {} is a custom type" > > (this string output matches the convention used by DefaultSourceFactory) ok On 2012-03-02 16:47:32, Will McQueen wrote: > flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java, line 66 > < https://reviews.apache.org/r/4152/diff/2/?file=87334#file87334line66 > > > add "sink", remove comma: > > =>"Unable to create sink processor type: " > done Juhani ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/#review5557 ----------------------------------------------------------- On 2012-03-02 07:51:49, Juhani Connolly wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/ ----------------------------------------------------------- (Updated 2012-03-02 07:51:49) Review request for Flume. Summary ------- Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef. This addresses bug FLUME-1001 . https://issues.apache.org/jira/browse/FLUME-1001 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION Diff: https://reviews.apache.org/r/4152/diff Testing ------- Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works Thanks, Juhani
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4152/
        -----------------------------------------------------------

        (Updated 2012-03-05 03:25:20.490086)

        Review request for Flume.

        Changes
        -------

        Addressed the suggested changes

        Summary
        -------

        Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.

        This addresses bug FLUME-1001.
        https://issues.apache.org/jira/browse/FLUME-1001

        Diffs (updated)


        flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION
        flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b
        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e

        Diff: https://reviews.apache.org/r/4152/diff

        Testing
        -------

        Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

        Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works

        Thanks,

        Juhani

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/ ----------------------------------------------------------- (Updated 2012-03-05 03:25:20.490086) Review request for Flume. Changes ------- Addressed the suggested changes Summary ------- Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef. This addresses bug FLUME-1001 . https://issues.apache.org/jira/browse/FLUME-1001 Diffs (updated) flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e Diff: https://reviews.apache.org/r/4152/diff Testing ------- Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works Thanks, Juhani
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4152/#review6704
        -----------------------------------------------------------

        Ship it!

        looks good to me, just one nit. After you address to the nit please attache the patch to the JIRA.

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
        <https://reviews.apache.org/r/4152/#comment14508>

        INFO is more appropriate, people freak out when they see WARN.

        • Brock

        On 2012-03-05 03:25:20, Juhani Connolly wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4152/

        -----------------------------------------------------------

        (Updated 2012-03-05 03:25:20)

        Review request for Flume.

        Summary

        -------

        Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.

        This addresses bug FLUME-1001.

        https://issues.apache.org/jira/browse/FLUME-1001

        Diffs

        -----

        flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION

        flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e

        Diff: https://reviews.apache.org/r/4152/diff

        Testing

        -------

        Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

        Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works

        Thanks,

        Juhani

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/#review6704 ----------------------------------------------------------- Ship it! looks good to me, just one nit. After you address to the nit please attache the patch to the JIRA. flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java < https://reviews.apache.org/r/4152/#comment14508 > INFO is more appropriate, people freak out when they see WARN. Brock On 2012-03-05 03:25:20, Juhani Connolly wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/ ----------------------------------------------------------- (Updated 2012-03-05 03:25:20) Review request for Flume. Summary ------- Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef. This addresses bug FLUME-1001 . https://issues.apache.org/jira/browse/FLUME-1001 Diffs ----- flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e Diff: https://reviews.apache.org/r/4152/diff Testing ------- Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works Thanks, Juhani
        Hide
        Juhani Connolly added a comment -

        Changed the warn mentioned in the review to info

        Show
        Juhani Connolly added a comment - Changed the warn mentioned in the review to info
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-05 03:18:45, Brock Noland wrote:

        > looks good to me, just one nit. After you address to the nit please attache the patch to the JIRA.

        Done and updated the issue.
        This review is against the flume-728 so can't update the review file here, but it's just one row

        • Juhani

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4152/#review6704
        -----------------------------------------------------------

        On 2012-03-05 03:25:20, Juhani Connolly wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4152/

        -----------------------------------------------------------

        (Updated 2012-03-05 03:25:20)

        Review request for Flume.

        Summary

        -------

        Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.

        This addresses bug FLUME-1001.

        https://issues.apache.org/jira/browse/FLUME-1001

        Diffs

        -----

        flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION

        flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e

        Diff: https://reviews.apache.org/r/4152/diff

        Testing

        -------

        Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

        Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works

        Thanks,

        Juhani

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-05 03:18:45, Brock Noland wrote: > looks good to me, just one nit. After you address to the nit please attache the patch to the JIRA. Done and updated the issue. This review is against the flume-728 so can't update the review file here, but it's just one row Juhani ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/#review6704 ----------------------------------------------------------- On 2012-03-05 03:25:20, Juhani Connolly wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/ ----------------------------------------------------------- (Updated 2012-03-05 03:25:20) Review request for Flume. Summary ------- Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef. This addresses bug FLUME-1001 . https://issues.apache.org/jira/browse/FLUME-1001 Diffs ----- flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e Diff: https://reviews.apache.org/r/4152/diff Testing ------- Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works Thanks, Juhani
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-05 03:18:45, Brock Noland wrote:

        > looks good to me, just one nit. After you address to the nit please attache the patch to the JIRA.

        Juhani Connolly wrote:

        Done and updated the issue.

        This review is against the flume-728 so can't update the review file here, but it's just one row

        Yep, I think we can just attach the patch to the JIRA since it's just a single line.

        • Brock

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4152/#review6704
        -----------------------------------------------------------

        On 2012-03-05 03:25:20, Juhani Connolly wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4152/

        -----------------------------------------------------------

        (Updated 2012-03-05 03:25:20)

        Review request for Flume.

        Summary

        -------

        Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.

        This addresses bug FLUME-1001.

        https://issues.apache.org/jira/browse/FLUME-1001

        Diffs

        -----

        flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION

        flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e

        Diff: https://reviews.apache.org/r/4152/diff

        Testing

        -------

        Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

        Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works

        Thanks,

        Juhani

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-05 03:18:45, Brock Noland wrote: > looks good to me, just one nit. After you address to the nit please attache the patch to the JIRA. Juhani Connolly wrote: Done and updated the issue. This review is against the flume-728 so can't update the review file here, but it's just one row Yep, I think we can just attach the patch to the JIRA since it's just a single line. Brock ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/#review6704 ----------------------------------------------------------- On 2012-03-05 03:25:20, Juhani Connolly wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/ ----------------------------------------------------------- (Updated 2012-03-05 03:25:20) Review request for Flume. Summary ------- Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef. This addresses bug FLUME-1001 . https://issues.apache.org/jira/browse/FLUME-1001 Diffs ----- flume-ng-core/src/test/java/org/apache/flume/sink/TestSinkProcessorFactory.java PRE-CREATION flume-ng-core/src/main/java/org/apache/flume/SinkProcessorType.java be1891b flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 10f9f4e Diff: https://reviews.apache.org/r/4152/diff Testing ------- Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works Thanks, Juhani
        Hide
        Juhani Connolly added a comment -

        Reuploading this with a cleaner patch file, tests pass and the warn/info matter from the review https://reviews.apache.org/r/4152/ is fixed

        Show
        Juhani Connolly added a comment - Reuploading this with a cleaner patch file, tests pass and the warn/info matter from the review https://reviews.apache.org/r/4152/ is fixed
        Hide
        Arvind Prabhakar added a comment -

        Apologies for not getting to this patch earlier. Can you please rebase the patch to the trunk? I will be happy to commit it.

        Show
        Arvind Prabhakar added a comment - Apologies for not getting to this patch earlier. Can you please rebase the patch to the trunk? I will be happy to commit it.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4152/
        -----------------------------------------------------------

        (Updated 2012-05-09 09:56:14.961414)

        Review request for Flume.

        Changes
        -------

        Updated against trunk

        Summary
        -------

        Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.

        This addresses bug FLUME-1001.
        https://issues.apache.org/jira/browse/FLUME-1001

        Diffs (updated)


        flume-ng-core/src/test/java/org/apache/flume/sink/SinkProcessorFactoryTest.java PRE-CREATION
        flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java d397bd3
        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 3ff6737

        Diff: https://reviews.apache.org/r/4152/diff

        Testing
        -------

        Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

        Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works

        Thanks,

        Juhani

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/ ----------------------------------------------------------- (Updated 2012-05-09 09:56:14.961414) Review request for Flume. Changes ------- Updated against trunk Summary ------- Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef. This addresses bug FLUME-1001 . https://issues.apache.org/jira/browse/FLUME-1001 Diffs (updated) flume-ng-core/src/test/java/org/apache/flume/sink/SinkProcessorFactoryTest.java PRE-CREATION flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java d397bd3 flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 3ff6737 Diff: https://reviews.apache.org/r/4152/diff Testing ------- Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works Thanks, Juhani
        Hide
        Juhani Connolly added a comment -

        Updated version for current trunk(as posted to reviewboard)

        Show
        Juhani Connolly added a comment - Updated version for current trunk(as posted to reviewboard)
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4152/#review7829
        -----------------------------------------------------------

        Ship it!

        +1

        • Arvind

        On 2012-05-09 09:56:14, Juhani Connolly wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4152/

        -----------------------------------------------------------

        (Updated 2012-05-09 09:56:14)

        Review request for Flume.

        Summary

        -------

        Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef.

        This addresses bug FLUME-1001.

        https://issues.apache.org/jira/browse/FLUME-1001

        Diffs

        -----

        flume-ng-core/src/test/java/org/apache/flume/sink/SinkProcessorFactoryTest.java PRE-CREATION

        flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java d397bd3

        flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 3ff6737

        Diff: https://reviews.apache.org/r/4152/diff

        Testing

        -------

        Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment

        Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works

        Thanks,

        Juhani

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/#review7829 ----------------------------------------------------------- Ship it! +1 Arvind On 2012-05-09 09:56:14, Juhani Connolly wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4152/ ----------------------------------------------------------- (Updated 2012-05-09 09:56:14) Review request for Flume. Summary ------- Made custom processors possible in the same way as sink and source: the type can represent a classname or a shorthand name from the typedef. This addresses bug FLUME-1001 . https://issues.apache.org/jira/browse/FLUME-1001 Diffs ----- flume-ng-core/src/test/java/org/apache/flume/sink/SinkProcessorFactoryTest.java PRE-CREATION flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java d397bd3 flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java 3ff6737 Diff: https://reviews.apache.org/r/4152/diff Testing ------- Existing tests pass, except I'm having an issue with TestNetcatSource which seems to be totally unrelated... It turns up on flume-728 head too, probably something in my environment Added a new test to verify that processors created by shorthand name and by full class create the same class, verifying that creation by classname also works Thanks, Juhani
        Hide
        Arvind Prabhakar added a comment -

        Patch committed. Thanks Juhani!

        Show
        Arvind Prabhakar added a comment - Patch committed. Thanks Juhani!
        Hide
        Hudson added a comment -

        Integrated in flume-trunk #203 (See https://builds.apache.org/job/flume-trunk/203/)
        FLUME-1001. Support custom sink processors.

        (Juhani Connolly via Arvind Prabhakar) (Revision 1337577)

        Result = SUCCESS
        arvind : http://svn.apache.org/viewvc/?view=rev&rev=1337577
        Files :

        • /incubator/flume/trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java
        • /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java
        • /incubator/flume/trunk/flume-ng-core/src/test/java/org/apache/flume/sink/SinkProcessorFactoryTest.java
        Show
        Hudson added a comment - Integrated in flume-trunk #203 (See https://builds.apache.org/job/flume-trunk/203/ ) FLUME-1001 . Support custom sink processors. (Juhani Connolly via Arvind Prabhakar) (Revision 1337577) Result = SUCCESS arvind : http://svn.apache.org/viewvc/?view=rev&rev=1337577 Files : /incubator/flume/trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkProcessorType.java /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/sink/SinkProcessorFactory.java /incubator/flume/trunk/flume-ng-core/src/test/java/org/apache/flume/sink/SinkProcessorFactoryTest.java

          People

          • Assignee:
            Juhani Connolly
            Reporter:
            Will McQueen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development