Flume
  1. Flume
  2. FLUME-821

Derby schema handler should create the necessary indexes for fast lookups.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: NG alpha 2
    • Component/s: None
    • Labels:
      None
    1. FLUME-821-2.patch
      13 kB
      Arvind Prabhakar
    2. FLUME-821-1.patch
      14 kB
      Arvind Prabhakar

      Activity

      Hide
      Hudson added a comment -

      Integrated in flume-728 #63 (See https://builds.apache.org/job/flume-728/63/)
      FLUME-821. Support for creating indexes in JDBC channel.

      arvind : http://svn.apache.org/viewvc/?view=rev&rev=1208624
      Files :

      • /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java
      • /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java
      • /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java
      • /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java
      • /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java
      • /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java
      Show
      Hudson added a comment - Integrated in flume-728 #63 (See https://builds.apache.org/job/flume-728/63/ ) FLUME-821 . Support for creating indexes in JDBC channel. arvind : http://svn.apache.org/viewvc/?view=rev&rev=1208624 Files : /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java /incubator/flume/branches/flume-728/flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java
      Hide
      Arvind Prabhakar added a comment -

      Patch committed.

      Show
      Arvind Prabhakar added a comment - Patch committed.
      Hide
      Arvind Prabhakar added a comment -

      Attached the updated patch with review feedback.

      Show
      Arvind Prabhakar added a comment - Attached the updated patch with review feedback.
      Hide
      jiraposter@reviews.apache.org added a comment -

      On 2011-11-30 18:04:58, Eric Sammer wrote:

      > Maybe a tweak on the log message level - up to you. Otherwise, lgtm!

      Thanks for the reivew Eric. My comments follow.

      On 2011-11-30 18:04:58, Eric Sammer wrote:

      > flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java, line 120

      > <https://reviews.apache.org/r/2938/diff/1/?file=60141#file60141line120>

      >

      > Is this really a warning or just INFO?

      Will change it before commit.

      On 2011-11-30 18:04:58, Eric Sammer wrote:

      > flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java, line 143

      > <https://reviews.apache.org/r/2938/diff/1/?file=60140#file60140line143>

      >

      > Should all of these constants (at some future point) be broken out into an external properties file for simpler updating? Just food for thought.

      This is an implementation specific constant. Hence it is private scoped within the schema handler.

      • Arvind

      -----------------------------------------------------------
      This is an automatically generated e-mail. To reply, visit:
      https://reviews.apache.org/r/2938/#review3579
      -----------------------------------------------------------

      On 2011-11-27 05:25:12, Arvind Prabhakar wrote:

      -----------------------------------------------------------

      This is an automatically generated e-mail. To reply, visit:

      https://reviews.apache.org/r/2938/

      -----------------------------------------------------------

      (Updated 2011-11-27 05:25:12)

      Review request for Flume and Prasad Mujumdar.

      Summary

      -------

      This change introduces the ability to create indexes while schema creation via the derby schema handler. The implementation defaults to the creation of indexes which can be turned off via configuration.

      This addresses bug FLUME-821.

      https://issues.apache.org/jira/browse/FLUME-821

      Diffs

      -----

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java 7e0b3ac

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java 7b29c7d

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java c1bc400

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java 1065ef9

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java 68946c9

      flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java 9ce6d16

      Diff: https://reviews.apache.org/r/2938/diff

      Testing

      -------

      All unit tests run successfully.

      Thanks,

      Arvind

      Show
      jiraposter@reviews.apache.org added a comment - On 2011-11-30 18:04:58, Eric Sammer wrote: > Maybe a tweak on the log message level - up to you. Otherwise, lgtm! Thanks for the reivew Eric. My comments follow. On 2011-11-30 18:04:58, Eric Sammer wrote: > flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java, line 120 > < https://reviews.apache.org/r/2938/diff/1/?file=60141#file60141line120 > > > Is this really a warning or just INFO? Will change it before commit. On 2011-11-30 18:04:58, Eric Sammer wrote: > flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java, line 143 > < https://reviews.apache.org/r/2938/diff/1/?file=60140#file60140line143 > > > Should all of these constants (at some future point) be broken out into an external properties file for simpler updating? Just food for thought. This is an implementation specific constant. Hence it is private scoped within the schema handler. Arvind ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2938/#review3579 ----------------------------------------------------------- On 2011-11-27 05:25:12, Arvind Prabhakar wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2938/ ----------------------------------------------------------- (Updated 2011-11-27 05:25:12) Review request for Flume and Prasad Mujumdar. Summary ------- This change introduces the ability to create indexes while schema creation via the derby schema handler. The implementation defaults to the creation of indexes which can be turned off via configuration. This addresses bug FLUME-821 . https://issues.apache.org/jira/browse/FLUME-821 Diffs ----- flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java 7e0b3ac flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java 7b29c7d flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java c1bc400 flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java 1065ef9 flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java 68946c9 flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java 9ce6d16 Diff: https://reviews.apache.org/r/2938/diff Testing ------- All unit tests run successfully. Thanks, Arvind
      Hide
      jiraposter@reviews.apache.org added a comment -

      -----------------------------------------------------------
      This is an automatically generated e-mail. To reply, visit:
      https://reviews.apache.org/r/2938/#review3579
      -----------------------------------------------------------

      Ship it!

      Maybe a tweak on the log message level - up to you. Otherwise, lgtm!

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java
      <https://reviews.apache.org/r/2938/#comment7991>

      Should all of these constants (at some future point) be broken out into an external properties file for simpler updating? Just food for thought.

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java
      <https://reviews.apache.org/r/2938/#comment7992>

      Is this really a warning or just INFO?

      • Eric

      On 2011-11-27 05:25:12, Arvind Prabhakar wrote:

      -----------------------------------------------------------

      This is an automatically generated e-mail. To reply, visit:

      https://reviews.apache.org/r/2938/

      -----------------------------------------------------------

      (Updated 2011-11-27 05:25:12)

      Review request for Flume and Prasad Mujumdar.

      Summary

      -------

      This change introduces the ability to create indexes while schema creation via the derby schema handler. The implementation defaults to the creation of indexes which can be turned off via configuration.

      This addresses bug FLUME-821.

      https://issues.apache.org/jira/browse/FLUME-821

      Diffs

      -----

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java 7e0b3ac

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java 7b29c7d

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java c1bc400

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java 1065ef9

      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java 68946c9

      flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java 9ce6d16

      Diff: https://reviews.apache.org/r/2938/diff

      Testing

      -------

      All unit tests run successfully.

      Thanks,

      Arvind

      Show
      jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2938/#review3579 ----------------------------------------------------------- Ship it! Maybe a tweak on the log message level - up to you. Otherwise, lgtm! flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java < https://reviews.apache.org/r/2938/#comment7991 > Should all of these constants (at some future point) be broken out into an external properties file for simpler updating? Just food for thought. flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java < https://reviews.apache.org/r/2938/#comment7992 > Is this really a warning or just INFO? Eric On 2011-11-27 05:25:12, Arvind Prabhakar wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2938/ ----------------------------------------------------------- (Updated 2011-11-27 05:25:12) Review request for Flume and Prasad Mujumdar. Summary ------- This change introduces the ability to create indexes while schema creation via the derby schema handler. The implementation defaults to the creation of indexes which can be turned off via configuration. This addresses bug FLUME-821 . https://issues.apache.org/jira/browse/FLUME-821 Diffs ----- flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java 7e0b3ac flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java 7b29c7d flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java c1bc400 flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java 1065ef9 flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java 68946c9 flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java 9ce6d16 Diff: https://reviews.apache.org/r/2938/diff Testing ------- All unit tests run successfully. Thanks, Arvind
      Hide
      jiraposter@reviews.apache.org added a comment -

      -----------------------------------------------------------
      This is an automatically generated e-mail. To reply, visit:
      https://reviews.apache.org/r/2938/
      -----------------------------------------------------------

      Review request for Flume and Prasad Mujumdar.

      Summary
      -------

      This change introduces the ability to create indexes while schema creation via the derby schema handler. The implementation defaults to the creation of indexes which can be turned off via configuration.

      This addresses bug FLUME-821.
      https://issues.apache.org/jira/browse/FLUME-821

      Diffs


      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java 7e0b3ac
      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java 7b29c7d
      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java c1bc400
      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java 1065ef9
      flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java 68946c9
      flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java 9ce6d16

      Diff: https://reviews.apache.org/r/2938/diff

      Testing
      -------

      All unit tests run successfully.

      Thanks,

      Arvind

      Show
      jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2938/ ----------------------------------------------------------- Review request for Flume and Prasad Mujumdar. Summary ------- This change introduces the ability to create indexes while schema creation via the derby schema handler. The implementation defaults to the creation of indexes which can be turned off via configuration. This addresses bug FLUME-821 . https://issues.apache.org/jira/browse/FLUME-821 Diffs flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/ConfigurationConstants.java 7e0b3ac flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/DerbySchemaHandler.java 7b29c7d flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java c1bc400 flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/MySQLSchemaHandler.java 1065ef9 flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/SchemaHandler.java 68946c9 flume-ng-channels/flume-jdbc-channel/src/test/java/org/apache/flume/channel/jdbc/TestDerbySchemaHandlerQueries.java 9ce6d16 Diff: https://reviews.apache.org/r/2938/diff Testing ------- All unit tests run successfully. Thanks, Arvind

        People

        • Assignee:
          Arvind Prabhakar
          Reporter:
          Arvind Prabhakar
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development