Flume
  1. Flume
  2. FLUME-1183

Implement an HBase Sink which supports table level access

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.2.0
    • Fix Version/s: v1.2.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      This is what I intend to do:

      • Insert the row key from event headers. Pick the column family and column from configuration, not from headers.
      • Allow configuration to specify default for row key. If no row key exists in header, then take the default value from the configuration. I don't want to dump everything into the table with the same row key.
      • I don't intend to support multiple tables, column families or columns in the same sink. We can use multiplexing channel selector to use different sinks for different tables/columns.

      I know the existence of another jira for porting the HBase sink from OG, but didn't see any activity for a while on that.

      1. FLUME-1183-4.patch
        31 kB
        Hari Shreedharan

        Issue Links

          Activity

          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/
          -----------------------------------------------------------

          Review request for Flume.

          Summary
          -------

          Hbase sink.

          This addresses bug FLUME-1183.
          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs


          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION
          flume-ng-sinks/pom.xml acb3087
          pom.xml 8c11a2d
          flume-ng-dist/pom.xml 5bdcfe7
          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION
          bin/flume-ng 0108997

          Diff: https://reviews.apache.org/r/5073/diff

          Testing
          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION bin/flume-ng 0108997 Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7720
          -----------------------------------------------------------

          Hari, thank you for the patch! I don't plan on doing the full review since I don't have the cycles right now, but I saw this come up right before bed so I thought I would add my input.

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17006>

          Wish: It'd be ideal if we could use mockito to pass it a fake HTable object and then test that transactions are handled correctly if Error and RuntimeException are thrown.

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17005>

          super.stop(); ?

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17004>

          Most components are taking CamelCap parameters. https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17001>

          Maybe we should use a different default row key? I am guessing the row key prefix is supposed to be used to get around hot spotting due to the timestamp. Maybe UUID would be a better default?

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17002>

          Should we specify the UTF-8 charset here?

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17003>

          We are just dropping the headers altogether? I think we should insert them as cells (a flag could turn this off).

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment16998>

          I don't plan on doing the major review of this, but Throwable should be caught here and rollback called. If the throwable is Error or RuntimeException it should be thrown without being wrapped.

          • Brock

          On 2012-05-09 03:04:07, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-09 03:04:07)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          bin/flume-ng 0108997

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7720 ----------------------------------------------------------- Hari, thank you for the patch! I don't plan on doing the full review since I don't have the cycles right now, but I saw this come up right before bed so I thought I would add my input. flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17006 > Wish: It'd be ideal if we could use mockito to pass it a fake HTable object and then test that transactions are handled correctly if Error and RuntimeException are thrown. flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17005 > super.stop(); ? flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17004 > Most components are taking CamelCap parameters. https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17001 > Maybe we should use a different default row key? I am guessing the row key prefix is supposed to be used to get around hot spotting due to the timestamp. Maybe UUID would be a better default? flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17002 > Should we specify the UTF-8 charset here? flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17003 > We are just dropping the headers altogether? I think we should insert them as cells (a flag could turn this off). flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment16998 > I don't plan on doing the major review of this, but Throwable should be caught here and rollback called. If the throwable is Error or RuntimeException it should be thrown without being wrapped. Brock On 2012-05-09 03:04:07, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-09 03:04:07) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION bin/flume-ng 0108997 Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > Hari, thank you for the patch! I don't plan on doing the full review since I don't have the cycles right now, but I saw this come up right before bed so I thought I would add my input.

          Thanks for you comments, Brock! I will make sure that these are addressed in the next patch, which I will submit after someone does a full review.

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 132

          > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line132>

          >

          > Wish: It'd be ideal if we could use mockito to pass it a fake HTable object and then test that transactions are handled correctly if Error and RuntimeException are thrown.

          I am not too familiar with mockito. I will do that in a different patch, when I have time to pick up Mockito.

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 176

          > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line176>

          >

          > super.stop(); ?

          Oops! Will add in the next patch.

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 182

          > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line182>

          >

          > Most components are taking CamelCap parameters. https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java

          I was not actually sure, will change this.

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 231

          > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line231>

          >

          > Maybe we should use a different default row key? I am guessing the row key prefix is supposed to be used to get around hot spotting due to the timestamp. Maybe UUID would be a better default?

          The idea is that the user gives an initial prefix in the conf. This way they can supply different prefixes for different sinks, within the same agent(and later identify which sink each of the rows came from). I agree that using uuid is a better default, but the concerns I have are its size, and also that scans will return the rows in a different order than inserted, while inserting it using timestamps will guarantee that values inserted in a specific order will be returned together. I would like your feedback on that, if that is not a major use case, then I will change it to uuid, since the implementation is also cleaner. Please let me know.

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 234

          > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line234>

          >

          > Should we specify the UTF-8 charset here?

          will do.

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 239

          > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line239>

          >

          > We are just dropping the headers altogether? I think we should insert them as cells (a flag could turn this off).

          I am considering using something like gson to convert the event into json and writing that. We can provide a wrapper called Serializable event, which allows serialize and deserialize. The event serializer is available, but it is not a good fit, and it also expects an output stream.

          I don't want to separate the headers and body while writing to the hbase. Using different cells would do this. Easiest if we can take the byte array form hbase and then convert it back to the json and return the headers and body.

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 263

          > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line263>

          >

          > I don't plan on doing the major review of this, but Throwable should be caught here and rollback called. If the throwable is Error or RuntimeException it should be thrown without being wrapped.

          Will do.

          • Hari

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7720
          -----------------------------------------------------------

          On 2012-05-09 03:04:07, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-09 03:04:07)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          bin/flume-ng 0108997

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-09 05:04:12, Brock Noland wrote: > Hari, thank you for the patch! I don't plan on doing the full review since I don't have the cycles right now, but I saw this come up right before bed so I thought I would add my input. Thanks for you comments, Brock! I will make sure that these are addressed in the next patch, which I will submit after someone does a full review. On 2012-05-09 05:04:12, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 132 > < https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line132 > > > Wish: It'd be ideal if we could use mockito to pass it a fake HTable object and then test that transactions are handled correctly if Error and RuntimeException are thrown. I am not too familiar with mockito. I will do that in a different patch, when I have time to pick up Mockito. On 2012-05-09 05:04:12, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 176 > < https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line176 > > > super.stop(); ? Oops! Will add in the next patch. On 2012-05-09 05:04:12, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 182 > < https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line182 > > > Most components are taking CamelCap parameters. https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java I was not actually sure, will change this. On 2012-05-09 05:04:12, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 231 > < https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line231 > > > Maybe we should use a different default row key? I am guessing the row key prefix is supposed to be used to get around hot spotting due to the timestamp. Maybe UUID would be a better default? The idea is that the user gives an initial prefix in the conf. This way they can supply different prefixes for different sinks, within the same agent(and later identify which sink each of the rows came from). I agree that using uuid is a better default, but the concerns I have are its size, and also that scans will return the rows in a different order than inserted, while inserting it using timestamps will guarantee that values inserted in a specific order will be returned together. I would like your feedback on that, if that is not a major use case, then I will change it to uuid, since the implementation is also cleaner. Please let me know. On 2012-05-09 05:04:12, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 234 > < https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line234 > > > Should we specify the UTF-8 charset here? will do. On 2012-05-09 05:04:12, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 239 > < https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line239 > > > We are just dropping the headers altogether? I think we should insert them as cells (a flag could turn this off). I am considering using something like gson to convert the event into json and writing that. We can provide a wrapper called Serializable event, which allows serialize and deserialize. The event serializer is available, but it is not a good fit, and it also expects an output stream. I don't want to separate the headers and body while writing to the hbase. Using different cells would do this. Easiest if we can take the byte array form hbase and then convert it back to the json and return the headers and body. On 2012-05-09 05:04:12, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 263 > < https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line263 > > > I don't plan on doing the major review of this, but Throwable should be caught here and rollback called. If the throwable is Error or RuntimeException it should be thrown without being wrapped. Will do. Hari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7720 ----------------------------------------------------------- On 2012-05-09 03:04:07, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-09 03:04:07) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION bin/flume-ng 0108997 Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 132

          > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line132>

          >

          > Wish: It'd be ideal if we could use mockito to pass it a fake HTable object and then test that transactions are handled correctly if Error and RuntimeException are thrown.

          Hari Shreedharan wrote:

          I am not too familiar with mockito. I will do that in a different patch, when I have time to pick up Mockito.

          Sounds good, for a future reference FLUME-1131 uses Mockito.

          On 2012-05-09 05:04:12, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 231

          > <https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line231>

          >

          > Maybe we should use a different default row key? I am guessing the row key prefix is supposed to be used to get around hot spotting due to the timestamp. Maybe UUID would be a better default?

          Hari Shreedharan wrote:

          The idea is that the user gives an initial prefix in the conf. This way they can supply different prefixes for different sinks, within the same agent(and later identify which sink each of the rows came from). I agree that using uuid is a better default, but the concerns I have are its size, and also that scans will return the rows in a different order than inserted, while inserting it using timestamps will guarantee that values inserted in a specific order will be returned together. I would like your feedback on that, if that is not a major use case, then I will change it to uuid, since the implementation is also cleaner. Please let me know.

          Keys are so important, I wonder if this isn't an area where we should provide options via an interface and then three default implementations:

          1) Prefixed timestamp (current)
          2) Prefixed reverse timestamp (Long.MAX_VALUE - timestamp) (sorts newest to the top)
          3) uuid/md5sum (randomly distributed keys)
          4) User can extend the interface and generate their own keys based on the headers/body.

          • Brock

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7720
          -----------------------------------------------------------

          On 2012-05-09 03:04:07, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-09 03:04:07)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          bin/flume-ng 0108997

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-09 05:04:12, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 132 > < https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line132 > > > Wish: It'd be ideal if we could use mockito to pass it a fake HTable object and then test that transactions are handled correctly if Error and RuntimeException are thrown. Hari Shreedharan wrote: I am not too familiar with mockito. I will do that in a different patch, when I have time to pick up Mockito. Sounds good, for a future reference FLUME-1131 uses Mockito. On 2012-05-09 05:04:12, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 231 > < https://reviews.apache.org/r/5073/diff/1/?file=107970#file107970line231 > > > Maybe we should use a different default row key? I am guessing the row key prefix is supposed to be used to get around hot spotting due to the timestamp. Maybe UUID would be a better default? Hari Shreedharan wrote: The idea is that the user gives an initial prefix in the conf. This way they can supply different prefixes for different sinks, within the same agent(and later identify which sink each of the rows came from). I agree that using uuid is a better default, but the concerns I have are its size, and also that scans will return the rows in a different order than inserted, while inserting it using timestamps will guarantee that values inserted in a specific order will be returned together. I would like your feedback on that, if that is not a major use case, then I will change it to uuid, since the implementation is also cleaner. Please let me know. Keys are so important, I wonder if this isn't an area where we should provide options via an interface and then three default implementations: 1) Prefixed timestamp (current) 2) Prefixed reverse timestamp (Long.MAX_VALUE - timestamp) (sorts newest to the top) 3) uuid/md5sum (randomly distributed keys) 4) User can extend the interface and generate their own keys based on the headers/body. Brock ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7720 ----------------------------------------------------------- On 2012-05-09 03:04:07, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-09 03:04:07) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION bin/flume-ng 0108997 Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7759
          -----------------------------------------------------------

          I've taken a look over this too and don't really have anything to add to brocks comments.
          I've also compiled it and run the tests and everything seems good though needed to run mvn install first, and set my umask to 022 (otherwise the minicluster datanode would complain and abort everything).

          I don't see anything wrong with it, and once you've added the changes suggested by brock I'll put it through a quick spin on a test machine and it should probably be good to go.

          • Juhani

          On 2012-05-09 03:04:07, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-09 03:04:07)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          bin/flume-ng 0108997

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7759 ----------------------------------------------------------- I've taken a look over this too and don't really have anything to add to brocks comments. I've also compiled it and run the tests and everything seems good though needed to run mvn install first, and set my umask to 022 (otherwise the minicluster datanode would complain and abort everything). I don't see anything wrong with it, and once you've added the changes suggested by brock I'll put it through a quick spin on a test machine and it should probably be good to go. Juhani On 2012-05-09 03:04:07, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-09 03:04:07) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION bin/flume-ng 0108997 Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/
          -----------------------------------------------------------

          (Updated 2012-05-11 20:25:23.775754)

          Review request for Flume.

          Changes
          -------

          Moving some functionality out to Hbase event serializer.

          Summary
          -------

          Hbase sink.

          This addresses bug FLUME-1183.
          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs (updated)


          bin/flume-ng 0108997
          flume-ng-dist/pom.xml 5bdcfe7
          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/RowKeyGenerator.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION
          flume-ng-sinks/pom.xml acb3087
          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing
          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-11 20:25:23.775754) Review request for Flume. Changes ------- Moving some functionality out to Hbase event serializer. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs (updated) bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/RowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/
          -----------------------------------------------------------

          (Updated 2012-05-11 20:29:38.540592)

          Review request for Flume.

          Changes
          -------

          Removed row key generator interface.

          Summary
          -------

          Hbase sink.

          This addresses bug FLUME-1183.
          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs (updated)


          bin/flume-ng 0108997
          flume-ng-dist/pom.xml 5bdcfe7
          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION
          flume-ng-sinks/pom.xml acb3087
          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing
          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-11 20:29:38.540592) Review request for Flume. Changes ------- Removed row key generator interface. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs (updated) bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7815
          -----------------------------------------------------------

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java
          <https://reviews.apache.org/r/5073/#comment17146>

          It's very common for people to use "promoted fields" in their key. I think that the user should be able to use configuration to plugin code which generates a key.

          • Brock

          On 2012-05-11 20:29:38, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-11 20:29:38)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7815 ----------------------------------------------------------- flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java < https://reviews.apache.org/r/5073/#comment17146 > It's very common for people to use "promoted fields" in their key. I think that the user should be able to use configuration to plugin code which generates a key. Brock On 2012-05-11 20:29:38, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-11 20:29:38) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-11 22:02:52, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java, line 119

          > <https://reviews.apache.org/r/5073/diff/3/?file=108615#file108615line119>

          >

          > It's very common for people to use "promoted fields" in their key. I think that the user should be able to use configuration to plugin code which generates a key.

          Hi Brock,

          This is only an example class. They can use any class that implements the HbaseEventHandler interface(as noted in the java doc for the HBaseSink). So this was just added as a demo for the sink's serializer component. Similarly, the SimpleRowKeyGenerator is also a utility.

          • Hari

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7815
          -----------------------------------------------------------

          On 2012-05-11 20:29:38, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-11 20:29:38)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-11 22:02:52, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java, line 119 > < https://reviews.apache.org/r/5073/diff/3/?file=108615#file108615line119 > > > It's very common for people to use "promoted fields" in their key. I think that the user should be able to use configuration to plugin code which generates a key. Hi Brock, This is only an example class. They can use any class that implements the HbaseEventHandler interface(as noted in the java doc for the HBaseSink). So this was just added as a demo for the sink's serializer component. Similarly, the SimpleRowKeyGenerator is also a utility. Hari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7815 ----------------------------------------------------------- On 2012-05-11 20:29:38, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-11 20:29:38) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-11 22:02:52, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java, line 119

          > <https://reviews.apache.org/r/5073/diff/3/?file=108615#file108615line119>

          >

          > It's very common for people to use "promoted fields" in their key. I think that the user should be able to use configuration to plugin code which generates a key.

          Hari Shreedharan wrote:

          Hi Brock,

          This is only an example class. They can use any class that implements the HbaseEventHandler interface(as noted in the java doc for the HBaseSink). So this was just added as a demo for the sink's serializer component. Similarly, the SimpleRowKeyGenerator is also a utility.

          I see now! Sorry.

          • Brock

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7815
          -----------------------------------------------------------

          On 2012-05-11 20:29:38, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-11 20:29:38)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-11 22:02:52, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java, line 119 > < https://reviews.apache.org/r/5073/diff/3/?file=108615#file108615line119 > > > It's very common for people to use "promoted fields" in their key. I think that the user should be able to use configuration to plugin code which generates a key. Hari Shreedharan wrote: Hi Brock, This is only an example class. They can use any class that implements the HbaseEventHandler interface(as noted in the java doc for the HBaseSink). So this was just added as a demo for the sink's serializer component. Similarly, the SimpleRowKeyGenerator is also a utility. I see now! Sorry. Brock ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7815 ----------------------------------------------------------- On 2012-05-11 20:29:38, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-11 20:29:38) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7828
          -----------------------------------------------------------

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17153>

          Doesn't look like you can set this?

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17155>

          There are some missing @Override's here, correct?

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17154>

          cf.getBytes(Charsets.UTF8) doesn't throw a checked exception.

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17156>

          Shouldn't we flush the table after batchSize has been processed?

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17157>

          Since we have a pending exception to throw, how about logging any Exception from rollback? That is the tack I took in the ChannelProcessor.

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java
          <https://reviews.apache.org/r/5073/#comment17158>

          Same as above, an alternative getBytes() method does not throw the checked exception.

          • Brock

          On 2012-05-11 20:29:38, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-11 20:29:38)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7828 ----------------------------------------------------------- flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17153 > Doesn't look like you can set this? flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17155 > There are some missing @Override's here, correct? flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17154 > cf.getBytes(Charsets.UTF8) doesn't throw a checked exception. flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17156 > Shouldn't we flush the table after batchSize has been processed? flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17157 > Since we have a pending exception to throw, how about logging any Exception from rollback? That is the tack I took in the ChannelProcessor. flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java < https://reviews.apache.org/r/5073/#comment17158 > Same as above, an alternative getBytes() method does not throw the checked exception. Brock On 2012-05-11 20:29:38, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-11 20:29:38) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-12 15:42:20, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 103

          > <https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line103>

          >

          > There are some missing @Override's here, correct?

          Yep. Adding it.

          On 2012-05-12 15:42:20, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 201

          > <https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line201>

          >

          > Shouldn't we flush the table after batchSize has been processed?

          Actually, it is ok not to. This is because we use Htable.batch(), which flushes immediately. Even the increment is supposed to send individual increments immediately to HBase. Only puts are affected by flushes.

          On 2012-05-12 15:42:20, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 164

          > <https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line164>

          >

          > cf.getBytes(Charsets.UTF8) doesn't throw a checked exception.

          Done.

          On 2012-05-12 15:42:20, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 216

          > <https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line216>

          >

          > Since we have a pending exception to throw, how about logging any Exception from rollback? That is the tack I took in the ChannelProcessor.

          Done.

          On 2012-05-12 15:42:20, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java, line 96

          > <https://reviews.apache.org/r/5073/diff/3/?file=108615#file108615line96>

          >

          > Same as above, an alternative getBytes() method does not throw the checked exception.

          Done.

          On 2012-05-12 15:42:20, Brock Noland wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 87

          > <https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line87>

          >

          > Doesn't look like you can set this?

          Seems like the last update, I removed the configurability of this - will put it back.

          • Hari

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7828
          -----------------------------------------------------------

          On 2012-05-11 20:29:38, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-11 20:29:38)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-12 15:42:20, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 103 > < https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line103 > > > There are some missing @Override's here, correct? Yep. Adding it. On 2012-05-12 15:42:20, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 201 > < https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line201 > > > Shouldn't we flush the table after batchSize has been processed? Actually, it is ok not to. This is because we use Htable.batch(), which flushes immediately. Even the increment is supposed to send individual increments immediately to HBase. Only puts are affected by flushes. On 2012-05-12 15:42:20, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 164 > < https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line164 > > > cf.getBytes(Charsets.UTF8) doesn't throw a checked exception. Done. On 2012-05-12 15:42:20, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 216 > < https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line216 > > > Since we have a pending exception to throw, how about logging any Exception from rollback? That is the tack I took in the ChannelProcessor. Done. On 2012-05-12 15:42:20, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java, line 96 > < https://reviews.apache.org/r/5073/diff/3/?file=108615#file108615line96 > > > Same as above, an alternative getBytes() method does not throw the checked exception. Done. On 2012-05-12 15:42:20, Brock Noland wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 87 > < https://reviews.apache.org/r/5073/diff/3/?file=108613#file108613line87 > > > Doesn't look like you can set this? Seems like the last update, I removed the configurability of this - will put it back. Hari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7828 ----------------------------------------------------------- On 2012-05-11 20:29:38, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-11 20:29:38) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/
          -----------------------------------------------------------

          (Updated 2012-05-14 09:02:56.900957)

          Review request for Flume.

          Changes
          -------

          Incorporated Brock's feedback.

          Summary
          -------

          Hbase sink.

          This addresses bug FLUME-1183.
          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs (updated)


          bin/flume-ng 0108997
          flume-ng-dist/pom.xml 5bdcfe7
          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION
          flume-ng-sinks/pom.xml acb3087
          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing
          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-14 09:02:56.900957) Review request for Flume. Changes ------- Incorporated Brock's feedback. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs (updated) bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7835
          -----------------------------------------------------------

          The added customizability is nice to have.

          Confirmed that the unit test works, however it does not verify some failure cases(at least no 2 and 3 that I suspect below).

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17168>

          I'm not entirely convinced by the need to set all puts to the same column family, when everything else is pretty customizable. Saving on transferring the columnFamily data everytime? Perhaps make it overridable with an optional header.

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17170>

          Looks to me like if there is no configuration for this in the context, you'll be passing a null here... Which will then result in a NPE?

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17169>

          I think you should be calling putEventsAndCommit here? Otherwise it looks to me like you're losing anything that was take'd between the last PEAC and reaching the batchsize

          • Juhani

          On 2012-05-14 09:02:56, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-14 09:02:56)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7835 ----------------------------------------------------------- The added customizability is nice to have. Confirmed that the unit test works, however it does not verify some failure cases(at least no 2 and 3 that I suspect below). flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17168 > I'm not entirely convinced by the need to set all puts to the same column family, when everything else is pretty customizable. Saving on transferring the columnFamily data everytime? Perhaps make it overridable with an optional header. flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17170 > Looks to me like if there is no configuration for this in the context, you'll be passing a null here... Which will then result in a NPE? flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17169 > I think you should be calling putEventsAndCommit here? Otherwise it looks to me like you're losing anything that was take'd between the last PEAC and reaching the batchsize Juhani On 2012-05-14 09:02:56, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-14 09:02:56) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/
          -----------------------------------------------------------

          (Updated 2012-05-14 18:31:15.214613)

          Review request for Flume.

          Changes
          -------

          Thanks for your feedback Juhani! Incorporating your feedback.

          Summary
          -------

          Hbase sink.

          This addresses bug FLUME-1183.
          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs (updated)


          bin/flume-ng 0108997
          flume-ng-dist/pom.xml 5bdcfe7
          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION
          flume-ng-sinks/pom.xml acb3087
          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing
          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-14 18:31:15.214613) Review request for Flume. Changes ------- Thanks for your feedback Juhani! Incorporating your feedback. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs (updated) bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-14 10:20:20, Juhani Connolly wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 152

          > <https://reviews.apache.org/r/5073/diff/4/?file=108798#file108798line152>

          >

          > I'm not entirely convinced by the need to set all puts to the same column family, when everything else is pretty customizable. Saving on transferring the columnFamily data everytime? Perhaps make it overridable with an optional header.

          I am going to file a bug for this, and for adding unit tests.

          On 2012-05-14 10:20:20, Juhani Connolly wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 175

          > <https://reviews.apache.org/r/5073/diff/4/?file=108798#file108798line175>

          >

          > Looks to me like if there is no configuration for this in the context, you'll be passing a null here... Which will then result in a NPE?

          fixed.

          On 2012-05-14 10:20:20, Juhani Connolly wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 203

          > <https://reviews.apache.org/r/5073/diff/4/?file=108798#file108798line203>

          >

          > I think you should be calling putEventsAndCommit here? Otherwise it looks to me like you're losing anything that was take'd between the last PEAC and reaching the batchsize

          Fixed.

          • Hari

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7835
          -----------------------------------------------------------

          On 2012-05-14 18:31:15, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-14 18:31:15)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-14 10:20:20, Juhani Connolly wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 152 > < https://reviews.apache.org/r/5073/diff/4/?file=108798#file108798line152 > > > I'm not entirely convinced by the need to set all puts to the same column family, when everything else is pretty customizable. Saving on transferring the columnFamily data everytime? Perhaps make it overridable with an optional header. I am going to file a bug for this, and for adding unit tests. On 2012-05-14 10:20:20, Juhani Connolly wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 175 > < https://reviews.apache.org/r/5073/diff/4/?file=108798#file108798line175 > > > Looks to me like if there is no configuration for this in the context, you'll be passing a null here... Which will then result in a NPE? fixed. On 2012-05-14 10:20:20, Juhani Connolly wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 203 > < https://reviews.apache.org/r/5073/diff/4/?file=108798#file108798line203 > > > I think you should be calling putEventsAndCommit here? Otherwise it looks to me like you're losing anything that was take'd between the last PEAC and reaching the batchsize Fixed. Hari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7835 ----------------------------------------------------------- On 2012-05-14 18:31:15, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-14 18:31:15) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7873
          -----------------------------------------------------------

          Thanks for the fixe, looks good, I had missed these two on my check yesterday evening, but otherwise everything looks good to me.

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          <https://reviews.apache.org/r/5073/#comment17202>

          You should probably be checking the results here.

          Error handling for partial successes seems like it would be awkward, so might want to just get this committed and file another ticket for it.

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java
          <https://reviews.apache.org/r/5073/#comment17203>

          I'm not sure what this is for? Leftovers from a refactoring?

          • Juhani

          On 2012-05-14 18:31:15, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-14 18:31:15)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7873 ----------------------------------------------------------- Thanks for the fixe, looks good, I had missed these two on my check yesterday evening, but otherwise everything looks good to me. flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java < https://reviews.apache.org/r/5073/#comment17202 > You should probably be checking the results here. Error handling for partial successes seems like it would be awkward, so might want to just get this committed and file another ticket for it. flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java < https://reviews.apache.org/r/5073/#comment17203 > I'm not sure what this is for? Leftovers from a refactoring? Juhani On 2012-05-14 18:31:15, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-14 18:31:15) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-14 23:40:24, Juhani Connolly wrote:

          > Thanks for the fixe, looks good, I had missed these two on my check yesterday evening, but otherwise everything looks good to me.

          Thanks Juhani. I removed the comment mentioned. Regarding the partial success, we should probably look into it later. I will file a jira for that once this is committed.

          On 2012-05-14 23:40:24, Juhani Connolly wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 210

          > <https://reviews.apache.org/r/5073/diff/5/?file=108844#file108844line210>

          >

          > You should probably be checking the results here.

          >

          > Error handling for partial successes seems like it would be awkward, so might want to just get this committed and file another ticket for it.

          Yes. The only reason I didn't do that here was, I don't really know how far we should go with our contract. I don't think we should really bother if the HBase writes fail due to something outside our control. Hbase failures,we cause the transaction to be rolled back, but I don't think we should bother about individual failures. We can file a jira, and discuss it there - once we make a conclusive decision on that we can add it.

          So for now, I am not going to change this - once we decide, we can put it in a patch.

          On 2012-05-14 23:40:24, Juhani Connolly wrote:

          > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java, lines 119-121

          > <https://reviews.apache.org/r/5073/diff/5/?file=108846#file108846line119>

          >

          > I'm not sure what this is for? Leftovers from a refactoring?

          Yes, seems like. Removing this and uploading new patch.

          • Hari

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7873
          -----------------------------------------------------------

          On 2012-05-14 18:31:15, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-14 18:31:15)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-14 23:40:24, Juhani Connolly wrote: > Thanks for the fixe, looks good, I had missed these two on my check yesterday evening, but otherwise everything looks good to me. Thanks Juhani. I removed the comment mentioned. Regarding the partial success, we should probably look into it later. I will file a jira for that once this is committed. On 2012-05-14 23:40:24, Juhani Connolly wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java, line 210 > < https://reviews.apache.org/r/5073/diff/5/?file=108844#file108844line210 > > > You should probably be checking the results here. > > Error handling for partial successes seems like it would be awkward, so might want to just get this committed and file another ticket for it. Yes. The only reason I didn't do that here was, I don't really know how far we should go with our contract. I don't think we should really bother if the HBase writes fail due to something outside our control. Hbase failures,we cause the transaction to be rolled back, but I don't think we should bother about individual failures. We can file a jira, and discuss it there - once we make a conclusive decision on that we can add it. So for now, I am not going to change this - once we decide, we can put it in a patch. On 2012-05-14 23:40:24, Juhani Connolly wrote: > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java, lines 119-121 > < https://reviews.apache.org/r/5073/diff/5/?file=108846#file108846line119 > > > I'm not sure what this is for? Leftovers from a refactoring? Yes, seems like. Removing this and uploading new patch. Hari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7873 ----------------------------------------------------------- On 2012-05-14 18:31:15, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-14 18:31:15) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/
          -----------------------------------------------------------

          (Updated 2012-05-15 00:03:00.888106)

          Review request for Flume.

          Changes
          -------

          Removed comment, will attach patch to jira.

          Summary
          -------

          Hbase sink.

          This addresses bug FLUME-1183.
          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs (updated)


          bin/flume-ng 0108997
          flume-ng-dist/pom.xml 5bdcfe7
          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION
          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION
          flume-ng-sinks/pom.xml acb3087
          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing
          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-15 00:03:00.888106) Review request for Flume. Changes ------- Removed comment, will attach patch to jira. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs (updated) bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7876
          -----------------------------------------------------------

          Ship it!

          lgtm, a couple of discussed issues(checking put results and unit tests) to be handled in separate issues.

          • Juhani

          On 2012-05-15 00:03:00, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-15 00:03:00)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7876 ----------------------------------------------------------- Ship it! lgtm, a couple of discussed issues(checking put results and unit tests) to be handled in separate issues. Juhani On 2012-05-15 00:03:00, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-15 00:03:00) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-15 00:08:55, Juhani Connolly wrote:

          > lgtm, a couple of discussed issues(checking put results and unit tests) to be handled in separate issues.

          Thanks Juhani. Jira server seems to have some issues - not letting me upload the patch. I will upload it as soon as it is back up. Filed FLUME-1203 and FLUME-1204 for the unit tests and individual results.

          • Hari

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7876
          -----------------------------------------------------------

          On 2012-05-15 00:03:00, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-15 00:03:00)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-15 00:08:55, Juhani Connolly wrote: > lgtm, a couple of discussed issues(checking put results and unit tests) to be handled in separate issues. Thanks Juhani. Jira server seems to have some issues - not letting me upload the patch. I will upload it as soon as it is back up. Filed FLUME-1203 and FLUME-1204 for the unit tests and individual results. Hari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7876 ----------------------------------------------------------- On 2012-05-15 00:03:00, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-15 00:03:00) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7886
          -----------------------------------------------------------

          Ship it!

          • Brock

          On 2012-05-15 00:03:00, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-15 00:03:00)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7886 ----------------------------------------------------------- Ship it! Brock On 2012-05-15 00:03:00, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-15 00:03:00) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-15 05:54:16, Brock Noland wrote:

          >

          Please attach the patch to the JIRA.

          • Brock

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7886
          -----------------------------------------------------------

          On 2012-05-15 00:03:00, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-15 00:03:00)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-15 05:54:16, Brock Noland wrote: > Please attach the patch to the JIRA. Brock ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7886 ----------------------------------------------------------- On 2012-05-15 00:03:00, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-15 00:03:00) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-05-15 05:54:16, Brock Noland wrote:

          >

          Brock Noland wrote:

          Please attach the patch to the JIRA.

          Sorry, I see you tried!

          • Brock

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/5073/#review7886
          -----------------------------------------------------------

          On 2012-05-15 00:03:00, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/5073/

          -----------------------------------------------------------

          (Updated 2012-05-15 00:03:00)

          Review request for Flume.

          Summary

          -------

          Hbase sink.

          This addresses bug FLUME-1183.

          https://issues.apache.org/jira/browse/FLUME-1183

          Diffs

          -----

          bin/flume-ng 0108997

          flume-ng-dist/pom.xml 5bdcfe7

          flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION

          flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION

          flume-ng-sinks/pom.xml acb3087

          pom.xml 8c11a2d

          Diff: https://reviews.apache.org/r/5073/diff

          Testing

          -------

          Unit tests added

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-05-15 05:54:16, Brock Noland wrote: > Brock Noland wrote: Please attach the patch to the JIRA. Sorry, I see you tried! Brock ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/#review7886 ----------------------------------------------------------- On 2012-05-15 00:03:00, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5073/ ----------------------------------------------------------- (Updated 2012-05-15 00:03:00) Review request for Flume. Summary ------- Hbase sink. This addresses bug FLUME-1183 . https://issues.apache.org/jira/browse/FLUME-1183 Diffs ----- bin/flume-ng 0108997 flume-ng-dist/pom.xml 5bdcfe7 flume-ng-sinks/flume-ng-hbase-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java PRE-CREATION flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java PRE-CREATION flume-ng-sinks/pom.xml acb3087 pom.xml 8c11a2d Diff: https://reviews.apache.org/r/5073/diff Testing ------- Unit tests added Thanks, Hari
          Hide
          Arvind Prabhakar added a comment -

          Patch committed. Thanks Hari!

          Show
          Arvind Prabhakar added a comment - Patch committed. Thanks Hari!
          Hide
          Hudson added a comment -

          Integrated in flume-trunk #204 (See https://builds.apache.org/job/flume-trunk/204/)
          FLUME-1183. Implement an HBase Sink which supports table level access.

          (Hari Shreedharan via Arvind Prabhakar) (Revision 1338820)

          Result = SUCCESS
          arvind : http://svn.apache.org/viewvc/?view=rev&rev=1338820
          Files :

          • /incubator/flume/trunk/bin/flume-ng
          • /incubator/flume/trunk/flume-ng-dist/pom.xml
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/pom.xml
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase
          • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java
          • /incubator/flume/trunk/flume-ng-sinks/pom.xml
          • /incubator/flume/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in flume-trunk #204 (See https://builds.apache.org/job/flume-trunk/204/ ) FLUME-1183 . Implement an HBase Sink which supports table level access. (Hari Shreedharan via Arvind Prabhakar) (Revision 1338820) Result = SUCCESS arvind : http://svn.apache.org/viewvc/?view=rev&rev=1338820 Files : /incubator/flume/trunk/bin/flume-ng /incubator/flume/trunk/flume-ng-dist/pom.xml /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/pom.xml /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HbaseEventSerializer.java /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleHbaseEventSerializer.java /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/SimpleRowKeyGenerator.java /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java /incubator/flume/trunk/flume-ng-sinks/pom.xml /incubator/flume/trunk/pom.xml

            People

            • Assignee:
              Hari Shreedharan
              Reporter:
              Hari Shreedharan
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development