Flume
  1. Flume
  2. FLUME-1212

Flume should pick HBase jars from HBASE_HOME

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.2.0
    • Component/s: Technical Debt
    • Labels:
      None

      Description

      Currently Flume distribution bundles the following in the lib directory:

      • hbase-0.92.1.jar
      • hadoop-core-1.0.1.jar
      • zookeeper-3.4.3.jar

      Instead it should follow the same mechanism that is used for HDFS sink - where the necessary jars are picked up dynamically from the set HBASE_HOME, preferably using the hbase scripts that enumerate it's classpath.

      1. FLUME-1212.patch
        0.6 kB
        Hari Shreedharan

        Activity

        Hide
        Hudson added a comment -

        Integrated in flume-trunk #206 (See https://builds.apache.org/job/flume-trunk/206/)
        FLUME-1212. Flume should pick HBase jars from HBASE_HOME.

        (Hari Shreedharan via Arvind Prabhakar) (Revision 1339773)

        Result = SUCCESS
        arvind : http://svn.apache.org/viewvc/?view=rev&rev=1339773
        Files :

        • /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/pom.xml
        Show
        Hudson added a comment - Integrated in flume-trunk #206 (See https://builds.apache.org/job/flume-trunk/206/ ) FLUME-1212 . Flume should pick HBase jars from HBASE_HOME. (Hari Shreedharan via Arvind Prabhakar) (Revision 1339773) Result = SUCCESS arvind : http://svn.apache.org/viewvc/?view=rev&rev=1339773 Files : /incubator/flume/trunk/flume-ng-sinks/flume-ng-hbase-sink/pom.xml
        Hide
        Arvind Prabhakar added a comment -

        Patch committed. Thanks Hari.

        Show
        Arvind Prabhakar added a comment - Patch committed. Thanks Hari.
        Hide
        Mingjie Lai added a comment -

        @hari, @arvind.

        What if $HBASE_HOME is not set? I don't think HBASE_HOME or HADOOP_HOME is required to run a hbase or hadoop client. Loading jars by an env variable can be problematic. Eventually, the right way is to set classpath at something like flume-env.sh, etc.

        Show
        Mingjie Lai added a comment - @hari, @arvind. What if $HBASE_HOME is not set? I don't think HBASE_HOME or HADOOP_HOME is required to run a hbase or hadoop client. Loading jars by an env variable can be problematic. Eventually, the right way is to set classpath at something like flume-env.sh, etc.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/5155/#review7960
        -----------------------------------------------------------

        Ship it!

        +1

        • Arvind

        On 2012-05-17 16:53:21, Hari Shreedharan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/5155/

        -----------------------------------------------------------

        (Updated 2012-05-17 16:53:21)

        Review request for Flume.

        Summary

        -------

        Making hbase and hadoop optional.

        This addresses bug FLUME-1212.

        https://issues.apache.org/jira/browse/FLUME-1212

        Diffs

        -----

        flume-ng-sinks/flume-ng-hbase-sink/pom.xml f20decd

        Diff: https://reviews.apache.org/r/5155/diff

        Testing

        -------

        Verified they are no longer in lib folder.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5155/#review7960 ----------------------------------------------------------- Ship it! +1 Arvind On 2012-05-17 16:53:21, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5155/ ----------------------------------------------------------- (Updated 2012-05-17 16:53:21) Review request for Flume. Summary ------- Making hbase and hadoop optional. This addresses bug FLUME-1212 . https://issues.apache.org/jira/browse/FLUME-1212 Diffs ----- flume-ng-sinks/flume-ng-hbase-sink/pom.xml f20decd Diff: https://reviews.apache.org/r/5155/diff Testing ------- Verified they are no longer in lib folder. Thanks, Hari
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/5155/
        -----------------------------------------------------------

        Review request for Flume.

        Summary
        -------

        Making hbase and hadoop optional.

        This addresses bug FLUME-1212.
        https://issues.apache.org/jira/browse/FLUME-1212

        Diffs


        flume-ng-sinks/flume-ng-hbase-sink/pom.xml f20decd

        Diff: https://reviews.apache.org/r/5155/diff

        Testing
        -------

        Verified they are no longer in lib folder.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5155/ ----------------------------------------------------------- Review request for Flume. Summary ------- Making hbase and hadoop optional. This addresses bug FLUME-1212 . https://issues.apache.org/jira/browse/FLUME-1212 Diffs flume-ng-sinks/flume-ng-hbase-sink/pom.xml f20decd Diff: https://reviews.apache.org/r/5155/diff Testing ------- Verified they are no longer in lib folder. Thanks, Hari

          People

          • Assignee:
            Hari Shreedharan
            Reporter:
            Arvind Prabhakar
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development