Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1974

Thrift compatibility issue with hbase-0.92

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0
    • Component/s: None
    • Labels:
      None

      Description

      HBase-0.92 pulls in Thrift 0.7.0, while Flume requires at least Thrift 0.9.0. We must downgrade thrift to restore apps which may require both the flume sdk and hbase client libraries. Adding both hbase-0.92 and flume in the same pom results in a java.lang.ClassNotFoundException: org.apache.thrift.scheme.StandardScheme.

      1. FLUME-1974.patch
        110 kB
        Hari Shreedharan

        Activity

        Hide
        hudson Hudson added a comment -

        Integrated in flume-trunk #389 (See https://builds.apache.org/job/flume-trunk/389/)
        FLUME-1974. Thrift compatibility issue with hbase-0.92. (Revision c72a3b1a5a2548f9157fe7913611c4318ce5e64d)

        Result = UNSTABLE
        mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c72a3b1a5a2548f9157fe7913611c4318ce5e64d
        Files :

        • pom.xml
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/EventStatus.java
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEvent.java
        • flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/LogEntry.java
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/Priority.java
        • flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ResultCode.java
        • flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java
        • flume-ng-sdk/src/main/java/org/apache/flume/thrift/Status.java
        • flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftFlumeEvent.java
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java
        • flume-ng-core/src/main/java/org/apache/flume/source/ThriftSource.java
        • flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java
        Show
        hudson Hudson added a comment - Integrated in flume-trunk #389 (See https://builds.apache.org/job/flume-trunk/389/ ) FLUME-1974 . Thrift compatibility issue with hbase-0.92. (Revision c72a3b1a5a2548f9157fe7913611c4318ce5e64d) Result = UNSTABLE mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c72a3b1a5a2548f9157fe7913611c4318ce5e64d Files : pom.xml flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/EventStatus.java flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEvent.java flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/LogEntry.java flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/Priority.java flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ResultCode.java flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java flume-ng-sdk/src/main/java/org/apache/flume/thrift/Status.java flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftFlumeEvent.java flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java flume-ng-core/src/main/java/org/apache/flume/source/ThriftSource.java flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java
        Hide
        mpercy Mike Percy added a comment -

        Pushed to trunk & flume-1.4 branches. Thanks for the patch Hari!

        Show
        mpercy Mike Percy added a comment - Pushed to trunk & flume-1.4 branches. Thanks for the patch Hari!
        Hide
        mpercy Mike Percy added a comment -

        +1, lgtm, good to reduce the hbase version dep for compat. Unit tests pass so I'm committing this.

        Show
        mpercy Mike Percy added a comment - +1, lgtm, good to reduce the hbase version dep for compat. Unit tests pass so I'm committing this.
        Hide
        hshreedharan Hari Shreedharan added a comment -

        Seems like in hbase-94 we can use thrift-0.8+. The TThreadSelectorServer gives far better perf (available only in thrift-0.8), so we should use that in the Hadoop-2 profile. It will be a bit "dirty", since we need to use reflection, but I think it is worth it.

        Show
        hshreedharan Hari Shreedharan added a comment - Seems like in hbase-94 we can use thrift-0.8+. The TThreadSelectorServer gives far better perf (available only in thrift-0.8), so we should use that in the Hadoop-2 profile. It will be a bit "dirty", since we need to use reflection, but I think it is worth it.
        Hide
        hshreedharan Hari Shreedharan added a comment -

        The relevant changes are in the top level pom and ThriftSource.java - all other changes are just changes in generated code.

        Show
        hshreedharan Hari Shreedharan added a comment - The relevant changes are in the top level pom and ThriftSource.java - all other changes are just changes in generated code.
        Hide
        hshreedharan Hari Shreedharan added a comment -

        Most of the code is generated code. The code change itself is minimal.

        Show
        hshreedharan Hari Shreedharan added a comment - Most of the code is generated code. The code change itself is minimal.

          People

          • Assignee:
            hshreedharan Hari Shreedharan
            Reporter:
            hshreedharan Hari Shreedharan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development