Flume
  1. Flume
  2. FLUME-1974

Thrift compatibility issue with hbase-0.92

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.4.0
    • Component/s: None
    • Labels:
      None

      Description

      HBase-0.92 pulls in Thrift 0.7.0, while Flume requires at least Thrift 0.9.0. We must downgrade thrift to restore apps which may require both the flume sdk and hbase client libraries. Adding both hbase-0.92 and flume in the same pom results in a java.lang.ClassNotFoundException: org.apache.thrift.scheme.StandardScheme.

      1. FLUME-1974.patch
        110 kB
        Hari Shreedharan

        Activity

        Hide
        Hari Shreedharan added a comment -

        Most of the code is generated code. The code change itself is minimal.

        Show
        Hari Shreedharan added a comment - Most of the code is generated code. The code change itself is minimal.
        Hide
        Hari Shreedharan added a comment -

        The relevant changes are in the top level pom and ThriftSource.java - all other changes are just changes in generated code.

        Show
        Hari Shreedharan added a comment - The relevant changes are in the top level pom and ThriftSource.java - all other changes are just changes in generated code.
        Hide
        Hari Shreedharan added a comment -

        Seems like in hbase-94 we can use thrift-0.8+. The TThreadSelectorServer gives far better perf (available only in thrift-0.8), so we should use that in the Hadoop-2 profile. It will be a bit "dirty", since we need to use reflection, but I think it is worth it.

        Show
        Hari Shreedharan added a comment - Seems like in hbase-94 we can use thrift-0.8+. The TThreadSelectorServer gives far better perf (available only in thrift-0.8), so we should use that in the Hadoop-2 profile. It will be a bit "dirty", since we need to use reflection, but I think it is worth it.
        Hide
        Mike Percy added a comment -

        +1, lgtm, good to reduce the hbase version dep for compat. Unit tests pass so I'm committing this.

        Show
        Mike Percy added a comment - +1, lgtm, good to reduce the hbase version dep for compat. Unit tests pass so I'm committing this.
        Hide
        Mike Percy added a comment -

        Pushed to trunk & flume-1.4 branches. Thanks for the patch Hari!

        Show
        Mike Percy added a comment - Pushed to trunk & flume-1.4 branches. Thanks for the patch Hari!
        Hide
        Hudson added a comment -

        Integrated in flume-trunk #389 (See https://builds.apache.org/job/flume-trunk/389/)
        FLUME-1974. Thrift compatibility issue with hbase-0.92. (Revision c72a3b1a5a2548f9157fe7913611c4318ce5e64d)

        Result = UNSTABLE
        mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c72a3b1a5a2548f9157fe7913611c4318ce5e64d
        Files :

        • pom.xml
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/EventStatus.java
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEvent.java
        • flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/LogEntry.java
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/Priority.java
        • flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ResultCode.java
        • flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java
        • flume-ng-sdk/src/main/java/org/apache/flume/thrift/Status.java
        • flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftFlumeEvent.java
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java
        • flume-ng-core/src/main/java/org/apache/flume/source/ThriftSource.java
        • flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java
        Show
        Hudson added a comment - Integrated in flume-trunk #389 (See https://builds.apache.org/job/flume-trunk/389/ ) FLUME-1974 . Thrift compatibility issue with hbase-0.92. (Revision c72a3b1a5a2548f9157fe7913611c4318ce5e64d) Result = UNSTABLE mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c72a3b1a5a2548f9157fe7913611c4318ce5e64d Files : pom.xml flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/EventStatus.java flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEvent.java flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/LogEntry.java flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/Priority.java flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ResultCode.java flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java flume-ng-sdk/src/main/java/org/apache/flume/thrift/Status.java flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftFlumeEvent.java flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java flume-ng-core/src/main/java/org/apache/flume/source/ThriftSource.java flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java

          People

          • Assignee:
            Hari Shreedharan
            Reporter:
            Hari Shreedharan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development