Flume
  1. Flume
  2. FLUME-1974

Thrift compatibility issue with hbase-0.92

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.4.0
    • Component/s: None
    • Labels:
      None

      Description

      HBase-0.92 pulls in Thrift 0.7.0, while Flume requires at least Thrift 0.9.0. We must downgrade thrift to restore apps which may require both the flume sdk and hbase client libraries. Adding both hbase-0.92 and flume in the same pom results in a java.lang.ClassNotFoundException: org.apache.thrift.scheme.StandardScheme.

      1. FLUME-1974.patch
        110 kB
        Hari Shreedharan

        Activity

        Hari Shreedharan created issue -
        Hide
        Hari Shreedharan added a comment -

        Most of the code is generated code. The code change itself is minimal.

        Show
        Hari Shreedharan added a comment - Most of the code is generated code. The code change itself is minimal.
        Hari Shreedharan made changes -
        Field Original Value New Value
        Attachment FLUME-1974.patch [ 12577429 ]
        Hide
        Hari Shreedharan added a comment -

        The relevant changes are in the top level pom and ThriftSource.java - all other changes are just changes in generated code.

        Show
        Hari Shreedharan added a comment - The relevant changes are in the top level pom and ThriftSource.java - all other changes are just changes in generated code.
        Hari Shreedharan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Hari Shreedharan [ hshreedharan ]
        Hide
        Hari Shreedharan added a comment -

        Seems like in hbase-94 we can use thrift-0.8+. The TThreadSelectorServer gives far better perf (available only in thrift-0.8), so we should use that in the Hadoop-2 profile. It will be a bit "dirty", since we need to use reflection, but I think it is worth it.

        Show
        Hari Shreedharan added a comment - Seems like in hbase-94 we can use thrift-0.8+. The TThreadSelectorServer gives far better perf (available only in thrift-0.8), so we should use that in the Hadoop-2 profile. It will be a bit "dirty", since we need to use reflection, but I think it is worth it.
        Hide
        Mike Percy added a comment -

        +1, lgtm, good to reduce the hbase version dep for compat. Unit tests pass so I'm committing this.

        Show
        Mike Percy added a comment - +1, lgtm, good to reduce the hbase version dep for compat. Unit tests pass so I'm committing this.
        Hide
        Mike Percy added a comment -

        Pushed to trunk & flume-1.4 branches. Thanks for the patch Hari!

        Show
        Mike Percy added a comment - Pushed to trunk & flume-1.4 branches. Thanks for the patch Hari!
        Mike Percy made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s v1.4.0 [ 12323372 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in flume-trunk #389 (See https://builds.apache.org/job/flume-trunk/389/)
        FLUME-1974. Thrift compatibility issue with hbase-0.92. (Revision c72a3b1a5a2548f9157fe7913611c4318ce5e64d)

        Result = UNSTABLE
        mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c72a3b1a5a2548f9157fe7913611c4318ce5e64d
        Files :

        • pom.xml
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/EventStatus.java
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEvent.java
        • flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/LogEntry.java
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/Priority.java
        • flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ResultCode.java
        • flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java
        • flume-ng-sdk/src/main/java/org/apache/flume/thrift/Status.java
        • flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftFlumeEvent.java
        • flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java
        • flume-ng-core/src/main/java/org/apache/flume/source/ThriftSource.java
        • flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java
        Show
        Hudson added a comment - Integrated in flume-trunk #389 (See https://builds.apache.org/job/flume-trunk/389/ ) FLUME-1974 . Thrift compatibility issue with hbase-0.92. (Revision c72a3b1a5a2548f9157fe7913611c4318ce5e64d) Result = UNSTABLE mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c72a3b1a5a2548f9157fe7913611c4318ce5e64d Files : pom.xml flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/EventStatus.java flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEvent.java flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/LogEntry.java flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/Priority.java flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ResultCode.java flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java flume-ng-sdk/src/main/java/org/apache/flume/thrift/Status.java flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftFlumeEvent.java flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java flume-ng-core/src/main/java/org/apache/flume/source/ThriftSource.java flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        3h 18m 1 Hari Shreedharan 07/Apr/13 06:54
        Patch Available Patch Available Resolved Resolved
        23h 11m 1 Mike Percy 08/Apr/13 06:05

          People

          • Assignee:
            Hari Shreedharan
            Reporter:
            Hari Shreedharan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development