Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.7.0
    • Fix Version/s: 1.7.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      The current version of AsyncHBase ( http://opentsdb.github.io/asynchbase/ ) is outdated.

      Reason for the upgrade would be many bug fixes and some new public APIs. (See https://github.com/OpenTSDB/asynchbase/blob/master/NEWS )

      1. FLUME-2923.patch
        0.4 kB
        Bessenyei Balázs Donát
      2. FLUME-2923-2.patch
        0.8 kB
        Bessenyei Balázs Donát
      3. FLUME-2923-3.patch
        0.7 kB
        Bessenyei Balázs Donát

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Flume-trunk-hbase-1 #163 (See https://builds.apache.org/job/Flume-trunk-hbase-1/163/ ) FLUME-2923 . Bump asynchbase version to 1.7.0 (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=1b9e589152f0093b6ef7cea580c00ff52944d2cf ) pom.xml
          Hide
          mpercy Mike Percy added a comment -

          Committed to trunk. Thanks for the patch, Bessenyei Balázs Donát!

          Show
          mpercy Mike Percy added a comment - Committed to trunk. Thanks for the patch, Bessenyei Balázs Donát !
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1b9e589152f0093b6ef7cea580c00ff52944d2cf in flume's branch refs/heads/trunk from Mike Percy
          [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=1b9e589 ]

          FLUME-2923. Bump asynchbase version to 1.7.0

          Newer versions of HBase and the asynchbase library support error types
          such as RegionMovedException. Also, asynchbase has incorporated several
          fixes since version 1.5.0.

          See https://github.com/OpenTSDB/asynchbase/blob/v1.7.0/NEWS

          (Bessenyei Balázs Donát via Mike Percy)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 1b9e589152f0093b6ef7cea580c00ff52944d2cf in flume's branch refs/heads/trunk from Mike Percy [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=1b9e589 ] FLUME-2923 . Bump asynchbase version to 1.7.0 Newer versions of HBase and the asynchbase library support error types such as RegionMovedException. Also, asynchbase has incorporated several fixes since version 1.5.0. See https://github.com/OpenTSDB/asynchbase/blob/v1.7.0/NEWS (Bessenyei Balázs Donát via Mike Percy)
          Hide
          mpercy Mike Percy added a comment -

          +1 looks good. The Kafka source unit tests are not passing for me but I think they're broken or flaky in general. I'm going to commit this.

          Show
          mpercy Mike Percy added a comment - +1 looks good. The Kafka source unit tests are not passing for me but I think they're broken or flaky in general. I'm going to commit this.
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          Mike Percy: I have updated the netty version in the parent pom.xml. Please, see the latest patch attached.
          The tests you have provided in one of the previous comment seem to be working fine, as well as the other Flume unit tests.

          Show
          bessbd Bessenyei Balázs Donát added a comment - Mike Percy : I have updated the netty version in the parent pom.xml. Please, see the latest patch attached. The tests you have provided in one of the previous comment seem to be working fine, as well as the other Flume unit tests.
          Hide
          mpercy Mike Percy added a comment -

          @bessbd Thanks for updating the patch. Please update the netty version in the parent pom.xml file in the dependencyMananagement section instead of the child pom, since netty is used by multiple components: https://github.com/apache/flume/blob/trunk/pom.xml#L1311

          You will also need to run all of the Flume unit tests because of that change.

          Show
          mpercy Mike Percy added a comment - @bessbd Thanks for updating the patch. Please update the netty version in the parent pom.xml file in the dependencyMananagement section instead of the child pom, since netty is used by multiple components: https://github.com/apache/flume/blob/trunk/pom.xml#L1311 You will also need to run all of the Flume unit tests because of that change.
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          Mike Percy: Please find the new patch attached. The tests for flume-ng-sinks/flume-ng-hbase-sink seem to be running fine again now.

          Show
          bessbd Bessenyei Balázs Donát added a comment - Mike Percy : Please find the new patch attached. The tests for flume-ng-sinks/flume-ng-hbase-sink seem to be running fine again now.
          Hide
          mpercy Mike Percy added a comment -

          Bessenyei Balázs Donát: Did you run the asynchbase unit tests after applying this patch?

          I get a bunch of failures. Are you seeing the same thing?

          mpercy@mpercy-ThinkPad-T530:~/src/flume$ git log --oneline -n5
          ee4999b FLUME-2910. AsyncHBaseSink: Failure callbacks should log the exception that caused them
          f46bee0 FLUME-2920: Kafka Channel Should Not Commit Offsets When Stopping
          a619cc1 FLUME-2915: The kafka channel using new APIs will be stuck when the sink is avro sink
          f4f15be FLUME-2913. Don't strip SLF4J from imported classpaths
          37ddd1b FLUME-2909. Upgrade RAT to 0.11
          
          mpercy@mpercy-ThinkPad-T530:~/src/flume$ git diff
          diff --git a/pom.xml b/pom.xml
          index 0929095..110c216 100644
          --- a/pom.xml
          +++ b/pom.xml
          @@ -1292,7 +1292,7 @@ limitations under the License.
                 <dependency>
                   <groupId>org.hbase</groupId>
                   <artifactId>asynchbase</artifactId>
          -        <version>1.5.0</version>
          +        <version>1.7.0</version>
                   <exclusions>
                     <exclusion>
                       <groupId>org.slf4j</groupId>
          
          mpercy@mpercy-ThinkPad-T530:~/src/flume$ mvn clean install -DskipTests -Drat.skip=true; mvn -pl flume-ng-sinks/flume-ng-hbase-sink -Drat.skip=true test
          ...
          Results :
          
          Tests in error:
            testMultipleBatches(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool
            testMultipleBatchesBatchIncrementsWithCoalescing(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool
            testMultipleBatchesBatchIncrementsNoCoalescing(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool
            testWithoutConfigurationObject(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool
            testMissingTable(org.apache.flume.sink.hbase.TestAsyncHBaseSink): Unexpected exception, expected<org.apache.flume.FlumeException> but was<java.lang.NoClassDefFoundError>
            testFDLeakOnShutdown(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool
            testOneEventWithDefaults(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool
            testOneEvent(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool
            testThreeEvents(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool
            testTimeOut(org.apache.flume.sink.hbase.TestAsyncHBaseSink): Unexpected exception, expected<org.apache.flume.EventDeliveryException> but was<java.lang.NoClassDefFoundError>
          
          Tests run: 31, Failures: 0, Errors: 10, Skipped: 2
          

          Note that the above command causes all tests to pass for me on trunk without applying your patch.

          Mike

          Show
          mpercy Mike Percy added a comment - Bessenyei Balázs Donát : Did you run the asynchbase unit tests after applying this patch? I get a bunch of failures. Are you seeing the same thing? mpercy@mpercy-ThinkPad-T530:~/src/flume$ git log --oneline -n5 ee4999b FLUME-2910. AsyncHBaseSink: Failure callbacks should log the exception that caused them f46bee0 FLUME-2920: Kafka Channel Should Not Commit Offsets When Stopping a619cc1 FLUME-2915: The kafka channel using new APIs will be stuck when the sink is avro sink f4f15be FLUME-2913. Don't strip SLF4J from imported classpaths 37ddd1b FLUME-2909. Upgrade RAT to 0.11 mpercy@mpercy-ThinkPad-T530:~/src/flume$ git diff diff --git a/pom.xml b/pom.xml index 0929095..110c216 100644 --- a/pom.xml +++ b/pom.xml @@ -1292,7 +1292,7 @@ limitations under the License. <dependency> <groupId>org.hbase</groupId> <artifactId>asynchbase</artifactId> - <version>1.5.0</version> + <version>1.7.0</version> <exclusions> <exclusion> <groupId>org.slf4j</groupId> mpercy@mpercy-ThinkPad-T530:~/src/flume$ mvn clean install -DskipTests -Drat.skip= true ; mvn -pl flume-ng-sinks/flume-ng-hbase-sink -Drat.skip= true test ... Results : Tests in error: testMultipleBatches(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool testMultipleBatchesBatchIncrementsWithCoalescing(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool testMultipleBatchesBatchIncrementsNoCoalescing(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool testWithoutConfigurationObject(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool testMissingTable(org.apache.flume.sink.hbase.TestAsyncHBaseSink): Unexpected exception, expected<org.apache.flume.FlumeException> but was<java.lang.NoClassDefFoundError> testFDLeakOnShutdown(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool testOneEventWithDefaults(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool testOneEvent(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool testThreeEvents(org.apache.flume.sink.hbase.TestAsyncHBaseSink): org/jboss/netty/channel/socket/nio/BossPool testTimeOut(org.apache.flume.sink.hbase.TestAsyncHBaseSink): Unexpected exception, expected<org.apache.flume.EventDeliveryException> but was<java.lang.NoClassDefFoundError> Tests run: 31, Failures: 0, Errors: 10, Skipped: 2 Note that the above command causes all tests to pass for me on trunk without applying your patch. Mike
          Hide
          mpercy Mike Percy added a comment -

          I believe this is important for supporting newer versions of HBase. Let's pull this in if the unit tests pass.

          Show
          mpercy Mike Percy added a comment - I believe this is important for supporting newer versions of HBase. Let's pull this in if the unit tests pass.
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          This patch changes version of AsyncHBase from 1.5.0 to 1.7.0.

          According to the documentation ( http://opentsdb.github.io/asynchbase/docs/build/html/installation.html#upgrading ), backwards-compatibility is guaranteed, only needs changing the version number.

          After applying the patch the build remains successful.

          Show
          bessbd Bessenyei Balázs Donát added a comment - This patch changes version of AsyncHBase from 1.5.0 to 1.7.0. According to the documentation ( http://opentsdb.github.io/asynchbase/docs/build/html/installation.html#upgrading ), backwards-compatibility is guaranteed, only needs changing the version number. After applying the patch the build remains successful.

            People

            • Assignee:
              bessbd Bessenyei Balázs Donát
              Reporter:
              bessbd Bessenyei Balázs Donát
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development