Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.12.0-incubating
    • Fix Version/s: 0.13.0-incubating
    • Component/s: core
    • Labels:
      None
    • Environment:
      Phoenix 4.11.0
      HBase 1.3.1

      Description

      It would be nice if tephra can be made to be compatible with HBase 1.3.1.

      I am using the apache-phoenix-4.11.0-HBase-1.3 release with HBase 1.3.1 and tephra logs the following when starting:

      Wed Jul 19 00:36:32 UTC 2017 Starting tephra service on mf82633-phoenix.mf82633
      -f: file size (blocks)             unlimited
      -t: cpu time (seconds)             unlimited
      -d: data seg size (kb)             unlimited
      -s: stack size (kb)                8192
      -c: core file size (blocks)        unlimited
      -m: resident set size (kb)         unlimited
      -l: locked memory (kb)             64
      -p: processes                      unlimited
      -n: file descriptors               1048576
      -v: address space (kb)             unlimited
      -w: locks                          unlimited
      -e: scheduling priority            0
      -r: real-time priority             0
      Command:  /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -XX:+UseConcMarkSweepGC -cp /opt/hbase/bin/../lib/*:/opt/hbase/bin/../conf/:/opt/hbase/phoenix-client/target/*:/opt/hbase/conf:/usr/lib/jvm/java-1.8-openjdk/jre/lib/tools.jar:/opt/hbase:/opt/hbase/lib/activation-1.1.jar:/opt/hbase/lib/aopalliance-1.0.jar:/opt/hbase/lib/apacheds-i18n-2.0.0-M15.jar:/opt/hbase/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hbase/lib/api-asn1-api-1.0.0-M20.jar:/opt/hbase/lib/api-util-1.0.0-M20.jar:/opt/hbase/lib/asm-3.1.jar:/opt/hbase/lib/avro-1.7.4.jar:/opt/hbase/lib/commons-beanutils-1.7.0.jar:/opt/hbase/lib/commons-beanutils-core-1.8.0.jar:/opt/hbase/lib/commons-cli-1.2.jar:/opt/hbase/lib/commons-codec-1.9.jar:/opt/hbase/lib/commons-collections-3.2.2.jar:/opt/hbase/lib/commons-compress-1.4.1.jar:/opt/hbase/lib/commons-configuration-1.6.jar:/opt/hbase/lib/commons-daemon-1.0.13.jar:/opt/hbase/lib/commons-digester-1.8.jar:/opt/hbase/lib/commons-el-1.0.jar:/opt/hbase/lib/commons-httpclient-3.1.jar:/opt/hbase/lib/commons-io-2.4.jar:/opt/hbase/lib/commons-lang-2.6.jar:/opt/hbase/lib/commons-logging-1.2.jar:/opt/hbase/lib/commons-math-2.2.jar:/opt/hbase/lib/commons-math3-3.1.1.jar:/opt/hbase/lib/commons-net-3.1.jar:/opt/hbase/lib/disruptor-3.3.0.jar:/opt/hbase/lib/findbugs-annotations-1.3.9-1.jar:/opt/hbase/lib/guice-3.0.jar:/opt/hbase/lib/guice-servlet-3.0.jar:/opt/hbase/lib/hadoop-annotations-2.5.1.jar:/opt/hbase/lib/hadoop-auth-2.5.1.jar:/opt/hbase/lib/hadoop-client-2.5.1.jar:/opt/hbase/lib/hadoop-common-2.5.1.jar:/opt/hbase/lib/hadoop-hdfs-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-app-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-common-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-core-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-jobclient-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-shuffle-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-api-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-client-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-common-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-server-common-2.5.1.jar:/opt/hbase/lib/hbase-annotations-1.3.1-tests.jar:/opt/hbase/lib/hbase-annotations-1.3.1.jar:/opt/hbase/lib/hbase-client-1.3.1.jar:/opt/hbase/lib/hbase-common-1.3.1-tests.jar:/opt/hbase/lib/hbase-common-1.3.1.jar:/opt/hbase/lib/hbase-examples-1.3.1.jar:/opt/hbase/lib/hbase-external-blockcache-1.3.1.jar:/opt/hbase/lib/hbase-hadoop-compat-1.3.1.jar:/opt/hbase/lib/hbase-hadoop2-compat-1.3.1.jar:/opt/hbase/lib/hbase-it-1.3.1-tests.jar:/opt/hbase/lib/hbase-it-1.3.1.jar:/opt/hbase/lib/hbase-prefix-tree-1.3.1.jar:/opt/hbase/lib/hbase-procedure-1.3.1.jar:/opt/hbase/lib/hbase-protocol-1.3.1.jar:/opt/hbase/lib/hbase-resource-bundle-1.3.1.jar:/opt/hbase/lib/hbase-rest-1.3.1.jar:/opt/hbase/lib/hbase-server-1.3.1-tests.jar:/opt/hbase/lib/hbase-server-1.3.1.jar:/opt/hbase/lib/hbase-shell-1.3.1.jar:/opt/hbase/lib/hbase-thrift-1.3.1.jar:/opt/hbase/lib/htrace-core-3.1.0-incubating.jar:/opt/hbase/lib/httpclient-4.2.5.jar:/opt/hbase/lib/httpcore-4.4.1.jar:/opt/hbase/lib/jackson-core-asl-1.9.13.jar:/opt/hbase/lib/jackson-jaxrs-1.9.13.jar:/opt/hbase/lib/jackson-mapper-asl-1.9.13.jar:/opt/hbase/lib/jackson-xc-1.9.13.jar:/opt/hbase/lib/jamon-runtime-2.4.1.jar:/opt/hbase/lib/jasper-compiler-5.5.23.jar:/opt/hbase/lib/jasper-runtime-5.5.23.jar:/opt/hbase/lib/java-xmlbuilder-0.4.jar:/opt/hbase/lib/javax.inject-1.jar:/opt/hbase/lib/jaxb-api-2.2.2.jar:/opt/hbase/lib/jaxb-impl-2.2.3-1.jar:/opt/hbase/lib/jcodings-1.0.8.jar:/opt/hbase/lib/jersey-client-1.9.jar:/opt/hbase/lib/jersey-core-1.9.jar:/opt/hbase/lib/jersey-guice-1.9.jar:/opt/hbase/lib/jersey-json-1.9.jar:/opt/hbase/lib/jersey-server-1.9.jar:/opt/hbase/lib/jets3t-0.9.0.jar:/opt/hbase/lib/jettison-1.3.3.jar:/opt/hbase/lib/jetty-6.1.26.jar:/opt/hbase/lib/jetty-sslengine-6.1.26.jar:/opt/hbase/lib/jetty-util-6.1.26.jar:/opt/hbase/lib/joni-2.1.2.jar:/opt/hbase/lib/jruby-complete-1.6.8.jar:/opt/hbase/lib/jsch-0.1.42.jar:/opt/hbase/lib/jsp-2.1-6.1.14.jar:/opt/hbase/lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/lib/junit-4.12.jar:/opt/hbase/lib/leveldbjni-all-1.8.jar:/opt/hbase/lib/libthrift-0.9.3.jar:/opt/hbase/lib/log4j-1.2.17.jar:/opt/hbase/lib/metrics-core-2.2.0.jar:/opt/hbase/lib/netty-all-4.0.23.Final.jar:/opt/hbase/lib/paranamer-2.3.jar:/opt/hbase/lib/phoenix-4.11.0-HBase-1.3-client.jar:/opt/hbase/lib/phoenix-4.11.0-HBase-1.3-server.jar:/opt/hbase/lib/protobuf-java-2.5.0.jar:/opt/hbase/lib/remotecontent?filepath=com%2Fgoogle%2Fguava%2Fguava%2F13.0.1%2Fguava-13.0.1.jar:/opt/hbase/lib/servlet-api-2.5-6.1.14.jar:/opt/hbase/lib/servlet-api-2.5.jar:/opt/hbase/lib/slf4j-api-1.7.7.jar:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar:/opt/hbase/lib/snappy-java-1.0.4.1.jar:/opt/hbase/lib/spymemcached-2.11.6.jar:/opt/hbase/lib/xmlenc-0.52.jar:/opt/hbase/lib/xz-1.0.jar:/opt/hbase/lib/zookeeper-3.4.6.jar:: org.apache.tephra.TransactionServiceMain
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/opt/hbase/lib/phoenix-4.11.0-HBase-1.3-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
      Exception in thread "main" com.google.inject.ProvisionException: Guice provision errors:
      
      1) Unknown HBase version: 1.3.1
      
      1 error
      	at org.apache.tephra.util.HBaseVersionSpecificFactory.get(HBaseVersionSpecificFactory.java:56)
      	at org.apache.tephra.util.ConfigurationFactory.<init>(ConfigurationFactory.java:56)
      	at org.apache.tephra.TransactionServiceMain.init(TransactionServiceMain.java:95)
      	at org.apache.tephra.TransactionServiceMain.doMain(TransactionServiceMain.java:84)
      	at org.apache.tephra.TransactionServiceMain.main(TransactionServiceMain.java:49)
      2017-07-19 00:36:32,914 INFO  [Thread-0] tephra.TransactionServiceMain: Stopping TransactionServiceMain
      

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user anew commented on the issue:

          https://github.com/apache/incubator-tephra/pull/50

          We also want to do a release soon. But we must fix at least TEPHRA-239, 242 and 244 as they are show stoppers for people who use Tephra in production. 244 is definitely the most complex to fix. Anyway this is not the place to have a release discussion... let's discuss the release on the mailing list when 244 is fixed? For now, this issue is resolved, thanks for the help.

          Show
          githubbot ASF GitHub Bot added a comment - Github user anew commented on the issue: https://github.com/apache/incubator-tephra/pull/50 We also want to do a release soon. But we must fix at least TEPHRA-239 , 242 and 244 as they are show stoppers for people who use Tephra in production. 244 is definitely the most complex to fix. Anyway this is not the place to have a release discussion... let's discuss the release on the mailing list when 244 is fixed? For now, this issue is resolved, thanks for the help.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/incubator-tephra/pull/50

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/incubator-tephra/pull/50
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user JamesRTaylor commented on the issue:

          https://github.com/apache/incubator-tephra/pull/50

          The sooner the better for a Tephra release. This is the critical issue for us as we have no Tephra support for HBase 1.3 currently.

          Show
          githubbot ASF GitHub Bot added a comment - Github user JamesRTaylor commented on the issue: https://github.com/apache/incubator-tephra/pull/50 The sooner the better for a Tephra release. This is the critical issue for us as we have no Tephra support for HBase 1.3 currently.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user anew commented on the issue:

          https://github.com/apache/incubator-tephra/pull/50

          The travis test run for Java 8 failed while it passed for Java 7. It fails in a test for HBase 1.2, which is not affected at all by this change. That indicates that the failing test case is flaky. I created a new Jira TEPHRA-253 to track the flaky test.

          Since that failure is due to a flaky test, and it passed with Java 7, and the changes here do not have any influence on the failed test, I am confident that this change did not introduce the failure. Hence I will commit this.

          Show
          githubbot ASF GitHub Bot added a comment - Github user anew commented on the issue: https://github.com/apache/incubator-tephra/pull/50 The travis test run for Java 8 failed while it passed for Java 7. It fails in a test for HBase 1.2, which is not affected at all by this change. That indicates that the failing test case is flaky. I created a new Jira TEPHRA-253 to track the flaky test. Since that failure is due to a flaky test, and it passed with Java 7, and the changes here do not have any influence on the failed test, I am confident that this change did not introduce the failure. Hence I will commit this.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user anew commented on the issue:

          https://github.com/apache/incubator-tephra/pull/50

          Thanks, @JamesRTaylor, for the verification. I am waiting for the Travis build to succeed.
          What is your anticipated date for Phoenix 4.12? We have a few critical fixes that we would like to get done before the release, so a release in about 2 weeks seems realistic.

          Show
          githubbot ASF GitHub Bot added a comment - Github user anew commented on the issue: https://github.com/apache/incubator-tephra/pull/50 Thanks, @JamesRTaylor, for the verification. I am waiting for the Travis build to succeed. What is your anticipated date for Phoenix 4.12? We have a few critical fixes that we would like to get done before the release, so a release in about 2 weeks seems realistic.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user JamesRTaylor commented on the issue:

          https://github.com/apache/incubator-tephra/pull/50

          Verified that all test pass with usage of new HBase 1.3 compat module using this patch. Would be much appreciated if a Tephra release could be done with this patch so we can support transactions in our HBase 1.3 release for 4.12.

          Show
          githubbot ASF GitHub Bot added a comment - Github user JamesRTaylor commented on the issue: https://github.com/apache/incubator-tephra/pull/50 Verified that all test pass with usage of new HBase 1.3 compat module using this patch. Would be much appreciated if a Tephra release could be done with this patch so we can support transactions in our HBase 1.3 release for 4.12.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user anew commented on the issue:

          https://github.com/apache/incubator-tephra/pull/50

          Rebased on top of Tephra-252. Travis build should now pass.

          Show
          githubbot ASF GitHub Bot added a comment - Github user anew commented on the issue: https://github.com/apache/incubator-tephra/pull/50 Rebased on top of Tephra-252. Travis build should now pass.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user JamesRTaylor commented on the issue:

          https://github.com/apache/incubator-tephra/pull/50

          Will do - I'll get back to you when finished.

          Show
          githubbot ASF GitHub Bot added a comment - Github user JamesRTaylor commented on the issue: https://github.com/apache/incubator-tephra/pull/50 Will do - I'll get back to you when finished.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user anew commented on the issue:

          https://github.com/apache/incubator-tephra/pull/50

          Thanks for the quick review, @JamesRTaylor. As an additional validation, could you run the Phoenix tests with this branch of Tephra? Thanks a lot!

          Show
          githubbot ASF GitHub Bot added a comment - Github user anew commented on the issue: https://github.com/apache/incubator-tephra/pull/50 Thanks for the quick review, @JamesRTaylor. As an additional validation, could you run the Phoenix tests with this branch of Tephra? Thanks a lot!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user JamesRTaylor commented on the issue:

          https://github.com/apache/incubator-tephra/pull/50

          +1. Thanks, @anew!

          Show
          githubbot ASF GitHub Bot added a comment - Github user JamesRTaylor commented on the issue: https://github.com/apache/incubator-tephra/pull/50 +1. Thanks, @anew!
          Hide
          anew Andreas Neumann added a comment -

          I had to add a new compat module because the Table API also added new methods. Here is the pull request.

          Please help review. Thanks -Andreas

          Show
          anew Andreas Neumann added a comment - I had to add a new compat module because the Table API also added new methods. Here is the pull request. https://github.com/apache/incubator-tephra/pull/50 Please help review. Thanks -Andreas
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user anew opened a pull request:

          https://github.com/apache/incubator-tephra/pull/50

          TEPHRA-238 Support for HBase 1.3.x

          • Adds a new compat module using 1.3.1 as the HBase version. I could not reuse the compat-1.1-base because HTable added four new methods in 1.3 (get/setOperation/RpcTimeout)
          • All the classes in the new src/ tree are the same as in the compat-1.1-base, except:
            1. implemented the four new methods in TransactionAwareHTable
            2. change to a TransactionProcessorTest.createRegion(), because WALFactory.getWAL() now takes an extra argument for the namespace.
            3. Changing 1.1 to 1.3 in all comments etc.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/anew/incubator-tephra tephra-238

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/incubator-tephra/pull/50.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #50


          commit 75fb6df59ad6a040d46d8fccea0be51454b56d56
          Author: anew <anew@apache.org>
          Date: 2017-09-05T20:44:08Z

          TEPHRA-238 Support for HBase 1.3.x


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user anew opened a pull request: https://github.com/apache/incubator-tephra/pull/50 TEPHRA-238 Support for HBase 1.3.x Adds a new compat module using 1.3.1 as the HBase version. I could not reuse the compat-1.1-base because HTable added four new methods in 1.3 (get/setOperation/RpcTimeout) All the classes in the new src/ tree are the same as in the compat-1.1-base, except: 1. implemented the four new methods in TransactionAwareHTable 2. change to a TransactionProcessorTest.createRegion(), because WALFactory.getWAL() now takes an extra argument for the namespace. 3. Changing 1.1 to 1.3 in all comments etc. You can merge this pull request into a Git repository by running: $ git pull https://github.com/anew/incubator-tephra tephra-238 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-tephra/pull/50.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #50 commit 75fb6df59ad6a040d46d8fccea0be51454b56d56 Author: anew <anew@apache.org> Date: 2017-09-05T20:44:08Z TEPHRA-238 Support for HBase 1.3.x
          Hide
          anew Andreas Neumann added a comment -

          We could add reflection to the test class but the idea of a compat module is to avoid that... I could see that it is not a big deal for tests only. Let me see what else I find.

          Show
          anew Andreas Neumann added a comment - We could add reflection to the test class but the idea of a compat module is to avoid that... I could see that it is not a big deal for tests only. Let me see what else I find.
          Hide
          jamestaylor James Taylor added a comment -

          Looks like perhaps a test-only issue. Do you still need a new compat module even for that?

          Show
          jamestaylor James Taylor added a comment - Looks like perhaps a test-only issue. Do you still need a new compat module even for that?
          Hide
          anew Andreas Neumann added a comment -

          Hi James Taylor,

          I tried running the tests against HBase 1.3.1 and it fails to compile with:

          [ERROR] /Users/andreas/anew-tephra/tephra-hbase-compat-1.1-base/src/test/java/org/apache/tephra/hbase/coprocessor/TransactionProcessorTest.java:[572,26] method getWAL in class org.apache.hadoop.hbase.wal.WALFactory cannot be applied to given types;
          

          It looks like that method was changed between 1.2 and 1.3, to take an additional argument (the namespace). We probably have to create a new compatibility module, or do you see a better option?

          -Andreas

          Show
          anew Andreas Neumann added a comment - Hi James Taylor , I tried running the tests against HBase 1.3.1 and it fails to compile with: [ERROR] /Users/andreas/anew-tephra/tephra-hbase-compat-1.1-base/src/test/java/org/apache/tephra/hbase/coprocessor/TransactionProcessorTest.java:[572,26] method getWAL in class org.apache.hadoop.hbase.wal.WALFactory cannot be applied to given types; It looks like that method was changed between 1.2 and 1.3, to take an additional argument (the namespace). We probably have to create a new compatibility module, or do you see a better option? -Andreas
          Hide
          jamestaylor James Taylor added a comment -
          Show
          jamestaylor James Taylor added a comment - Ping Andreas Neumann .
          Hide
          jamestaylor James Taylor added a comment -

          Any ETA on this yet, Poorna Chandra?

          Show
          jamestaylor James Taylor added a comment - Any ETA on this yet, Poorna Chandra ?
          Hide
          poornachandra Poorna Chandra added a comment -

          James Taylor I think HBase 1.2 compatibility module should work with HBase 1.3.1. We will test it and make it available as part of 0.13.0-incubating Tephra release.

          Show
          poornachandra Poorna Chandra added a comment - James Taylor I think HBase 1.2 compatibility module should work with HBase 1.3.1. We will test it and make it available as part of 0.13.0-incubating Tephra release.
          Hide
          jamestaylor James Taylor added a comment -
          Show
          jamestaylor James Taylor added a comment - Poorna Chandra or Gokul Gunasekaran ?
          Hide
          jamestaylor James Taylor added a comment -

          Will the HBase 1.2 compatibility module work fine with HBase 1.3.1?

          Show
          jamestaylor James Taylor added a comment - Will the HBase 1.2 compatibility module work fine with HBase 1.3.1?

            People

            • Assignee:
              poornachandra Poorna Chandra
              Reporter:
              francischuang Francis Chuang
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development