Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0, 0.7.1, 0.8.0, 0.8.1
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      Both softwares have moved forward with significant improvements. Lets bump compile time dependency to keep up

      1. ASF.LICENSE.NOT.GRANTED--HIVE-2748.D1431.1.patch
        2 kB
        Phabricator
      2. HIVE-2748_v4.patch
        8 kB
        Enis Soztutar
      3. HIVE-2748_v5.patch
        9 kB
        Enis Soztutar
      4. HIVE-2748_v6.patch
        9 kB
        Enis Soztutar
      5. HIVE-2748_v7.patch
        9 kB
        Enis Soztutar
      6. HIVE-2748_v8.patch
        11 kB
        Enis Soztutar
      7. HIVE-2748.3.patch
        2 kB
        Alan Gates
      8. HIVE-2748.3-1.patch
        2 kB
        Alan Gates
      9. HIVE-2748.D1431.2.patch
        2 kB
        Giridharan Kesavan

        Issue Links

          Activity

          Hide
          Phabricator added a comment -

          ashutoshc requested code review of "HIVE-2748 [jira] Upgrade Hbase and ZK dependcies".
          Reviewers: JIRA

          https://issues.apache.org/jira/browse/HIVE-2748

          Update HBase to 0.92 and ZK to 3.4.2 no code change except for minor changes in test code.

          Both softwares have moved forward with significant improvements. Lets bump compile time dependency to keep up

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D1431

          AFFECTED FILES
          hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestSetup.java
          ivy/libraries.properties

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/2967/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - ashutoshc requested code review of " HIVE-2748 [jira] Upgrade Hbase and ZK dependcies". Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2748 Update HBase to 0.92 and ZK to 3.4.2 no code change except for minor changes in test code. Both softwares have moved forward with significant improvements. Lets bump compile time dependency to keep up TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1431 AFFECTED FILES hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestSetup.java ivy/libraries.properties MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/2967/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Phabricator added a comment -

          cwsteinbach has accepted the revision "HIVE-2748 [jira] Upgrade Hbase and ZK dependcies".

          +1

          @Ashutosh: Can you test and commit?

          REVISION DETAIL
          https://reviews.facebook.net/D1431

          Show
          Phabricator added a comment - cwsteinbach has accepted the revision " HIVE-2748 [jira] Upgrade Hbase and ZK dependcies". +1 @Ashutosh: Can you test and commit? REVISION DETAIL https://reviews.facebook.net/D1431
          Hide
          Phabricator added a comment -

          cwsteinbach has requested changes to the revision "HIVE-2748 [jira] Upgrade Hbase and ZK dependcies".

          INLINE COMMENTS
          ivy/libraries.properties:41 0.92.0 has been released. Let's use that instead of the SNAPSHOT.

          REVISION DETAIL
          https://reviews.facebook.net/D1431

          Show
          Phabricator added a comment - cwsteinbach has requested changes to the revision " HIVE-2748 [jira] Upgrade Hbase and ZK dependcies". INLINE COMMENTS ivy/libraries.properties:41 0.92.0 has been released. Let's use that instead of the SNAPSHOT. REVISION DETAIL https://reviews.facebook.net/D1431
          Hide
          Phabricator added a comment -

          ashutoshc has commented on the revision "HIVE-2748 [jira] Upgrade Hbase and ZK dependcies".

          INLINE COMMENTS
          ivy/libraries.properties:41 Yeah, thats what I intend to do. 0.92 is not available on mvn repos yet. As soon as its there, I will update the patch for it and then test.

          REVISION DETAIL
          https://reviews.facebook.net/D1431

          Show
          Phabricator added a comment - ashutoshc has commented on the revision " HIVE-2748 [jira] Upgrade Hbase and ZK dependcies". INLINE COMMENTS ivy/libraries.properties:41 Yeah, thats what I intend to do. 0.92 is not available on mvn repos yet. As soon as its there, I will update the patch for it and then test. REVISION DETAIL https://reviews.facebook.net/D1431
          Hide
          Phabricator added a comment -

          stack has commented on the revision "HIVE-2748 [jira] Upgrade Hbase and ZK dependcies".

          Looks good to me (0.92 does master name differently).

          REVISION DETAIL
          https://reviews.facebook.net/D1431

          Show
          Phabricator added a comment - stack has commented on the revision " HIVE-2748 [jira] Upgrade Hbase and ZK dependcies". Looks good to me (0.92 does master name differently). REVISION DETAIL https://reviews.facebook.net/D1431
          Hide
          Giridharan Kesavan added a comment -

          updating the patch so that it can apply cleanly on trunk

          Show
          Giridharan Kesavan added a comment - updating the patch so that it can apply cleanly on trunk
          Hide
          Alan Gates added a comment -

          A version of the patch that uses the released 0.92.0 artifacts instead of SNAPSHOT artifacts.

          Show
          Alan Gates added a comment - A version of the patch that uses the released 0.92.0 artifacts instead of SNAPSHOT artifacts.
          Hide
          Enis Soztutar added a comment -

          Attaching a patch with ZK updated to 3.4.3, and HBase 0.92.0 as in Alan's patch. Also adds a TokenStore.close() method, so that ZKTokenStore can properly close the client connection.

          Running the tests.

          Show
          Enis Soztutar added a comment - Attaching a patch with ZK updated to 3.4.3, and HBase 0.92.0 as in Alan's patch. Also adds a TokenStore.close() method, so that ZKTokenStore can properly close the client connection. Running the tests.
          Hide
          Ashutosh Chauhan added a comment -

          Few comments:

          • +hbase-test.version=0.92.0
            Remove this. This variable is not used anywhere later. Test version to download is same as hbase.version
          • Instead of
            hbaseConf.set("hbase.master", hbaseCluster.getMaster().toString());
            Do:
            hbaseConf.set("hbase.master", hbaseCluster.getMaster().getServerName().getHostAndPort());
          • There might be one more runtime dependency to run HBase test which you should be able to catch while running tests.
          Show
          Ashutosh Chauhan added a comment - Few comments: +hbase-test.version=0.92.0 Remove this. This variable is not used anywhere later. Test version to download is same as hbase.version Instead of hbaseConf.set("hbase.master", hbaseCluster.getMaster().toString()); Do: hbaseConf.set("hbase.master", hbaseCluster.getMaster().getServerName().getHostAndPort()); There might be one more runtime dependency to run HBase test which you should be able to catch while running tests.
          Hide
          Enis Soztutar added a comment -

          Updated the patch with review comments.

          • update guava's version to r09.
          • add hbase client's dependencies.
          • HBase related tests runs fine.
          Show
          Enis Soztutar added a comment - Updated the patch with review comments. update guava's version to r09. add hbase client's dependencies. HBase related tests runs fine.
          Hide
          Ashutosh Chauhan added a comment -

          Doesn't apply cleanly, patch needs a rebase.

          Show
          Ashutosh Chauhan added a comment - Doesn't apply cleanly, patch needs a rebase.
          Hide
          Enis Soztutar added a comment -

          Attaching rebased patch.

          Show
          Enis Soztutar added a comment - Attaching rebased patch.
          Hide
          Ashutosh Chauhan added a comment -

          Unable to compile tests. Probably, ivysettings.xml is missing m2:classifier bit.

          Show
          Ashutosh Chauhan added a comment - Unable to compile tests. Probably, ivysettings.xml is missing m2:classifier bit.
          Hide
          Enis Soztutar added a comment -

          Uploading another patch for fixing the maven pattern to add the classifier.

          Show
          Enis Soztutar added a comment - Uploading another patch for fixing the maven pattern to add the classifier.
          Hide
          Ashutosh Chauhan added a comment -

          TestHBaseSerDe fails with latest patch. Perhaps, jackson libs need to be upped to 1.7.1 as well.

          Show
          Ashutosh Chauhan added a comment - TestHBaseSerDe fails with latest patch. Perhaps, jackson libs need to be upped to 1.7.1 as well.
          Hide
          Enis Soztutar added a comment -

          Fixed the Jackson issue caused by old jackson version from Hadoop not running with HBase's updated Jackson version. See HADOOP-7470.

          Show
          Enis Soztutar added a comment - Fixed the Jackson issue caused by old jackson version from Hadoop not running with HBase's updated Jackson version. See HADOOP-7470 .
          Hide
          Ashutosh Chauhan added a comment -

          Patch looks good. Running tests.

          Show
          Ashutosh Chauhan added a comment - Patch looks good. Running tests.
          Hide
          Ashutosh Chauhan added a comment -

          Patch committed to trunk. Thanks, Enis for your persistence on this one.

          Show
          Ashutosh Chauhan added a comment - Patch committed to trunk. Thanks, Enis for your persistence on this one.
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1310 (See https://builds.apache.org/job/Hive-trunk-h0.21/1310/)
          HIVE-2748: Upgrade Hbase and ZK dependcies (enis via hashutosh) (Revision 1300818)

          Result = SUCCESS
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1300818
          Files :

          • /hive/trunk/hbase-handler/ivy.xml
          • /hive/trunk/hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestSetup.java
          • /hive/trunk/ivy/ivysettings.xml
          • /hive/trunk/ivy/libraries.properties
          • /hive/trunk/shims/ivy.xml
          • /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java
          • /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java
          • /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
          • /hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1310 (See https://builds.apache.org/job/Hive-trunk-h0.21/1310/ ) HIVE-2748 : Upgrade Hbase and ZK dependcies (enis via hashutosh) (Revision 1300818) Result = SUCCESS hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1300818 Files : /hive/trunk/hbase-handler/ivy.xml /hive/trunk/hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestSetup.java /hive/trunk/ivy/ivysettings.xml /hive/trunk/ivy/libraries.properties /hive/trunk/shims/ivy.xml /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java /hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java
          Hide
          Alan Gates added a comment -

          Version of patch 3 that uses 92.1-SNAPSHOT instead to deal with maven download issues.

          Show
          Alan Gates added a comment - Version of patch 3 that uses 92.1-SNAPSHOT instead to deal with maven download issues.
          Hide
          Ashutosh Chauhan added a comment -

          This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

          Show
          Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
          HIVE-2748: Upgrade Hbase and ZK dependcies (enis via hashutosh) (Revision 1300818)

          Result = ABORTED
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1300818
          Files :

          • /hive/trunk/hbase-handler/ivy.xml
          • /hive/trunk/hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestSetup.java
          • /hive/trunk/ivy/ivysettings.xml
          • /hive/trunk/ivy/libraries.properties
          • /hive/trunk/shims/ivy.xml
          • /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java
          • /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java
          • /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
          • /hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2748 : Upgrade Hbase and ZK dependcies (enis via hashutosh) (Revision 1300818) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1300818 Files : /hive/trunk/hbase-handler/ivy.xml /hive/trunk/hbase-handler/src/test/org/apache/hadoop/hive/hbase/HBaseTestSetup.java /hive/trunk/ivy/ivysettings.xml /hive/trunk/ivy/libraries.properties /hive/trunk/shims/ivy.xml /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java /hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java

            People

            • Assignee:
              Enis Soztutar
              Reporter:
              Ashutosh Chauhan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development