Hive
  1. Hive
  2. HIVE-2828

make timestamp accessible in the hbase KeyValue

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: HBase Handler
    • Labels:

      Description

      Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.

      By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.

      CREATE TABLE hbase_table (key int, value string, time timestamp)
        STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
        WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
      
      1. HIVE-2828.9.patch.txt
        64 kB
        Navis
      2. HIVE-2828.8.patch.txt
        67 kB
        Navis
      3. HIVE-2828.7.patch.txt
        61 kB
        Navis
      4. HIVE-2828.6.patch.txt
        58 kB
        Navis
      5. ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.5.patch
        45 kB
        Phabricator
      6. ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.4.patch
        42 kB
        Phabricator
      7. ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.3.patch
        38 kB
        Phabricator
      8. ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.2.patch
        57 kB
        Phabricator
      9. ASF.LICENSE.NOT.GRANTED--HIVE-2828.D1989.1.patch
        22 kB
        Phabricator

        Issue Links

          Activity

          Hide
          Lefty Leverenz added a comment -

          Doc note: This should be documented in the HBase Integration wikidoc, with version information, and a release note would be nice.

          Show
          Lefty Leverenz added a comment - Doc note: This should be documented in the HBase Integration wikidoc, with version information, and a release note would be nice. HBase Integration
          Hide
          Navis added a comment -

          My bad. Some codes not in the patch is included by mistake, failing hbase_custom_key* tests. I'll merge patch for that with HIVE-8465.

          Show
          Navis added a comment - My bad. Some codes not in the patch is included by mistake, failing hbase_custom_key* tests. I'll merge patch for that with HIVE-8465 .
          Hide
          Navis added a comment -

          Fails are not related to this. Committed to trunk. Thanks Swarnim for the review!

          Show
          Navis added a comment - Fails are not related to this. Committed to trunk. Thanks Swarnim for the review!
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12674889/HIVE-2828.9.patch.txt

          ERROR: -1 due to 2 failed/errored test(s), 6559 tests executed
          Failed tests:

          org.apache.hive.beeline.TestSchemaTool.testSchemaInit
          org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1280/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1280/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1280/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12674889

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12674889/HIVE-2828.9.patch.txt ERROR: -1 due to 2 failed/errored test(s), 6559 tests executed Failed tests: org.apache.hive.beeline.TestSchemaTool.testSchemaInit org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1280/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1280/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1280/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12674889
          Hide
          Navis added a comment -

          Addressed comment. Rerunning test.

          Show
          Navis added a comment - Addressed comment. Rerunning test.
          Hide
          Swarnim Kulkarni added a comment -

          Thanks Navis. The patch looks great to me. Left a minor question on the review. +1 otherwise.

          Show
          Swarnim Kulkarni added a comment - Thanks Navis . The patch looks great to me. Left a minor question on the review. +1 otherwise.
          Hide
          Navis added a comment -
          Show
          Navis added a comment - Nick Dimiduk It would be handled in HIVE-8267 . (see https://issues.apache.org/jira/browse/HIVE-8267?focusedCommentId=14154347&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14154347 ) Swarnim Kulkarni Cannot find your comments in phabricator and the patch in it would have gone stale. Could you use RB ( https://reviews.apache.org/r/25238/)?
          Hide
          Phabricator added a comment -

          ndimiduk added a subscriber: ndimiduk.
          ndimiduk accepted this revision.
          ndimiduk added a reviewer: ndimiduk.
          ndimiduk added a comment.
          This revision is now accepted and ready to land.

          +1

          BRANCH
          DPAL-861

          INLINE COMMENTS
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java:154 This is as good a decision as any. Maybe down the road someone will want to specify which cell's timestamp maps to which column?

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          To: navis, JIRA, ndimiduk
          Cc: ndimiduk

          Show
          Phabricator added a comment - ndimiduk added a subscriber: ndimiduk. ndimiduk accepted this revision. ndimiduk added a reviewer: ndimiduk. ndimiduk added a comment. This revision is now accepted and ready to land. +1 BRANCH DPAL-861 INLINE COMMENTS hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java:154 This is as good a decision as any. Maybe down the road someone will want to specify which cell's timestamp maps to which column? REVISION DETAIL https://reviews.facebook.net/D1989 To: navis, JIRA, ndimiduk Cc: ndimiduk
          Hide
          Nick Dimiduk added a comment -

          Left a comment on phabricator. +1

          Show
          Nick Dimiduk added a comment - Left a comment on phabricator. +1
          Hide
          Swarnim Kulkarni added a comment -

          Left comments on the phabricator review.

          Show
          Swarnim Kulkarni added a comment - Left comments on the phabricator review.
          Hide
          Swarnim Kulkarni added a comment -

          Sure. I will take a look at it today.

          Show
          Swarnim Kulkarni added a comment - Sure. I will take a look at it today.
          Hide
          Muhammad Ehsan ul Haque added a comment -

          Sushanth Sowmyan, Brock Noland, Swarnim Kulkarni, Sergey
          Also, HIVE-8267 is the sequel feature request, with the ability to make timestamp of any cell available. So could you guys also look into that?

          Show
          Muhammad Ehsan ul Haque added a comment - Sushanth Sowmyan , Brock Noland , Swarnim Kulkarni , Sergey Also, HIVE-8267 is the sequel feature request, with the ability to make timestamp of any cell available. So could you guys also look into that?
          Hide
          Sergey added a comment -

          Ok,thanks!

          Show
          Sergey added a comment - Ok,thanks!
          Hide
          Brock Noland added a comment -

          Swarnim Kulkarni has worked on the hbase serde lately.

          Show
          Brock Noland added a comment - Swarnim Kulkarni has worked on the hbase serde lately.
          Hide
          Sergey added a comment -

          Can you give a github link where to look at?

          Show
          Sergey added a comment - Can you give a github link where to look at?
          Hide
          Sushanth Sowmyan added a comment -

          Sorry, I've been swamped the last couple of days, and expect to be so for the next couple of days - this weekend is going to be the earliest I can look at it. I think Brock Noland and Nick Dimiduk also potentially have some context, so tagging them if they want to look at this before I do so.

          Show
          Sushanth Sowmyan added a comment - Sorry, I've been swamped the last couple of days, and expect to be so for the next couple of days - this weekend is going to be the earliest I can look at it. I think Brock Noland and Nick Dimiduk also potentially have some context, so tagging them if they want to look at this before I do so.
          Hide
          Sushanth Sowmyan added a comment -

          Sure, I'll try to look into this tonight.

          Show
          Sushanth Sowmyan added a comment - Sure, I'll try to look into this tonight.
          Hide
          Navis added a comment -

          Sushanth Sowmyan I think there are some requests on this. Could you review this if time allows?

          Show
          Navis added a comment - Sushanth Sowmyan I think there are some requests on this. Could you review this if time allows?
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12672261/HIVE-2828.8.patch.txt

          SUCCESS: +1 6488 tests passed

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1072/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1072/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1072/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12672261

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12672261/HIVE-2828.8.patch.txt SUCCESS: +1 6488 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1072/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1072/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1072/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12672261
          Hide
          Navis added a comment -

          Rebased to trunk.

          Show
          Navis added a comment - Rebased to trunk.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12665849/HIVE-2828.7.patch.txt

          ERROR: -1 due to 1 failed/errored test(s), 6135 tests executed
          Failed tests:

          org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/598/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/598/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-598/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12665849

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12665849/HIVE-2828.7.patch.txt ERROR: -1 due to 1 failed/errored test(s), 6135 tests executed Failed tests: org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/598/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/598/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-598/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12665849
          Hide
          Navis added a comment -

          Fixed test fails

          Show
          Navis added a comment - Fixed test fails
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12665836/HIVE-2828.6.patch.txt

          ERROR: -1 due to 4 failed/errored test(s), 6134 tests executed
          Failed tests:

          org.apache.hadoop.hive.hbase.TestLazyHBaseObject.testLazyHBaseRow1
          org.apache.hadoop.hive.hbase.TestLazyHBaseObject.testLazyHBaseRow2
          org.apache.hadoop.hive.hbase.TestLazyHBaseObject.testLazyHBaseRow3
          org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/595/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/595/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-595/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 4 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12665836

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12665836/HIVE-2828.6.patch.txt ERROR: -1 due to 4 failed/errored test(s), 6134 tests executed Failed tests: org.apache.hadoop.hive.hbase.TestLazyHBaseObject.testLazyHBaseRow1 org.apache.hadoop.hive.hbase.TestLazyHBaseObject.testLazyHBaseRow2 org.apache.hadoop.hive.hbase.TestLazyHBaseObject.testLazyHBaseRow3 org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/595/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/595/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-595/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed This message is automatically generated. ATTACHMENT ID: 12665836
          Hide
          Navis added a comment -

          2.5 year rebase.

          Show
          Navis added a comment - 2.5 year rebase.
          Hide
          Muhammad Ehsan ul Haque added a comment -

          What is the status of this ticket?
          Can someone tell me if this is going to be pushed? or any reason why its not pushed?

          This is a good feature and we have come to a couple of situations where this support is good to have.

          The current patch might not be applicable to the current version Hive 0.13, but I can work on it (in-fact I have a patch available with the current trunk but slightly different implementation).

          Show
          Muhammad Ehsan ul Haque added a comment - What is the status of this ticket? Can someone tell me if this is going to be pushed? or any reason why its not pushed? This is a good feature and we have come to a couple of situations where this support is good to have. The current patch might not be applicable to the current version Hive 0.13, but I can work on it (in-fact I have a patch available with the current trunk but slightly different implementation).
          Hide
          John Shields added a comment -

          What is the right process for applying these patches? Also, on which SVN paths would it be applicable? Obviously the code has changed since these were created so I'm trying to figure out how to apply them. We currently have a version of 0.8.0 that would be great to apply these patches against. I tried against branches/0.8.0 to no avail.

          Thanks!

          John

          Show
          John Shields added a comment - What is the right process for applying these patches? Also, on which SVN paths would it be applicable? Obviously the code has changed since these were created so I'm trying to figure out how to apply them. We currently have a version of 0.8.0 that would be great to apply these patches against. I tried against branches/0.8.0 to no avail. Thanks! John
          Hide
          Navis added a comment -

          If what you meant was is it possible to 'group by timestamp', it's possible.

          Show
          Navis added a comment - If what you meant was is it possible to 'group by timestamp', it's possible.
          Hide
          Hans Uhlig added a comment -

          Would this allow aggregation across timestamps as well?

          Show
          Hans Uhlig added a comment - Would this allow aggregation across timestamps as well?
          Hide
          Phabricator added a comment -

          navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          Rebased on trunk

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/hbase_timestamp.q.out

          Show
          Phabricator added a comment - navis updated the revision " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA Rebased on trunk REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/hbase_timestamp.q.out
          Hide
          Phabricator added a comment -

          navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          Rebased on trunk

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/hbase_timestamp.q.out

          Show
          Phabricator added a comment - navis updated the revision " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA Rebased on trunk REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/hbase_timestamp.q.out
          Hide
          Phabricator added a comment -

          navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          1. Rebased on trunk
          2. Support string type for timestamp column

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/hbase_timestamp.q.out

          Show
          Phabricator added a comment - navis updated the revision " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA 1. Rebased on trunk 2. Support string type for timestamp column REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/hbase_timestamp.q.out
          Hide
          Phabricator added a comment -

          navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          Applied HIVE-2771

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java.rej
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java.rej
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java
          hbase-handler/src/test/queries/external_table_ppd.q
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/external_table_ppd.q.out
          hbase-handler/src/test/results/hbase_timestamp.q.out

          Show
          Phabricator added a comment - navis updated the revision " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA Applied HIVE-2771 REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java.rej hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java.rej hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java hbase-handler/src/test/queries/external_table_ppd.q hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/external_table_ppd.q.out hbase-handler/src/test/results/hbase_timestamp.q.out
          Hide
          Navis added a comment -

          No and no as far as I know, but might be implemented by modifying HiveHBaseTableInputFormat and HBaseStorageHandler.

          Show
          Navis added a comment - No and no as far as I know, but might be implemented by modifying HiveHBaseTableInputFormat and HBaseStorageHandler.
          Hide
          Anoop Sam John added a comment -

          Does HBase integration in Hive allows retrieval of multiple versions ( diff time stamp ) of the same cell?
          Also whether allows specifying the time range while select such that the cell value versions within this time range will be retrieved?

          Show
          Anoop Sam John added a comment - Does HBase integration in Hive allows retrieval of multiple versions ( diff time stamp ) of the same cell? Also whether allows specifying the time range while select such that the cell value versions within this time range will be retrieved?
          Hide
          Phabricator added a comment -

          navis requested code review of "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          DPAL-861 make timestamp accessible in the hbase KeyValue

          Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.

          By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.

          CREATE TABLE hbase_table (key int, value string, time timestamp)
          STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
          WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/hbase_timestamp.q.out

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/4245/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - navis requested code review of " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA DPAL-861 make timestamp accessible in the hbase KeyValue Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone. By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue. CREATE TABLE hbase_table (key int, value string, time timestamp) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp") TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/hbase_timestamp.q.out MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/4245/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.

            People

            • Assignee:
              Navis
              Reporter:
              Navis
            • Votes:
              4 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development