Hive
  1. Hive
  2. HIVE-2828

make timestamp accessible in the hbase KeyValue

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Trivial Trivial
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: HBase Handler
    • Labels:
      None

      Description

      Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.

      By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.

      CREATE TABLE hbase_table (key int, value string, time timestamp)
        STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
        WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
      

        Issue Links

          Activity

          Hide
          John Shields added a comment -

          What is the right process for applying these patches? Also, on which SVN paths would it be applicable? Obviously the code has changed since these were created so I'm trying to figure out how to apply them. We currently have a version of 0.8.0 that would be great to apply these patches against. I tried against branches/0.8.0 to no avail.

          Thanks!

          John

          Show
          John Shields added a comment - What is the right process for applying these patches? Also, on which SVN paths would it be applicable? Obviously the code has changed since these were created so I'm trying to figure out how to apply them. We currently have a version of 0.8.0 that would be great to apply these patches against. I tried against branches/0.8.0 to no avail. Thanks! John
          Hide
          Navis added a comment -

          If what you meant was is it possible to 'group by timestamp', it's possible.

          Show
          Navis added a comment - If what you meant was is it possible to 'group by timestamp', it's possible.
          Hide
          Hans Uhlig added a comment -

          Would this allow aggregation across timestamps as well?

          Show
          Hans Uhlig added a comment - Would this allow aggregation across timestamps as well?
          Hide
          Phabricator added a comment -

          navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          Rebased on trunk

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/hbase_timestamp.q.out

          Show
          Phabricator added a comment - navis updated the revision " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA Rebased on trunk REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/hbase_timestamp.q.out
          Hide
          Phabricator added a comment -

          navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          Rebased on trunk

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/hbase_timestamp.q.out

          Show
          Phabricator added a comment - navis updated the revision " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA Rebased on trunk REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/hbase_timestamp.q.out
          Hide
          Phabricator added a comment -

          navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          1. Rebased on trunk
          2. Support string type for timestamp column

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/hbase_timestamp.q.out

          Show
          Phabricator added a comment - navis updated the revision " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA 1. Rebased on trunk 2. Support string type for timestamp column REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/hbase_timestamp.q.out
          Hide
          Phabricator added a comment -

          navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          Applied HIVE-2771

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java.rej
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java.rej
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java
          hbase-handler/src/test/queries/external_table_ppd.q
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/external_table_ppd.q.out
          hbase-handler/src/test/results/hbase_timestamp.q.out

          Show
          Phabricator added a comment - navis updated the revision " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA Applied HIVE-2771 REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java.rej hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java.rej hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java hbase-handler/src/test/queries/external_table_ppd.q hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/external_table_ppd.q.out hbase-handler/src/test/results/hbase_timestamp.q.out
          Hide
          Navis added a comment -

          No and no as far as I know, but might be implemented by modifying HiveHBaseTableInputFormat and HBaseStorageHandler.

          Show
          Navis added a comment - No and no as far as I know, but might be implemented by modifying HiveHBaseTableInputFormat and HBaseStorageHandler.
          Hide
          Anoop Sam John added a comment -

          Does HBase integration in Hive allows retrieval of multiple versions ( diff time stamp ) of the same cell?
          Also whether allows specifying the time range while select such that the cell value versions within this time range will be retrieved?

          Show
          Anoop Sam John added a comment - Does HBase integration in Hive allows retrieval of multiple versions ( diff time stamp ) of the same cell? Also whether allows specifying the time range while select such that the cell value versions within this time range will be retrieved?
          Hide
          Phabricator added a comment -

          navis requested code review of "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
          Reviewers: JIRA

          DPAL-861 make timestamp accessible in the hbase KeyValue

          Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.

          By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.

          CREATE TABLE hbase_table (key int, value string, time timestamp)
          STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
          WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")

          TEST PLAN
          EMPTY

          REVISION DETAIL
          https://reviews.facebook.net/D1989

          AFFECTED FILES
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
          hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
          hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java
          hbase-handler/src/test/queries/hbase_timestamp.q
          hbase-handler/src/test/results/hbase_timestamp.q.out

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/4245/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - navis requested code review of " HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue". Reviewers: JIRA DPAL-861 make timestamp accessible in the hbase KeyValue Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone. By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue. CREATE TABLE hbase_table (key int, value string, time timestamp) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp") TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D1989 AFFECTED FILES hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java hbase-handler/src/test/queries/hbase_timestamp.q hbase-handler/src/test/results/hbase_timestamp.q.out MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/4245/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.

            People

            • Assignee:
              Navis
              Reporter:
              Navis
            • Votes:
              3 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:

                Development