Hive
  1. Hive
  2. HIVE-1228

allow HBase key column to be anywhere in Hive table

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.0
    • Component/s: HBase Handler
    • Labels:
      None

      Description

      Current column mapping support requires key column to be first in Hive table (and implicit rather than explicit in column mapping). We should allow it to be specified explicitly at any position within column mapping, e.g. with syntax :key to distinguish it from normal columns/families.

      Together with this, allow :timestamp column to be mapped. For writes, this would allow timestamp to be explicitly provided by query (although we may want to prevent this in some cases; see for example HIVE-1223 and HIVE-1225). For reads, this would allow timestamp to be fetched, and filters on timestamp could be used to find particular row versions. (If no filter is supplied, should we return all row versions or only the default latest version? The semantics need to be well-defined here.)

        Issue Links

          Activity

          John Sichi created issue -
          John Sichi made changes -
          Field Original Value New Value
          Assignee John Sichi [ jvs ]
          Hide
          John Sichi added a comment -

          Using contrib as component for now, but we should create a separate HBase Handler component and move there.

          Show
          John Sichi added a comment - Using contrib as component for now, but we should create a separate HBase Handler component and move there.
          John Sichi made changes -
          Component/s Contrib [ 12313001 ]
          John Sichi made changes -
          Link This issue relates to HIVE-705 [ HIVE-705 ]
          John Sichi made changes -
          Link This issue is blocked by HIVE-1226 [ HIVE-1226 ]
          Hide
          John Sichi added a comment -

          Filtering enhancement applies to key as well.

          Show
          John Sichi added a comment - Filtering enhancement applies to key as well.
          John Sichi made changes -
          Link This issue is related to HIVE-1232 [ HIVE-1232 ]
          John Sichi made changes -
          Component/s HBase Handler [ 12313461 ]
          Component/s Contrib [ 12313001 ]
          Hide
          John Sichi added a comment -

          At the HBase meetup, Ryan also suggested allowing a compound key mapping.

          Show
          John Sichi added a comment - At the HBase meetup, Ryan also suggested allowing a compound key mapping.
          John Sichi made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          John Sichi added a comment -

          Hmmm, scratch the :timestamp thing for now, since timestamps are cell-level rather than row-level. I guess we could do :timestamp:column-name to map them individually, but that deserves its own entry in JIRA.

          Show
          John Sichi added a comment - Hmmm, scratch the :timestamp thing for now, since timestamps are cell-level rather than row-level. I guess we could do :timestamp:column-name to map them individually, but that deserves its own entry in JIRA.
          John Sichi made changes -
          Summary allow HBase key column to be anywhere in Hive table, and expose timestamp column for read and write allow HBase key column to be anywhere in Hive table
          Hide
          John Sichi added a comment -

          This also fixes a bug with inserting MAP values containing multiple entries.

          Show
          John Sichi added a comment - This also fixes a bug with inserting MAP values containing multiple entries.
          John Sichi made changes -
          Attachment HIVE-1228.1.patch [ 12444354 ]
          John Sichi made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Hide
          John Sichi added a comment -

          No compound key support (I'll open a separate issue for that).

          Show
          John Sichi added a comment - No compound key support (I'll open a separate issue for that).
          John Sichi made changes -
          Link This issue is blocked by HIVE-1226 [ HIVE-1226 ]
          Hide
          Ning Zhang added a comment -

          +1. had an offline review with John and everything looks fine. John is going to file 2 follow up JIRAs for negative test cases and generating splits based on selected columns.

          Tests also passed just now.

          Show
          Ning Zhang added a comment - +1. had an offline review with John and everything looks fine. John is going to file 2 follow up JIRAs for negative test cases and generating splits based on selected columns. Tests also passed just now.
          Hide
          Ning Zhang added a comment -

          Committed. Thanks John!

          Show
          Ning Zhang added a comment - Committed. Thanks John!
          Ning Zhang made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Carl Steinbach made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          56d 2h 56m 1 John Sichi 06/May/10 00:22
          In Progress In Progress Patch Available Patch Available
          6d 23h 30m 1 John Sichi 12/May/10 23:52
          Patch Available Patch Available Resolved Resolved
          22d 15m 1 Ning Zhang 04/Jun/10 00:08
          Resolved Resolved Closed Closed
          561d 54m 1 Carl Steinbach 17/Dec/11 00:03

            People

            • Assignee:
              John Sichi
              Reporter:
              John Sichi
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development