Hive
  1. Hive
  2. HIVE-1228

allow HBase key column to be anywhere in Hive table

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.0
    • Component/s: HBase Handler
    • Labels:
      None

      Description

      Current column mapping support requires key column to be first in Hive table (and implicit rather than explicit in column mapping). We should allow it to be specified explicitly at any position within column mapping, e.g. with syntax :key to distinguish it from normal columns/families.

      Together with this, allow :timestamp column to be mapped. For writes, this would allow timestamp to be explicitly provided by query (although we may want to prevent this in some cases; see for example HIVE-1223 and HIVE-1225). For reads, this would allow timestamp to be fetched, and filters on timestamp could be used to find particular row versions. (If no filter is supplied, should we return all row versions or only the default latest version? The semantics need to be well-defined here.)

        Issue Links

          Activity

          Hide
          John Sichi added a comment -

          Using contrib as component for now, but we should create a separate HBase Handler component and move there.

          Show
          John Sichi added a comment - Using contrib as component for now, but we should create a separate HBase Handler component and move there.
          Hide
          John Sichi added a comment -

          Filtering enhancement applies to key as well.

          Show
          John Sichi added a comment - Filtering enhancement applies to key as well.
          Hide
          John Sichi added a comment -

          At the HBase meetup, Ryan also suggested allowing a compound key mapping.

          Show
          John Sichi added a comment - At the HBase meetup, Ryan also suggested allowing a compound key mapping.
          Hide
          John Sichi added a comment -

          Hmmm, scratch the :timestamp thing for now, since timestamps are cell-level rather than row-level. I guess we could do :timestamp:column-name to map them individually, but that deserves its own entry in JIRA.

          Show
          John Sichi added a comment - Hmmm, scratch the :timestamp thing for now, since timestamps are cell-level rather than row-level. I guess we could do :timestamp:column-name to map them individually, but that deserves its own entry in JIRA.
          Hide
          John Sichi added a comment -

          This also fixes a bug with inserting MAP values containing multiple entries.

          Show
          John Sichi added a comment - This also fixes a bug with inserting MAP values containing multiple entries.
          Hide
          John Sichi added a comment -

          No compound key support (I'll open a separate issue for that).

          Show
          John Sichi added a comment - No compound key support (I'll open a separate issue for that).
          Hide
          Ning Zhang added a comment -

          +1. had an offline review with John and everything looks fine. John is going to file 2 follow up JIRAs for negative test cases and generating splits based on selected columns.

          Tests also passed just now.

          Show
          Ning Zhang added a comment - +1. had an offline review with John and everything looks fine. John is going to file 2 follow up JIRAs for negative test cases and generating splits based on selected columns. Tests also passed just now.
          Hide
          Ning Zhang added a comment -

          Committed. Thanks John!

          Show
          Ning Zhang added a comment - Committed. Thanks John!

            People

            • Assignee:
              John Sichi
              Reporter:
              John Sichi
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development