Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-6584

Add HiveHBaseTableSnapshotInputFormat

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.14.0
    • HBase Handler
    • Hide
      Hive can now execute queries against HBase table snapshots. This feature is available for any table defined using the HBaseStorageHandler. It requires at least HBase 0.98.3.

      To query against a snapshot instead of the online table, specify the snapshot name via hive.hbase.snapshot.name. The snapshot will be restored into a unique directory under /tmp. This location can be overridden by setting a path via hive.hbase.snapshot.restoredir.
      Show
      Hive can now execute queries against HBase table snapshots. This feature is available for any table defined using the HBaseStorageHandler. It requires at least HBase 0.98.3. To query against a snapshot instead of the online table, specify the snapshot name via hive.hbase.snapshot.name. The snapshot will be restored into a unique directory under /tmp. This location can be overridden by setting a path via hive.hbase.snapshot.restoredir.

    Description

      HBASE-8369 provided mapreduce support for reading from HBase table snapsopts. This allows a MR job to consume a stable, read-only view of an HBase table directly off of HDFS. Bypassing the online region server API provides a nice performance boost for the full scan. HBASE-10642 is backporting that feature to 0.94/0.96 and also adding a mapred implementation. Once that's available, we should add an input format. A follow-on patch could work out how to integrate this functionality into the StorageHandler, similar to how HIVE-6473 integrates the HFileOutputFormat into existing table definitions.

      Attachments

        1. HIVE-6584.0.patch
          21 kB
          Nick Dimiduk
        2. HIVE-6584.1.patch
          21 kB
          Teng Yutong
        3. HIVE-6584.10.patch
          48 kB
          Nick Dimiduk
        4. HIVE-6584.11.patch
          48 kB
          Nick Dimiduk
        5. HIVE-6584.12.patch
          48 kB
          Nick Dimiduk
        6. HIVE-6584.13.patch
          47 kB
          Nick Dimiduk
        7. HIVE-6584.14.patch
          50 kB
          Nick Dimiduk
        8. HIVE-6584.2.patch
          22 kB
          Nick Dimiduk
        9. HIVE-6584.3.patch
          23 kB
          Nick Dimiduk
        10. HIVE-6584.4.patch
          25 kB
          Nick Dimiduk
        11. HIVE-6584.5.patch
          26 kB
          Teng Yutong
        12. HIVE-6584.6.patch
          26 kB
          Teng Yutong
        13. HIVE-6584.7.patch
          46 kB
          Teng Yutong
        14. HIVE-6584.8.patch
          58 kB
          Nick Dimiduk
        15. HIVE-6584.9.patch
          41 kB
          Nick Dimiduk

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ndimiduk Nick Dimiduk Assign to me
            ndimiduk Nick Dimiduk
            Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment