Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-6584

Add HiveHBaseTableSnapshotInputFormat

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: HBase Handler
    • Labels:
    • Release Note:
      Hide
      Hive can now execute queries against HBase table snapshots. This feature is available for any table defined using the HBaseStorageHandler. It requires at least HBase 0.98.3.

      To query against a snapshot instead of the online table, specify the snapshot name via hive.hbase.snapshot.name. The snapshot will be restored into a unique directory under /tmp. This location can be overridden by setting a path via hive.hbase.snapshot.restoredir.
      Show
      Hive can now execute queries against HBase table snapshots. This feature is available for any table defined using the HBaseStorageHandler. It requires at least HBase 0.98.3. To query against a snapshot instead of the online table, specify the snapshot name via hive.hbase.snapshot.name. The snapshot will be restored into a unique directory under /tmp. This location can be overridden by setting a path via hive.hbase.snapshot.restoredir.

      Description

      HBASE-8369 provided mapreduce support for reading from HBase table snapsopts. This allows a MR job to consume a stable, read-only view of an HBase table directly off of HDFS. Bypassing the online region server API provides a nice performance boost for the full scan. HBASE-10642 is backporting that feature to 0.94/0.96 and also adding a mapred implementation. Once that's available, we should add an input format. A follow-on patch could work out how to integrate this functionality into the StorageHandler, similar to how HIVE-6473 integrates the HFileOutputFormat into existing table definitions.

        Attachments

        1. HIVE-6584.0.patch
          21 kB
          Nick Dimiduk
        2. HIVE-6584.1.patch
          21 kB
          Teng Yutong
        3. HIVE-6584.10.patch
          48 kB
          Nick Dimiduk
        4. HIVE-6584.11.patch
          48 kB
          Nick Dimiduk
        5. HIVE-6584.12.patch
          48 kB
          Nick Dimiduk
        6. HIVE-6584.13.patch
          47 kB
          Nick Dimiduk
        7. HIVE-6584.14.patch
          50 kB
          Nick Dimiduk
        8. HIVE-6584.2.patch
          22 kB
          Nick Dimiduk
        9. HIVE-6584.3.patch
          23 kB
          Nick Dimiduk
        10. HIVE-6584.4.patch
          25 kB
          Nick Dimiduk
        11. HIVE-6584.5.patch
          26 kB
          Teng Yutong
        12. HIVE-6584.6.patch
          26 kB
          Teng Yutong
        13. HIVE-6584.7.patch
          46 kB
          Teng Yutong
        14. HIVE-6584.8.patch
          58 kB
          Nick Dimiduk
        15. HIVE-6584.9.patch
          41 kB
          Nick Dimiduk

          Issue Links

            Activity

              People

              • Assignee:
                ndimiduk Nick Dimiduk
                Reporter:
                ndimiduk Nick Dimiduk
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: