Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-6584

Add HiveHBaseTableSnapshotInputFormat

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.14.0
    • HBase Handler
    • Hide
      Hive can now execute queries against HBase table snapshots. This feature is available for any table defined using the HBaseStorageHandler. It requires at least HBase 0.98.3.

      To query against a snapshot instead of the online table, specify the snapshot name via hive.hbase.snapshot.name. The snapshot will be restored into a unique directory under /tmp. This location can be overridden by setting a path via hive.hbase.snapshot.restoredir.
      Show
      Hive can now execute queries against HBase table snapshots. This feature is available for any table defined using the HBaseStorageHandler. It requires at least HBase 0.98.3. To query against a snapshot instead of the online table, specify the snapshot name via hive.hbase.snapshot.name. The snapshot will be restored into a unique directory under /tmp. This location can be overridden by setting a path via hive.hbase.snapshot.restoredir.

    Description

      HBASE-8369 provided mapreduce support for reading from HBase table snapsopts. This allows a MR job to consume a stable, read-only view of an HBase table directly off of HDFS. Bypassing the online region server API provides a nice performance boost for the full scan. HBASE-10642 is backporting that feature to 0.94/0.96 and also adding a mapred implementation. Once that's available, we should add an input format. A follow-on patch could work out how to integrate this functionality into the StorageHandler, similar to how HIVE-6473 integrates the HFileOutputFormat into existing table definitions.

      Attachments

        1. HIVE-6584.0.patch
          21 kB
          Nick Dimiduk
        2. HIVE-6584.1.patch
          21 kB
          Teng Yutong
        3. HIVE-6584.2.patch
          22 kB
          Nick Dimiduk
        4. HIVE-6584.3.patch
          23 kB
          Nick Dimiduk
        5. HIVE-6584.4.patch
          25 kB
          Nick Dimiduk
        6. HIVE-6584.5.patch
          26 kB
          Teng Yutong
        7. HIVE-6584.6.patch
          26 kB
          Teng Yutong
        8. HIVE-6584.7.patch
          46 kB
          Teng Yutong
        9. HIVE-6584.8.patch
          58 kB
          Nick Dimiduk
        10. HIVE-6584.9.patch
          41 kB
          Nick Dimiduk
        11. HIVE-6584.10.patch
          48 kB
          Nick Dimiduk
        12. HIVE-6584.11.patch
          48 kB
          Nick Dimiduk
        13. HIVE-6584.12.patch
          48 kB
          Nick Dimiduk
        14. HIVE-6584.13.patch
          47 kB
          Nick Dimiduk
        15. HIVE-6584.14.patch
          50 kB
          Nick Dimiduk

        Issue Links

          Activity

            People

              ndimiduk Nick Dimiduk
              ndimiduk Nick Dimiduk
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: