Hive
  1. Hive
  2. HIVE-3727

Implement join for aliases from storage handler which supports indexed access

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Some storage like hbase supports indexed access on particular columns. But hive does not exploits that kind of traits currently.

      Further, i heard that some companies already implemented secondary indexing on hbase. It will be a great to provide pluggable interfaces for them.

        Activity

        Hide
        Phabricator added a comment -

        navis requested code review of "HIVE-3727 [jira] Implement join for aliases from storage handler which supports indexed access".
        Reviewers: JIRA

        Implement join for aliases from storage handler which supports indexed access

        Some storage like hbase supports indexed access on particular columns. But hive does not exploits that kind of traits currently.

        Further, i heard that some companies already implemented secondary indexing on hbase. It will be a great to provide pluggable interfaces for them.

        TEST PLAN
        EMPTY

        REVISION DETAIL
        https://reviews.facebook.net/D6861

        AFFECTED FILES
        common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseIndexedReader.java
        hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
        hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
        hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
        hbase-handler/src/test/queries/positive/hbase_index_joins.q
        hbase-handler/src/test/results/positive/hbase_index_joins.q.out
        ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractMapJoinOperator.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapper.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableDummyOperator.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/IndexJoinOperator.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/ListSinkOperator.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/LookupJoinOperator.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
        ql/src/java/org/apache/hadoop/hive/ql/plan/ListSinkDesc.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/RandomFetchOperator.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/AbstractMapJoinKey.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinDoubleKeys.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinObjectKey.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinSingleKey.java
        ql/src/java/org/apache/hadoop/hive/ql/io/IndexedReader.java
        ql/src/java/org/apache/hadoop/hive/ql/metadata/IndexedStorageHandler.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeBucketMapJoinOptimizer.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinResolver.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/IndexJoinOptimizer.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/IndexJoinResolver.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/LocalMapJoinProcFactory.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/MapJoinResolver.java
        ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java
        ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
        ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
        ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
        ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java
        ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
        ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java
        ql/src/java/org/apache/hadoop/hive/ql/plan/IndexJoinDesc.java
        ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java
        ql/src/java/org/apache/hadoop/hive/ql/plan/SMBJoinDesc.java
        serde/src/java/org/apache/hadoop/hive/serde2/ByteStream.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/16269/

        To: JIRA, navis

        Show
        Phabricator added a comment - navis requested code review of " HIVE-3727 [jira] Implement join for aliases from storage handler which supports indexed access". Reviewers: JIRA Implement join for aliases from storage handler which supports indexed access Some storage like hbase supports indexed access on particular columns. But hive does not exploits that kind of traits currently. Further, i heard that some companies already implemented secondary indexing on hbase. It will be a great to provide pluggable interfaces for them. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D6861 AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseIndexedReader.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java hbase-handler/src/test/queries/positive/hbase_index_joins.q hbase-handler/src/test/results/positive/hbase_index_joins.q.out ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractMapJoinOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapper.java ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableDummyOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/IndexJoinOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/ListSinkOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/LookupJoinOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java ql/src/java/org/apache/hadoop/hive/ql/plan/ListSinkDesc.java ql/src/java/org/apache/hadoop/hive/ql/exec/RandomFetchOperator.java ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/AbstractMapJoinKey.java ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinDoubleKeys.java ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinObjectKey.java ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinSingleKey.java ql/src/java/org/apache/hadoop/hive/ql/io/IndexedReader.java ql/src/java/org/apache/hadoop/hive/ql/metadata/IndexedStorageHandler.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeBucketMapJoinOptimizer.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinResolver.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenMRSkewJoinProcessor.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/IndexJoinOptimizer.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/IndexJoinResolver.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/LocalMapJoinProcFactory.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/MapJoinResolver.java ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java ql/src/java/org/apache/hadoop/hive/ql/plan/IndexJoinDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java ql/src/java/org/apache/hadoop/hive/ql/plan/SMBJoinDesc.java serde/src/java/org/apache/hadoop/hive/serde2/ByteStream.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/16269/ To: JIRA, navis

          People

          • Assignee:
            Navis
            Reporter:
            Navis
          • Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development