Hive
  1. Hive
  2. HIVE-3767

BucketizedHiveInputFormat should be automatically used with Bucketized Map Joins also

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Query Processor
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      It seems like even after https://issues.apache.org/jira/browse/HIVE-3219 we still need to set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat for bucket map joins (though not SMB joins)

        Issue Links

          Activity

          Show
          Gang Tim Liu added a comment - https://reviews.facebook.net/D7209
          Hide
          Gang Tim Liu added a comment -

          patch is available.

          Show
          Gang Tim Liu added a comment - patch is available.
          Hide
          Namit Jain added a comment -

          +1

          looks good, can you file a follow-up for adding MapJoinDesc.isBucketMapJoin() as part of explain plan ?

          Show
          Namit Jain added a comment - +1 looks good, can you file a follow-up for adding MapJoinDesc.isBucketMapJoin() as part of explain plan ?
          Hide
          Gang Tim Liu added a comment -

          thanks a lot

          Yes, filed HIVE-3778.

          thanks a lot

          Show
          Gang Tim Liu added a comment - thanks a lot Yes, filed HIVE-3778 . thanks a lot
          Hide
          Namit Jain added a comment -

          Committed. Thanks Tim

          Show
          Namit Jain added a comment - Committed. Thanks Tim
          Hide
          Gang Tim Liu added a comment -

          Namit thanks a lot Tim

          Show
          Gang Tim Liu added a comment - Namit thanks a lot Tim
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1841 (See https://builds.apache.org/job/Hive-trunk-h0.21/1841/)
          HIVE-3767 BucketizedHiveInputFormat should be automatically used with
          Bucketized Map Joins also (Gang Tim Liu via namit) (Revision 1418286)

          Result = FAILURE
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1418286
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java
          • /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_2.q
          • /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin13.q
          • /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin6.q
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1841 (See https://builds.apache.org/job/Hive-trunk-h0.21/1841/ ) HIVE-3767 BucketizedHiveInputFormat should be automatically used with Bucketized Map Joins also (Gang Tim Liu via namit) (Revision 1418286) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1418286 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_1.q /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_2.q /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin13.q /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin6.q
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
          HIVE-3767 BucketizedHiveInputFormat should be automatically used with
          Bucketized Map Joins also (Gang Tim Liu via namit) (Revision 1418286)

          Result = ABORTED
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1418286
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java
          • /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_1.q
          • /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_2.q
          • /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin13.q
          • /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin6.q
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-3767 BucketizedHiveInputFormat should be automatically used with Bucketized Map Joins also (Gang Tim Liu via namit) (Revision 1418286) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1418286 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_1.q /hive/trunk/ql/src/test/queries/clientpositive/bucket_map_join_2.q /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin13.q /hive/trunk/ql/src/test/queries/clientpositive/bucketmapjoin6.q

            People

            • Assignee:
              Gang Tim Liu
              Reporter:
              Namit Jain
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development