Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13102

CBO: Reduce operations in Calcite do not fold as tight as rule-based folding

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 2.1.0
    • 2.1.0
    • CBO
    • None

    Description

      With CBO

      create temporary table table1(id int, val int, val1 int, dimid int);
      create temporary table table3(id int, val int, val1 int);
      
      hive> explain select table1.id, table1.val, table1.val1 from table1 inner join table3 on table1.dimid = table3.id and table3.id = 1 where table1.dimid <>1 ;
      Warning: Map Join MAPJOIN[14][bigTable=?] in task 'Map 1' is a cross product
      OK
      Plan optimized by CBO.
      
      Vertex dependency in root stage
      Map 1 <- Map 2 (BROADCAST_EDGE)
      
      Stage-0
        Fetch Operator
          limit:-1
          Stage-1
            Map 1 llap
            File Output Operator [FS_11]
              Map Join Operator [MAPJOIN_14] (rows=1 width=0)
                Conds:(Inner),Output:["_col0","_col1","_col2"]
              <-Map 2 [BROADCAST_EDGE] llap
                BROADCAST [RS_8]
                  Select Operator [SEL_5] (rows=1 width=0)
                    Filter Operator [FIL_13] (rows=1 width=0)
                      predicate:(id = 1)
                      TableScan [TS_3] (rows=1 width=0)
                        default@table3,table3,Tbl:PARTIAL,Col:NONE,Output:["id"]
              <-Select Operator [SEL_2] (rows=1 width=0)
                  Output:["_col0","_col1","_col2"]
                  Filter Operator [FIL_12] (rows=1 width=0)
                    predicate:((dimid = 1) and (dimid <> 1))
                    TableScan [TS_0] (rows=1 width=0)
                      default@table1,table1,Tbl:PARTIAL,Col:NONE,Output:["id","val","val1","dimid"]
      

      without CBO

      hive> explain select table1.id, table1.val, table1.val1 from table1 inner join table3 on table1.dimid = table3.id and table3.id = 1 where table1.dimid <>1 ;
      OK
      Vertex dependency in root stage
      Map 1 <- Map 2 (BROADCAST_EDGE)
      
      Stage-0
        Fetch Operator
          limit:-1
          Stage-1
            Map 1 llap
            File Output Operator [FS_9]
              Map Join Operator [MAPJOIN_14] (rows=1 width=0)
                Conds:FIL_12.1=RS_17.1(Inner),Output:["_col0","_col1","_col2"]
              <-Map 2 [BROADCAST_EDGE] vectorized, llap
                BROADCAST [RS_17]
                  PartitionCols:1
                  Filter Operator [FIL_16] (rows=1 width=0)
                    predicate:false
                    TableScan [TS_1] (rows=1 width=0)
                      default@table3,table3,Tbl:PARTIAL,Col:COMPLETE
              <-Filter Operator [FIL_12] (rows=1 width=0)
                  predicate:false
                  TableScan [TS_0] (rows=1 width=0)
                    default@table1,table1,Tbl:PARTIAL,Col:NONE,Output:["id","val","val1"]
      
      Time taken: 0.044 seconds, Fetched: 23 row(s)
      

      Attachments

        1. HIVE-13102.01.patch
          50 kB
          jcamachorodriguez
        2. HIVE-13102.patch
          16 kB
          jcamachorodriguez

        Issue Links

          Activity

            People

              jcamacho Jesús Camacho Rodríguez
              gopalv Gopal Vijayaraghavan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: