Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9132

CBO: Calcite Operator To Hive Operator (Calcite Return Path)

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: CBO
    • Labels:
      None

      Description

      This is the umbrella issue that captures conversion of Calcite Operator Tree to Hive Operator Tree.

        Attachments

          Issue Links

          1.
          CBO (Calcite Return Path): Refactor Semantic Analyzer to Move CBO code out Sub-task Resolved Laljo John Pullokkaran  
          2.
          CBO (Calcite Return Path): Inline Join, Properties Sub-task Resolved Jesus Camacho Rodriguez  
          3.
          CBO (Calcite Return Path): Inline Table, Properties Sub-task Resolved Laljo John Pullokkaran  
          4.
          CBO (Calcite Return Path): Inline FileSinkOperator, Properties Sub-task Resolved Jesus Camacho Rodriguez  
          5.
          CBO (Calcite Return Path): Removing Row Resolvers from ParseContext Sub-task Closed Jesus Camacho Rodriguez  
          6.
          CBO (Calcite Return Path): Inline GroupBy, Properties Sub-task Resolved Jesus Camacho Rodriguez  
          7.
          CBO (Calcite Return Path): Removing AST from ParseContext Sub-task Resolved Jesus Camacho Rodriguez  
          8.
          CBO (Calcite Return Path): Rewrite GlobalLimitOptimizer Sub-task Resolved Jesus Camacho Rodriguez  
          9.
          CBO (Calcite Return Path): Removing QB from ParseContext Sub-task Closed Jesus Camacho Rodriguez  
          10.
          CBO(Calcite Return Path): Translate TS, Select to Hive OP [CBO branch] Sub-task Resolved Laljo John Pullokkaran  
          11.
          CBO(Calcite Return Path): Translate FilterRel to Hive OP [CBO branch] Sub-task Resolved Laljo John Pullokkaran  
          12.
          CBO(Calcite Return Path): Translate GB to Hive OP [CBO branch] Sub-task Closed Laljo John Pullokkaran  
          13.
          CBO (Calcite Return Path): Generate FileSink Op [CBO branch] Sub-task Closed Laljo John Pullokkaran  
          14.
          CBO (Calcite Return Path): Translate Join to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          15.
          CBO (Calcite Return Path): Translate Sort/Limit to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          16.
          CBO (Calcite Return Path): Translate Union to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          17.
          CBO (Calcite Return Path): Translate Exchange to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          18.
          CBO (Calcite Return Path): HiveJoinAddNotNullRule causes StackOverflowError [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          19.
          CBO (Calcite Return Path): Add field nullable check to HiveJoinAddNotNullRule [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          20.
          CBO (Calcite Return Path): Translate PTFs and Windowing to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          21.
          CBO (Calcite Return Path): Join cost calculation improvements and algorithm selection implementation [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          22.
          CBO (Calcite Return Path): GroupingID translation from Calcite [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          23.
          CBO (Calcite Return Path): Metadata provider for bucketing [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          24.
          CBO (Calcite Return Path): Use table md to calculate column size instead of Calcite default values [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          25.
          CBO (Calcite Return Path): Traits propagation for Aggregate operator [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez  
          26.
          CBO (Calcite Return Path): Improvement in the cost calculation algorithm for Aggregate and Join operators [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez  
          27.
          CBO (Calcite Return Path): Prune TS Relnode schema Sub-task Closed Laljo John Pullokkaran  
          28.
          CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          29.
          CBO (Calcite Return Path): Ambiguity table name causes problem in field trimmer [CBO Branch] Sub-task Resolved Laljo John Pullokkaran  
          30.
          CBO (Calcite Return Path): Upgrade to new Calcite snapshot [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez  
          31.
          CBO (Calcite Return Path): RelMdSize throws an Exception when Join is actually a Semijoin [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          32.
          CBO (Calcite Return Path): Internal error: Cannot find common type for join keys Sub-task Resolved Laljo John Pullokkaran  
          33.
          CBO (Calcite Return Path): HiveTableScan copy does not take into account new row schema [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez  
          34.
          CBO (Calcite Return Path): Missing types in averageTypeValueSize method in HiveRelMdSize [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez  
          35.
          CBO (Calcite Return Path): Improve CBO cost related debugging messages for Join operators [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          36.
          CBO (Calcite Return Path): Equi join followed by theta join produces a cross product Sub-task Open Laljo John Pullokkaran  
          37.
          CBO (Calcite Return Path): More details for Join operator in CBO cost based algorithm selection [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          38.
          CBO (Calcite Return Path): Introduce JoinAlgorithm Interface Sub-task Closed Jesus Camacho Rodriguez  
          39.
          CBO (Calcite Return Path): Temporarily disable Aggregate check input for bucketing Sub-task Closed Laljo John Pullokkaran  
          40.
          CBO (Calcite Return Path): Aggregate checking input for bucketing should be conditional Sub-task Closed Jesus Camacho Rodriguez  
          41.
          CBO (Calcite Return Path): TOK_ALLCOLREF not being replaced in GroupBy clause [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          42.
          CBO (Calcite Return Path): HiveRelSize accessing columns without available stats [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          43.
          CBO (Calcite Return Path): Disable choosing streaming side at join creation time [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez  
          44.
          CBO (Calcite Return Path): Invoke Hive's Cumulative Cost Sub-task Closed Laljo John Pullokkaran  
          45.
          CBO (Calcite Return Path): Use SortExchange rather than LogicalExchange for HiveOpConverter Sub-task Resolved Pengcheng Xiong  
          46.
          CBO (Calcite Return Path): TraitSets not correctly propagated in HiveSortExchange causes Assertion error Sub-task Resolved Jesus Camacho Rodriguez  
          47.
          CBO (Calcite Return Path): Parameterize algorithm cost model Sub-task Closed Laljo John Pullokkaran  
          48.
          CBO (Calcite Return Path): Use newInstance to create ExprNodeGenericFuncDesc rather than construction function Sub-task Resolved Pengcheng Xiong  
          49.
          CBO (Calcite Return Path):Handle count ( * ) case for cbo return path Sub-task Resolved Ashutosh Chauhan  
          50.
          CBO: Use total size instead of bucket count to determine number of splits & parallelism Sub-task Closed Mostafa Mokhtar  
          51.
          CBO (Calcite Return Path): Remove HiveCostUtil Sub-task Resolved Jesus Camacho Rodriguez  
          52.
          CBO (Calcite Return Path): Overriding toString method in join algorithms Sub-task Resolved Jesus Camacho Rodriguez  
          53.
          CBO: Don't use HiveDefaultCostModel when With Tez and hive.cbo.costmodel.extended enabled Sub-task Closed Laljo John Pullokkaran  
          54.
          CBO (Calcite Return Path): Tag not set up correctly for Join operator in HiveOpConverter Sub-task Resolved Jesus Camacho Rodriguez  
          55.
          CBO (Calcite Return Path): Complete translation for all kinds of RexNode in Windowing Sub-task Resolved Jesus Camacho Rodriguez  
          56.
          CBO (Calcite Return Path): disable the identity project remover for some union operators Sub-task Closed Pengcheng Xiong  
          57.
          CBO (Calcite Return Path): move qbid from RelOptHiveTable to HiveTableScan Sub-task Resolved Pengcheng Xiong  
          58.
          CBO (Calcite Return Path): Add QBId to TableScan digest Sub-task Resolved Jesus Camacho Rodriguez  
          59.
          CBO (Calcite Return Path): Disable Trivial Project Removal on ret path Sub-task Closed Laljo John Pullokkaran  
          60.
          CBO (Calcite Return Path): splitJoinCondition does not behave correctly when one side of the condition references columns from different inputs Sub-task Resolved Jesus Camacho Rodriguez  
          61.
          CBO (Calcite Return Path): Wrong type for windowing UDFs that produce different values for each row Sub-task Resolved Jesus Camacho Rodriguez  
          62.
          CBO (Calcite Return Path): HiveOpConverter always assumes that HiveFilter does not include a partition column Sub-task Closed Laljo John Pullokkaran  
          63.
          CBO (Calcite Return Path): colExprMap not generated for SelectOperator in HiveOpConverter causes problems in PPD Sub-task Resolved Jesus Camacho Rodriguez  
          64.
          CBO (Calcite Return Path): Exception when column name contains dot or colon characters Sub-task Closed Jesus Camacho Rodriguez  
          65.
          CBO : Calculate join selectivity when computing HiveJoin cost Sub-task Open Laljo John Pullokkaran  
          66.
          CBO (Calcite Return Path): Fix return columns if Sort operator is on top of plan returned by Calcite Sub-task Closed Jesus Camacho Rodriguez  
          67.
          [CBO] Return path assumes distinct column cant be same as grouping column Sub-task Closed Laljo John Pullokkaran  
          68.
          CBO (Calcite Return Path): Different data types at Reducer before JoinOp Sub-task Closed Pengcheng Xiong  
          69.
          CBO (Calcite Return Path): MapJoin and SMBJoin conversion not triggered Sub-task Closed Jesus Camacho Rodriguez  
          70.
          CBO (Calcite Return Path): Insert overwrite... select * from... queries failing for bucketed tables Sub-task Resolved Laljo John Pullokkaran  
          71.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) Empty tabAlias in columnInfo which triggers PPD Sub-task Closed Pengcheng Xiong  
          72.
          Merge multiple joins when join keys are the same Sub-task Closed Pengcheng Xiong  
          73.
          CBO (Calcite Return Path): Disallow return path to be enable if CBO is off Sub-task Closed Jesus Camacho Rodriguez  
          74.
          CBO (Calcite Return Path): SMBJoin conversion throws ClassCastException Sub-task Closed Jesus Camacho Rodriguez  
          75.
          CBO (Calcite Return Path): fix the wrong needed column names when TS is created Sub-task Closed Pengcheng Xiong  
          76.
          CBO (Calcite Return Path): HiveCost epsilon comparison should take row count in to account Sub-task Closed Laljo John Pullokkaran  
          77.
          CBO (Calcite Return Path): Join to MultiJoin rule Sub-task Closed Jesus Camacho Rodriguez  
          78.
          CBO (Calcite Return Path): Join to MultiJoin support for outer joins Sub-task Closed Jesus Camacho Rodriguez  
          79.
          CBO (Calcite Return Path) : genFileSinkPlan uses wrong partition col to create FS Sub-task Closed Pengcheng Xiong  
          80.
          CBO (Calcite Return Path): Enable NonBlockingOpDeDupProc Sub-task Closed Pengcheng Xiong  
          81.
          CBO (Calcite Return Path): Setup correct information if CBO succeeds Sub-task Closed Jesus Camacho Rodriguez  
          82.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): optimizer for limit 0 does not work Sub-task Closed Pengcheng Xiong  
          83.
          CBO (Calcite Return Path): auto_join2.q returning wrong results Sub-task Closed Jesus Camacho Rodriguez  
          84.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) empty filtersMap of join operator causes wrong results Sub-task Closed Jesus Camacho Rodriguez  
          85.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): NonBlockingOpDeDupProc did not kick in rcfile_merge2.q Sub-task Closed Jesus Camacho Rodriguez  
          86.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : Regression on the latest master Sub-task Resolved Jesus Camacho Rodriguez  
          87.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): dpCtx's mapInputToDP should depends on the last SEL Sub-task Closed Pengcheng Xiong  
          88.
          CBO (Calcite Return Path): Join translation should update all ExprNode recursively Sub-task Closed Jesus Camacho Rodriguez  
          89.
          CBO (Calcite Return Path): MapJoin and SMBJoin conversion not triggered Sub-task Closed Jesus Camacho Rodriguez  
          90.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): make the output of ba_table_union.q more stable Sub-task Closed Pengcheng Xiong  
          91.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix the output of select_same_col.q Sub-task Closed Pengcheng Xiong  
          92.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): reconsider the cross product in HiveJoinToMultiJoinRule Sub-task Resolved Pengcheng Xiong  
          93.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): filterMaps is null in vectorMapJoin Sub-task Resolved Pengcheng Xiong  
          94.
          CBO (Calcite Return Path): Extending ExprNodeConverter to consider additional types Sub-task Closed Jesus Camacho Rodriguez  
          95.
          CBO (Calcite Return Path): DUMMY project in plan Sub-task Closed Jesus Camacho Rodriguez  
          96.
          CBO (Calcite Return Path): Inferring Hive type char/varchar of length zero which is not allowed Sub-task Closed Jesus Camacho Rodriguez  
          97.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix CTAS Sub-task Closed Pengcheng Xiong  
          98.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): ExprNodeConverter should use HiveDecimal to create Decimal Sub-task Closed Pengcheng Xiong  
          99.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : fix reduce_deduplicate optimization Sub-task Closed Pengcheng Xiong  
          100.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): Method isCombinablePredicate in HiveJoinToMultiJoinRule should be extended to support MultiJoin operators merge Sub-task Closed Jesus Camacho Rodriguez  
          101.
          CBO (Calcite Return Path): Fix table alias propagation for windowing Sub-task Resolved Jesus Camacho Rodriguez  
          102.
          CBO (Calcite Return Path): Add CBO tests with return path on Sub-task Closed Jesus Camacho Rodriguez  
          103.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): add SEL before UNION Sub-task Closed Pengcheng Xiong  
          104.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): Groupby Optimizer assumes the schema can match after removing RS and GBY Sub-task Closed Pengcheng Xiong  
          105.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : dealing with empty char Sub-task Closed Pengcheng Xiong  
          106.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : dealing with insert into Sub-task Closed Pengcheng Xiong  
          107.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : groupby distinct does not work Sub-task Closed Jesus Camacho Rodriguez  
          108.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix stringToMap Sub-task Resolved Pengcheng Xiong  
          109.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): char/varchar as input to GenericUDAF Sub-task Resolved Pengcheng Xiong  
          110.
          CBO (Calcite Return Path): Convert to flat AND/OR Sub-task Closed Jesus Camacho Rodriguez  
          111.
          [CBO new return path] Handling of strings of zero-length Sub-task Closed Ashutosh Chauhan  
          112.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): ctas after order by has problem Sub-task Resolved Pengcheng Xiong  
          113.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix the tableAlias for ReduceSink operator Sub-task Resolved Pengcheng Xiong  
          114.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : fix the filter expressions for full outer join and right outer join Sub-task Resolved Pengcheng Xiong  
          115.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix multiple window spec for PTF operator Sub-task Closed Pengcheng Xiong  
          116.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): correct table column name in CTAS queries Sub-task Resolved Pengcheng Xiong  
          117.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix udaf_percentile_approx_23.q Sub-task Closed Pengcheng Xiong  
          118.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): deal with hive default partition when inserting data Sub-task Closed Ashutosh Chauhan  
          119.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix TestJdbcDriver2 Sub-task Resolved Pengcheng Xiong  
          120.
          CBO (Calcite Return Path): groupby_grouping_id2.q returns wrong results Sub-task Open Vineet Garg  
          121.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : columnPruner prunes everything when union is the last operator before FS Sub-task Closed Pengcheng Xiong  
          122.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : dealing with '$' in typeInfo Sub-task Closed Pengcheng Xiong  
          123.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix test failure for udf_percentile.q Sub-task Resolved Pengcheng Xiong  
          124.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): UDAF can not pull up constant expressions Sub-task Closed Pengcheng Xiong  
          125.
          CBO (Calcite Return Path): Support for unique join Sub-task Resolved Jesus Camacho Rodriguez  
          126.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix test failure for groupby_resolution.q Sub-task Resolved Laljo John Pullokkaran  
          127.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver.vector* queries failures due to NPE in Vectorizer.onExpressionHasNullSafes() Sub-task Closed Hari Sankar Sivarama Subramaniyan  
          128.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver.vector_join_filters.q failure Sub-task Closed Hari Sankar Sivarama Subramaniyan  
          129.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver count.q failure Sub-task Resolved Vineet Garg  
          130.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver skewjoin.q failure Sub-task Closed Hari Sankar Sivarama Subramaniyan  
          131.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver vector_auto_smb_mapjoin_14.q failure Sub-task Patch Available Vineet Garg  
          132.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_grouping_sets4.q failure Sub-task Patch Available Hari Sankar Sivarama Subramaniyan  
          133.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_ppr_multi_distinct.q failure Sub-task Resolved Vineet Garg  
          134.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): RexNode convert(ExprNodeConstantDesc literal) decimal support bug Sub-task Closed Hari Sankar Sivarama Subramaniyan  
          135.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver count.q failure Sub-task Closed Vineet Garg  
          136.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong result/plan in group by with hive.map.aggr=false Sub-task Resolved Vineet Garg  
          137.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Fix test failure for auto_join_filters Sub-task Resolved Vineet Garg  
          138.
          CBO (Calcite Return Path) Wrong results for limit + offset Sub-task Resolved Ashutosh Chauhan  
          139.
          CBO (Calcite Return Path) Handle field access for nested column Sub-task Resolved Ashutosh Chauhan  
          140.
          CBO (Calcite Return Path) Incorrect result set when limit is present in one of the union branches Sub-task Closed Ashutosh Chauhan  
          141.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Fix wrong result in join_filters_overlap Sub-task Open Vineet Garg  
          142.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Fix wrong result in input30 Sub-task Resolved Vineet Garg  
          143.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Support Tablesample clause Sub-task Open Vineet Garg  
          144.
          Create view doesn't work with Calcite Return path Sub-task Open Unassigned  
          145.
          Join fails on operator return path on CBO Sub-task Open Unassigned  
          146.
          Handle HiveTableFunctionScan at return path Sub-task Resolved Miklos Gergely

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h 20m
          147.
          Improve return path enabling/disabling Sub-task Patch Available Ashutosh Chauhan

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 3.5h
          148.
          Fix insert statement issue with return path Sub-task Resolved Miklos Gergely

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          149.
          Remove disabling of IdentityProjectRemover transformation for Return Path Sub-task Resolved Miklos Gergely  

            Activity

              People

              • Assignee:
                jcamachorodriguez Jesus Camacho Rodriguez
                Reporter:
                jpullokkaran Laljo John Pullokkaran
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 5h 10m
                  5h 10m