Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9132

CBO: Calcite Operator To Hive Operator (Calcite Return Path)

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: CBO
    • Labels:
      None

      Description

      This is the umbrella issue that captures conversion of Calcite Operator Tree to Hive Operator Tree.

        Attachments

          Issue Links

          1.
          CBO (Calcite Return Path): Refactor Semantic Analyzer to Move CBO code out Sub-task Resolved Laljo John Pullokkaran
          2.
          CBO (Calcite Return Path): Inline Join, Properties Sub-task Resolved Jesus Camacho Rodriguez
          3.
          CBO (Calcite Return Path): Inline Table, Properties Sub-task Resolved Laljo John Pullokkaran
          4.
          CBO (Calcite Return Path): Inline FileSinkOperator, Properties Sub-task Resolved Jesus Camacho Rodriguez
          5.
          CBO (Calcite Return Path): Removing Row Resolvers from ParseContext Sub-task Closed Jesus Camacho Rodriguez
          6.
          CBO (Calcite Return Path): Inline GroupBy, Properties Sub-task Resolved Jesus Camacho Rodriguez
          7.
          CBO (Calcite Return Path): Removing AST from ParseContext Sub-task Resolved Jesus Camacho Rodriguez
          8.
          CBO (Calcite Return Path): Rewrite GlobalLimitOptimizer Sub-task Resolved Jesus Camacho Rodriguez
          9.
          CBO (Calcite Return Path): Removing QB from ParseContext Sub-task Closed Jesus Camacho Rodriguez
          10.
          CBO(Calcite Return Path): Translate TS, Select to Hive OP [CBO branch] Sub-task Resolved Laljo John Pullokkaran
          11.
          CBO(Calcite Return Path): Translate FilterRel to Hive OP [CBO branch] Sub-task Resolved Laljo John Pullokkaran
          12.
          CBO(Calcite Return Path): Translate GB to Hive OP [CBO branch] Sub-task Closed Laljo John Pullokkaran
          13.
          CBO (Calcite Return Path): Generate FileSink Op [CBO branch] Sub-task Closed Laljo John Pullokkaran
          14.
          CBO (Calcite Return Path): Translate Join to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          15.
          CBO (Calcite Return Path): Translate Sort/Limit to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          16.
          CBO (Calcite Return Path): Translate Union to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          17.
          CBO (Calcite Return Path): Translate Exchange to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          18.
          CBO (Calcite Return Path): HiveJoinAddNotNullRule causes StackOverflowError [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          19.
          CBO (Calcite Return Path): Add field nullable check to HiveJoinAddNotNullRule [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          20.
          CBO (Calcite Return Path): Translate PTFs and Windowing to Hive Op [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          21.
          CBO (Calcite Return Path): Join cost calculation improvements and algorithm selection implementation [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          22.
          CBO (Calcite Return Path): GroupingID translation from Calcite [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          23.
          CBO (Calcite Return Path): Metadata provider for bucketing [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          24.
          CBO (Calcite Return Path): Use table md to calculate column size instead of Calcite default values [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          25.
          CBO (Calcite Return Path): Traits propagation for Aggregate operator [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez
          26.
          CBO (Calcite Return Path): Improvement in the cost calculation algorithm for Aggregate and Join operators [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez
          27.
          CBO (Calcite Return Path): Prune TS Relnode schema Sub-task Closed Laljo John Pullokkaran
          28.
          CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          29.
          CBO (Calcite Return Path): Ambiguity table name causes problem in field trimmer [CBO Branch] Sub-task Resolved Laljo John Pullokkaran
          30.
          CBO (Calcite Return Path): Upgrade to new Calcite snapshot [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez
          31.
          CBO (Calcite Return Path): RelMdSize throws an Exception when Join is actually a Semijoin [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          32.
          CBO (Calcite Return Path): Internal error: Cannot find common type for join keys Sub-task Resolved Laljo John Pullokkaran
          33.
          CBO (Calcite Return Path): HiveTableScan copy does not take into account new row schema [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez
          34.
          CBO (Calcite Return Path): Missing types in averageTypeValueSize method in HiveRelMdSize [CBO Branch] Sub-task Resolved Jesus Camacho Rodriguez
          35.
          CBO (Calcite Return Path): Improve CBO cost related debugging messages for Join operators [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          36.
          CBO (Calcite Return Path): Equi join followed by theta join produces a cross product Sub-task Open Laljo John Pullokkaran
          37.
          CBO (Calcite Return Path): More details for Join operator in CBO cost based algorithm selection [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          38.
          CBO (Calcite Return Path): Introduce JoinAlgorithm Interface Sub-task Closed Jesus Camacho Rodriguez
          39.
          CBO (Calcite Return Path): Temporarily disable Aggregate check input for bucketing Sub-task Closed Laljo John Pullokkaran
          40.
          CBO (Calcite Return Path): Aggregate checking input for bucketing should be conditional Sub-task Closed Jesus Camacho Rodriguez
          41.
          CBO (Calcite Return Path): TOK_ALLCOLREF not being replaced in GroupBy clause [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          42.
          CBO (Calcite Return Path): HiveRelSize accessing columns without available stats [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          43.
          CBO (Calcite Return Path): Disable choosing streaming side at join creation time [CBO branch] Sub-task Resolved Jesus Camacho Rodriguez
          44.
          CBO (Calcite Return Path): Invoke Hive's Cumulative Cost Sub-task Closed Laljo John Pullokkaran
          45.
          CBO (Calcite Return Path): Use SortExchange rather than LogicalExchange for HiveOpConverter Sub-task Resolved Pengcheng Xiong
          46.
          CBO (Calcite Return Path): TraitSets not correctly propagated in HiveSortExchange causes Assertion error Sub-task Resolved Jesus Camacho Rodriguez
          47.
          CBO (Calcite Return Path): Parameterize algorithm cost model Sub-task Closed Laljo John Pullokkaran
          48.
          CBO (Calcite Return Path): Use newInstance to create ExprNodeGenericFuncDesc rather than construction function Sub-task Resolved Pengcheng Xiong
          49.
          CBO (Calcite Return Path):Handle count ( * ) case for cbo return path Sub-task Resolved Ashutosh Chauhan
          50.
          CBO: Use total size instead of bucket count to determine number of splits & parallelism Sub-task Closed Mostafa Mokhtar
          51.
          CBO (Calcite Return Path): Remove HiveCostUtil Sub-task Resolved Jesus Camacho Rodriguez
          52.
          CBO (Calcite Return Path): Overriding toString method in join algorithms Sub-task Resolved Jesus Camacho Rodriguez
          53.
          CBO: Don't use HiveDefaultCostModel when With Tez and hive.cbo.costmodel.extended enabled Sub-task Closed Laljo John Pullokkaran
          54.
          CBO (Calcite Return Path): Tag not set up correctly for Join operator in HiveOpConverter Sub-task Resolved Jesus Camacho Rodriguez
          55.
          CBO (Calcite Return Path): Complete translation for all kinds of RexNode in Windowing Sub-task Resolved Jesus Camacho Rodriguez
          56.
          CBO (Calcite Return Path): disable the identity project remover for some union operators Sub-task Closed Pengcheng Xiong
          57.
          CBO (Calcite Return Path): move qbid from RelOptHiveTable to HiveTableScan Sub-task Resolved Pengcheng Xiong
          58.
          CBO (Calcite Return Path): Add QBId to TableScan digest Sub-task Resolved Jesus Camacho Rodriguez
          59.
          CBO (Calcite Return Path): Disable Trivial Project Removal on ret path Sub-task Closed Laljo John Pullokkaran
          60.
          CBO (Calcite Return Path): splitJoinCondition does not behave correctly when one side of the condition references columns from different inputs Sub-task Resolved Jesus Camacho Rodriguez
          61.
          CBO (Calcite Return Path): Wrong type for windowing UDFs that produce different values for each row Sub-task Resolved Jesus Camacho Rodriguez
          62.
          CBO (Calcite Return Path): HiveOpConverter always assumes that HiveFilter does not include a partition column Sub-task Closed Laljo John Pullokkaran
          63.
          CBO (Calcite Return Path): colExprMap not generated for SelectOperator in HiveOpConverter causes problems in PPD Sub-task Resolved Jesus Camacho Rodriguez
          64.
          CBO (Calcite Return Path): Exception when column name contains dot or colon characters Sub-task Closed Jesus Camacho Rodriguez
          65.
          CBO : Calculate join selectivity when computing HiveJoin cost Sub-task Open Laljo John Pullokkaran
          66.
          CBO (Calcite Return Path): Fix return columns if Sort operator is on top of plan returned by Calcite Sub-task Closed Jesus Camacho Rodriguez
          67.
          [CBO] Return path assumes distinct column cant be same as grouping column Sub-task Closed Laljo John Pullokkaran
          68.
          CBO (Calcite Return Path): Different data types at Reducer before JoinOp Sub-task Closed Pengcheng Xiong
          69.
          CBO (Calcite Return Path): MapJoin and SMBJoin conversion not triggered Sub-task Closed Jesus Camacho Rodriguez
          70.
          CBO (Calcite Return Path): Insert overwrite... select * from... queries failing for bucketed tables Sub-task Resolved Laljo John Pullokkaran
          71.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) Empty tabAlias in columnInfo which triggers PPD Sub-task Closed Pengcheng Xiong
          72.
          Merge multiple joins when join keys are the same Sub-task Closed Pengcheng Xiong
          73.
          CBO (Calcite Return Path): Disallow return path to be enable if CBO is off Sub-task Closed Jesus Camacho Rodriguez
          74.
          CBO (Calcite Return Path): SMBJoin conversion throws ClassCastException Sub-task Closed Jesus Camacho Rodriguez
          75.
          CBO (Calcite Return Path): fix the wrong needed column names when TS is created Sub-task Closed Pengcheng Xiong
          76.
          CBO (Calcite Return Path): HiveCost epsilon comparison should take row count in to account Sub-task Closed Laljo John Pullokkaran
          77.
          CBO (Calcite Return Path): Join to MultiJoin rule Sub-task Closed Jesus Camacho Rodriguez
          78.
          CBO (Calcite Return Path): Join to MultiJoin support for outer joins Sub-task Closed Jesus Camacho Rodriguez
          79.
          CBO (Calcite Return Path) : genFileSinkPlan uses wrong partition col to create FS Sub-task Closed Pengcheng Xiong
          80.
          CBO (Calcite Return Path): Enable NonBlockingOpDeDupProc Sub-task Closed Pengcheng Xiong
          81.
          CBO (Calcite Return Path): Setup correct information if CBO succeeds Sub-task Closed Jesus Camacho Rodriguez
          82.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): optimizer for limit 0 does not work Sub-task Closed Pengcheng Xiong
          83.
          CBO (Calcite Return Path): auto_join2.q returning wrong results Sub-task Closed Jesus Camacho Rodriguez
          84.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) empty filtersMap of join operator causes wrong results Sub-task Closed Jesus Camacho Rodriguez
          85.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): NonBlockingOpDeDupProc did not kick in rcfile_merge2.q Sub-task Closed Jesus Camacho Rodriguez
          86.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : Regression on the latest master Sub-task Resolved Jesus Camacho Rodriguez
          87.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): dpCtx's mapInputToDP should depends on the last SEL Sub-task Closed Pengcheng Xiong
          88.
          CBO (Calcite Return Path): Join translation should update all ExprNode recursively Sub-task Closed Jesus Camacho Rodriguez
          89.
          CBO (Calcite Return Path): MapJoin and SMBJoin conversion not triggered Sub-task Closed Jesus Camacho Rodriguez
          90.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): make the output of ba_table_union.q more stable Sub-task Closed Pengcheng Xiong
          91.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix the output of select_same_col.q Sub-task Closed Pengcheng Xiong
          92.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): reconsider the cross product in HiveJoinToMultiJoinRule Sub-task Resolved Pengcheng Xiong
          93.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): filterMaps is null in vectorMapJoin Sub-task Resolved Pengcheng Xiong
          94.
          CBO (Calcite Return Path): Extending ExprNodeConverter to consider additional types Sub-task Closed Jesus Camacho Rodriguez
          95.
          CBO (Calcite Return Path): DUMMY project in plan Sub-task Closed Jesus Camacho Rodriguez
          96.
          CBO (Calcite Return Path): Inferring Hive type char/varchar of length zero which is not allowed Sub-task Closed Jesus Camacho Rodriguez
          97.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix CTAS Sub-task Closed Pengcheng Xiong
          98.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): ExprNodeConverter should use HiveDecimal to create Decimal Sub-task Closed Pengcheng Xiong
          99.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : fix reduce_deduplicate optimization Sub-task Closed Pengcheng Xiong
          100.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): Method isCombinablePredicate in HiveJoinToMultiJoinRule should be extended to support MultiJoin operators merge Sub-task Closed Jesus Camacho Rodriguez
          101.
          CBO (Calcite Return Path): Fix table alias propagation for windowing Sub-task Resolved Jesus Camacho Rodriguez
          102.
          CBO (Calcite Return Path): Add CBO tests with return path on Sub-task Closed Jesus Camacho Rodriguez
          103.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): add SEL before UNION Sub-task Closed Pengcheng Xiong
          104.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): Groupby Optimizer assumes the schema can match after removing RS and GBY Sub-task Closed Pengcheng Xiong
          105.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : dealing with empty char Sub-task Closed Pengcheng Xiong
          106.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : dealing with insert into Sub-task Closed Pengcheng Xiong
          107.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : groupby distinct does not work Sub-task Closed Jesus Camacho Rodriguez
          108.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix stringToMap Sub-task Resolved Pengcheng Xiong
          109.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): char/varchar as input to GenericUDAF Sub-task Resolved Pengcheng Xiong
          110.
          CBO (Calcite Return Path): Convert to flat AND/OR Sub-task Closed Jesus Camacho Rodriguez
          111.
          [CBO new return path] Handling of strings of zero-length Sub-task Closed Ashutosh Chauhan
          112.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): ctas after order by has problem Sub-task Resolved Pengcheng Xiong
          113.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix the tableAlias for ReduceSink operator Sub-task Resolved Pengcheng Xiong
          114.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : fix the filter expressions for full outer join and right outer join Sub-task Resolved Pengcheng Xiong
          115.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix multiple window spec for PTF operator Sub-task Closed Pengcheng Xiong
          116.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): correct table column name in CTAS queries Sub-task Resolved Pengcheng Xiong
          117.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix udaf_percentile_approx_23.q Sub-task Closed Pengcheng Xiong
          118.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): deal with hive default partition when inserting data Sub-task Closed Ashutosh Chauhan
          119.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix TestJdbcDriver2 Sub-task Resolved Pengcheng Xiong
          120.
          CBO (Calcite Return Path): groupby_grouping_id2.q returns wrong results Sub-task Open Vineet Garg
          121.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : columnPruner prunes everything when union is the last operator before FS Sub-task Closed Pengcheng Xiong
          122.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path) : dealing with '$' in typeInfo Sub-task Closed Pengcheng Xiong
          123.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix test failure for udf_percentile.q Sub-task Resolved Pengcheng Xiong
          124.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): UDAF can not pull up constant expressions Sub-task Closed Pengcheng Xiong
          125.
          CBO (Calcite Return Path): Support for unique join Sub-task Resolved Jesus Camacho Rodriguez
          126.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): fix test failure for groupby_resolution.q Sub-task Resolved Laljo John Pullokkaran
          127.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver.vector* queries failures due to NPE in Vectorizer.onExpressionHasNullSafes() Sub-task Closed Hari Sankar Sivarama Subramaniyan
          128.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver.vector_join_filters.q failure Sub-task Closed Hari Sankar Sivarama Subramaniyan
          129.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver count.q failure Sub-task Resolved Vineet Garg
          130.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver skewjoin.q failure Sub-task Closed Hari Sankar Sivarama Subramaniyan
          131.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): MiniTezCliDriver vector_auto_smb_mapjoin_14.q failure Sub-task Patch Available Vineet Garg
          132.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_grouping_sets4.q failure Sub-task Patch Available Hari Sankar Sivarama Subramaniyan
          133.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver groupby_ppr_multi_distinct.q failure Sub-task Resolved Vineet Garg
          134.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): RexNode convert(ExprNodeConstantDesc literal) decimal support bug Sub-task Closed Hari Sankar Sivarama Subramaniyan
          135.
          CBO: Calcite Operator To Hive Operator (Calcite Return Path): TestCliDriver count.q failure Sub-task Closed Vineet Garg
          136.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Wrong result/plan in group by with hive.map.aggr=false Sub-task Resolved Vineet Garg
          137.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Fix test failure for auto_join_filters Sub-task Resolved Vineet Garg
          138.
          CBO (Calcite Return Path) Wrong results for limit + offset Sub-task Resolved Ashutosh Chauhan
          139.
          CBO (Calcite Return Path) Handle field access for nested column Sub-task Resolved Ashutosh Chauhan
          140.
          CBO (Calcite Return Path) Incorrect result set when limit is present in one of the union branches Sub-task Closed Ashutosh Chauhan
          141.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Fix wrong result in join_filters_overlap Sub-task Open Vineet Garg
          142.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Fix wrong result in input30 Sub-task Resolved Vineet Garg
          143.
          CBO: Calcite Operator To Hive Operator(Calcite Return Path): Support Tablesample clause Sub-task Open Vineet Garg
          144.
          Create view doesn't work with Calcite Return path Sub-task Open Unassigned
          145.
          Join fails on operator return path on CBO Sub-task Open Unassigned

            Activity

              People

              • Assignee:
                jcamachorodriguez Jesus Camacho Rodriguez
                Reporter:
                jpullokkaran Laljo John Pullokkaran
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: