Hive
  1. Hive
  2. HIVE-582

"SELECT count(1) as cnt from src x" not working for column pruning

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The query "SELECT count(1) as cnt from src x" has some problems, even though nullgroup*.q passed in tests.
      Will check and solve it in this jira.
      A wrong plan sometimes can be generated due to some unknow factor:
      Wrong plan:

      STAGE PLANS:
        Stage: Stage-1
          Map Reduce
            Alias -> Map Operator Tree:
              x 
                  Select Operator
                    Reduce Output Operator
                      sort order: 
                      tag: -1
            Reduce Operator Tree:
              Group By Operator
                aggregations:
                      expr: count(VALUE._col0)
                mode: complete
                Select Operator
                  expressions:
                        expr: _col0
                        type: bigint
                  File Output Operator
                    compressed: false
                    GlobalTableId: 0
                    table:
                        input format: org.apache.hadoop.mapred.TextInputFormat
                        output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
      
        Stage: Stage-0
          Fetch Operator
            limit: -1
      

      Without unknow factor, correct plan can be generated:

      STAGE PLANS:
        Stage: Stage-1
          Map Reduce
            Alias -> Map Operator Tree:
              x 
                  Select Operator
                    Group By Operator
                      aggregations:
                            expr: count(1)
                      mode: hash
                      Reduce Output Operator
                        sort order: 
                        tag: -1
                        value expressions:
                              expr: _col0
                              type: bigint
            Reduce Operator Tree:
              Group By Operator
                aggregations:
                      expr: count(VALUE._col0)
                mode: mergepartial
                Select Operator
                  expressions:
                        expr: _col0
                        type: bigint
                  File Output Operator
                    compressed: false
                    GlobalTableId: 0
                    table:
                        input format: org.apache.hadoop.mapred.TextInputFormat
                        output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
      

        Activity

        He Yongqiang created issue -
        He Yongqiang made changes -
        Field Original Value New Value
        Attachment hive-582-2009-06-25.patch [ 12411811 ]
        Zheng Shao made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.4.0 [ 12313714 ]
        Resolution Fixed [ 1 ]
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            He Yongqiang
            Reporter:
            He Yongqiang
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development