Hive
  1. Hive
  2. HIVE-3108

SELECT count(DISTINCT col) ... returns 0 if "col" is a partition column

    Details

    • Type: Bug Bug
    • Status: Reopened
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.8.0, 0.9.0
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
    • Environment:

      Mac OSX running Apache distribution of hadoop and hive natively.

      Description

      Suppose "stocks" is a managed OR external table, partitioned by "exchange" and "symbol". "count(DISTINCT x)" returns 0 for either "exchange", "symbol", or both:

      hive> SELECT count(DISTINCT exchange), count(DISTINCT symbol) from stocks;
      0 0

        Issue Links

          Activity

          Dean Wampler created issue -
          Navis made changes -
          Field Original Value New Value
          Link This issue duplicates HIVE-2955 [ HIVE-2955 ]
          Arun A K made changes -
          Labels Hive
          Arun A K made changes -
          Priority Major [ 3 ] Trivial [ 5 ]
          Arun A K made changes -
          Priority Trivial [ 5 ] Major [ 3 ]
          Arun A K made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Carl Steinbach added a comment -

          @Arun: Doesn't look like there's a patch here. Not sure why you changed the status to Patch Available.

          Show
          Carl Steinbach added a comment - @Arun: Doesn't look like there's a patch here. Not sure why you changed the status to Patch Available.
          Carl Steinbach made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Edward Capriolo added a comment -

          I am guessing he things that HIVE-2955 is patch available and is the same issue as this one. If that is that case we should just close this as a duplicate.

          Show
          Edward Capriolo added a comment - I am guessing he things that HIVE-2955 is patch available and is the same issue as this one. If that is that case we should just close this as a duplicate.
          Hide
          Edward Capriolo added a comment -

          Confirmed fixed in trunk.

          [edward@tablitha dist]$ bin/hive
          Logging initialized using configuration in jar:file:/home/edward/hive/trunk/build/dist/lib/hive-common-0.10.0-SNAPSHOT.jar!/hive-log4j.properties
          Hive history file=/tmp/edward/hive_job_log_edward_201206272349_386020253.txt
          hive> create table stocks (x int, y string) partitioned by (exchange string, symbol string);
          OK
          Time taken: 17.382 seconds
          hive> alter table stocks add partition (exchange='nasdaq', symbol='ed');
          OK
          Time taken: 2.022 seconds
          hive> alter table stocks add partition (exchange='nasdaq', symbol='guy');
          OK
          Time taken: 0.219 seconds
          hive> alter table stocks add partition (exchange='jp', symbol='bla');    
          OK
          Time taken: 0.245 seconds
          hive> select count(distinct exchange), count(distinct symbol) from stocks;
          2	3
          Time taken: 5.742 seconds
          
          Show
          Edward Capriolo added a comment - Confirmed fixed in trunk. [edward@tablitha dist]$ bin/hive Logging initialized using configuration in jar:file:/home/edward/hive/trunk/build/dist/lib/hive-common-0.10.0-SNAPSHOT.jar!/hive-log4j.properties Hive history file=/tmp/edward/hive_job_log_edward_201206272349_386020253.txt hive> create table stocks (x int, y string) partitioned by (exchange string, symbol string); OK Time taken: 17.382 seconds hive> alter table stocks add partition (exchange='nasdaq', symbol='ed'); OK Time taken: 2.022 seconds hive> alter table stocks add partition (exchange='nasdaq', symbol='guy'); OK Time taken: 0.219 seconds hive> alter table stocks add partition (exchange='jp', symbol='bla'); OK Time taken: 0.245 seconds hive> select count(distinct exchange), count(distinct symbol) from stocks; 2 3 Time taken: 5.742 seconds
          Edward Capriolo made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]
          Pengcheng Xiong made changes -
          Link This issue relates to HIVE-7817 [ HIVE-7817 ]
          Hide
          Mikko Kivistö added a comment -

          Doesn't work on Hive 0.13 or 0.14 (1.0) thus reopening this issue

          Show
          Mikko Kivistö added a comment - Doesn't work on Hive 0.13 or 0.14 (1.0) thus reopening this issue
          Mikko Kivistö made changes -
          Resolution Duplicate [ 3 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Mikko Kivistö added a comment -

          Ok, problems seems to relate when using a SerDe to read the data

          Show
          Mikko Kivistö added a comment - Ok, problems seems to relate when using a SerDe to read the data
          Hide
          Mikko Kivistö added a comment -

          I found the root cause: TBLPROPERTIES ('skip.header.line.count'='1') was specified in the table and that causes the answer not to return.

          Show
          Mikko Kivistö added a comment - I found the root cause: TBLPROPERTIES ('skip.header.line.count'='1') was specified in the table and that causes the answer not to return.
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          5d 3h 29m 1 Arun A K 14/Jun/12 06:11
          Patch Available Patch Available Open Open
          1h 32m 1 Carl Steinbach 14/Jun/12 07:44
          Open Open Resolved Resolved
          13d 20h 6m 1 Edward Capriolo 28/Jun/12 03:50
          Resolved Resolved Reopened Reopened
          1170d 5h 36m 1 Mikko Kivistö 11/Sep/15 09:27

            People

            • Assignee:
              Unassigned
              Reporter:
              Dean Wampler
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development