Hive
  1. Hive
  2. HIVE-3108

SELECT count(DISTINCT col) ... returns 0 if "col" is a partition column

    Details

    • Type: Bug Bug
    • Status: Reopened
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.8.0, 0.9.0
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
    • Environment:

      Mac OSX running Apache distribution of hadoop and hive natively.

      Description

      Suppose "stocks" is a managed OR external table, partitioned by "exchange" and "symbol". "count(DISTINCT x)" returns 0 for either "exchange", "symbol", or both:

      hive> SELECT count(DISTINCT exchange), count(DISTINCT symbol) from stocks;
      0 0

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          5d 3h 29m 1 Arun A K 14/Jun/12 06:11
          Patch Available Patch Available Open Open
          1h 32m 1 Carl Steinbach 14/Jun/12 07:44
          Open Open Resolved Resolved
          13d 20h 6m 1 Edward Capriolo 28/Jun/12 03:50
          Resolved Resolved Reopened Reopened
          1170d 5h 36m 1 Mikko Kivistö 11/Sep/15 09:27
          Hide
          Mikko Kivistö added a comment -

          I found the root cause: TBLPROPERTIES ('skip.header.line.count'='1') was specified in the table and that causes the answer not to return.

          Show
          Mikko Kivistö added a comment - I found the root cause: TBLPROPERTIES ('skip.header.line.count'='1') was specified in the table and that causes the answer not to return.
          Hide
          Mikko Kivistö added a comment -

          Ok, problems seems to relate when using a SerDe to read the data

          Show
          Mikko Kivistö added a comment - Ok, problems seems to relate when using a SerDe to read the data
          Mikko Kivistö made changes -
          Resolution Duplicate [ 3 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Mikko Kivistö added a comment -

          Doesn't work on Hive 0.13 or 0.14 (1.0) thus reopening this issue

          Show
          Mikko Kivistö added a comment - Doesn't work on Hive 0.13 or 0.14 (1.0) thus reopening this issue
          Pengcheng Xiong made changes -
          Link This issue relates to HIVE-7817 [ HIVE-7817 ]
          Edward Capriolo made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]
          Hide
          Edward Capriolo added a comment -

          Confirmed fixed in trunk.

          [edward@tablitha dist]$ bin/hive
          Logging initialized using configuration in jar:file:/home/edward/hive/trunk/build/dist/lib/hive-common-0.10.0-SNAPSHOT.jar!/hive-log4j.properties
          Hive history file=/tmp/edward/hive_job_log_edward_201206272349_386020253.txt
          hive> create table stocks (x int, y string) partitioned by (exchange string, symbol string);
          OK
          Time taken: 17.382 seconds
          hive> alter table stocks add partition (exchange='nasdaq', symbol='ed');
          OK
          Time taken: 2.022 seconds
          hive> alter table stocks add partition (exchange='nasdaq', symbol='guy');
          OK
          Time taken: 0.219 seconds
          hive> alter table stocks add partition (exchange='jp', symbol='bla');    
          OK
          Time taken: 0.245 seconds
          hive> select count(distinct exchange), count(distinct symbol) from stocks;
          2	3
          Time taken: 5.742 seconds
          
          Show
          Edward Capriolo added a comment - Confirmed fixed in trunk. [edward@tablitha dist]$ bin/hive Logging initialized using configuration in jar:file:/home/edward/hive/trunk/build/dist/lib/hive-common-0.10.0-SNAPSHOT.jar!/hive-log4j.properties Hive history file=/tmp/edward/hive_job_log_edward_201206272349_386020253.txt hive> create table stocks (x int, y string) partitioned by (exchange string, symbol string); OK Time taken: 17.382 seconds hive> alter table stocks add partition (exchange='nasdaq', symbol='ed'); OK Time taken: 2.022 seconds hive> alter table stocks add partition (exchange='nasdaq', symbol='guy'); OK Time taken: 0.219 seconds hive> alter table stocks add partition (exchange='jp', symbol='bla'); OK Time taken: 0.245 seconds hive> select count(distinct exchange), count(distinct symbol) from stocks; 2 3 Time taken: 5.742 seconds
          Hide
          Edward Capriolo added a comment -

          I am guessing he things that HIVE-2955 is patch available and is the same issue as this one. If that is that case we should just close this as a duplicate.

          Show
          Edward Capriolo added a comment - I am guessing he things that HIVE-2955 is patch available and is the same issue as this one. If that is that case we should just close this as a duplicate.
          Carl Steinbach made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Carl Steinbach added a comment -

          @Arun: Doesn't look like there's a patch here. Not sure why you changed the status to Patch Available.

          Show
          Carl Steinbach added a comment - @Arun: Doesn't look like there's a patch here. Not sure why you changed the status to Patch Available.
          Arun A K made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Arun A K made changes -
          Priority Trivial [ 5 ] Major [ 3 ]
          Arun A K made changes -
          Priority Major [ 3 ] Trivial [ 5 ]
          Arun A K made changes -
          Labels Hive
          Navis made changes -
          Field Original Value New Value
          Link This issue duplicates HIVE-2955 [ HIVE-2955 ]
          Dean Wampler created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Dean Wampler
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development