Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4614

test_aggregation.py fails with IllegalStateException on Kudu table

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Frontend
    • Labels:

      Description

      First failure is an IllegalStateException on a Kudu table:

      02:07:54 =================================== FAILURES ===================================
      02:07:54  TestAggregationQueries.test_aggregation[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: kudu/none] 
      02:07:54 [gw2] linux2 -- Python 2.7.5 /data/jenkins/workspace/impala-cdh5-2.7.0_5.10.0-exhaustive-rhel7/repos/Impala/bin/../infra/python/env/bin/python
      02:07:54 query_test/test_aggregation.py:202: in test_aggregation
      02:07:54     self.run_test_case('QueryTest/aggregation', vector)
      02:07:54 common/impala_test_suite.py:324: in run_test_case
      02:07:54     result = self.__execute_query(target_impalad_client, query, user=user)
      02:07:54 common/impala_test_suite.py:532: in __execute_query
      02:07:54     return impalad_client.execute(query, user=user)
      02:07:54 common/impala_connection.py:160: in execute
      02:07:54     return self.__beeswax_client.execute(sql_stmt, user=user)
      02:07:54 beeswax/impala_beeswax.py:173: in execute
      02:07:54     handle = self.__execute_query(query_string.strip(), user=user)
      02:07:54 beeswax/impala_beeswax.py:337: in __execute_query
      02:07:54     handle = self.execute_query_async(query_string, user=user)
      02:07:54 beeswax/impala_beeswax.py:333: in execute_query_async
      02:07:54     return self.__do_rpc(lambda: self.imp_service.query(query,))
      02:07:54 beeswax/impala_beeswax.py:458: in __do_rpc
      02:07:54     raise ImpalaBeeswaxException(self.__build_error_message(b), b)
      02:07:54 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      02:07:54 E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
      02:07:54 E    MESSAGE: IllegalStateException: functional_kudu.alltypesagg_idx.timestamp_col < TIMESTAMP '2010-01-01 01:05:20'
      02:07:54 ----------------------------- Captured stderr call -----------------------------
      02:07:54 -- executing against localhost:21000
      02:07:54 use functional_kudu;
      02:07:54 
      02:07:54 SET disable_codegen=False;
      02:07:54 SET abort_on_error=1;
      02:07:54 SET exec_single_node_rows_threshold=0;
      02:07:54 SET batch_size=0;
      02:07:54 SET num_nodes=0;
      02:07:54 -- executing against localhost:21000
      02:07:54 SELECT abs(cast(variance(tinyint_col) as double) - 6.66741) < 0.001,
      02:07:54  abs(cast(variance(double_col) as double) - 8470806.708) < 0.001
      02:07:54 from alltypesagg;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 SELECT variance(tinyint_col), stddev(smallint_col), variance_pop(int_col),
      02:07:54 stddev_pop(bigint_col)
      02:07:54 from alltypesagg WHERE id = -9999999;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 SELECT variance(tinyint_col), stddev(smallint_col), variance_pop(int_col),
      02:07:54 stddev_pop(bigint_col)
      02:07:54 from alltypesagg WHERE id = 1006;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 SELECT variance(tinyint_col), variance(smallint_col), variance(int_col),
      02:07:54 variance(bigint_col), variance(float_col), variance(double_col),
      02:07:54 var_samp(double_col), variance_samp(double_col)
      02:07:54 from alltypesagg WHERE id >= 1000 AND id < 1006;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 SELECT variance_pop(tinyint_col), variance_pop(smallint_col), variance_pop(int_col),
      02:07:54 variance_pop(bigint_col), variance_pop(float_col), variance_pop(double_col),
      02:07:54 var_pop(double_col)
      02:07:54 from alltypesagg WHERE id >= 1000 AND id < 1006;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 SELECT round(stddev(tinyint_col), 5),
      02:07:54        round(stddev(smallint_col), 5),
      02:07:54        round(stddev(int_col), 5),
      02:07:54        round(stddev(bigint_col), 5),
      02:07:54        round(stddev(float_col), 5),
      02:07:54        round(stddev(double_col), 5),
      02:07:54        round(stddev_samp(double_col), 5)
      02:07:54 from alltypesagg WHERE id >= 1000 AND id < 1006;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 SELECT round(stddev_pop(tinyint_col), 5),
      02:07:54        round(stddev_pop(smallint_col), 5),
      02:07:54        round(stddev_pop(int_col), 5),
      02:07:54        round(stddev_pop(bigint_col), 5),
      02:07:54        round(stddev_pop(float_col), 5),
      02:07:54        round(stddev_pop(double_col), 5)
      02:07:54 from alltypesagg WHERE id >= 1000 AND id < 1006;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(bool_col), min(bool_col), max(bool_col)
      02:07:54 from alltypesagg where day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), count(tinyint_col), min(tinyint_col), max(tinyint_col), sum(tinyint_col),
      02:07:54 avg(tinyint_col)
      02:07:54 from alltypesagg where day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), count(smallint_col), min(smallint_col), max(smallint_col), sum(smallint_col),
      02:07:54 avg(smallint_col)
      02:07:54 from alltypesagg where day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), count(int_col), min(int_col), max(int_col), sum(int_col), avg(int_col)
      02:07:54 from alltypesagg where day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), count(bigint_col), min(bigint_col), max(bigint_col), sum(bigint_col),
      02:07:54 avg(bigint_col)
      02:07:54 from alltypesagg where day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), count(float_col), min(float_col), max(float_col), sum(float_col),
      02:07:54 avg(float_col)
      02:07:54 from alltypesagg where day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), count(double_col), min(double_col), max(double_col), round(sum(double_col), 0),
      02:07:54 round(avg(double_col), 0)
      02:07:54 from alltypesagg where day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), min(string_col), max(string_col), min(date_string_col),
      02:07:54 max(date_string_col)
      02:07:54 from alltypesagg where day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select max(str), min(str) from (values ('aaa' as str), (''), ('123')) as tmp;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select last_value(b) over (partition by a order by d) from functional.nulltable;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select first_value(b) over (partition by a order by d) from functional.nulltable;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select tinyint_col, count(*) from alltypesagg where day is not null group by 1 order by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select bool_col,min(bool_col),max(bool_col) from alltypesagg where day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*) from alltypesagg where day is not null group by tinyint_col;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select smallint_col % 10, count(*) from alltypesagg where day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*) from alltypesagg where day is not null group by smallint_col % 10;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 10, count(*) from alltypesagg where day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*) from alltypesagg where day is not null group by int_col % 10;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(ALL *) from alltypesagg where day is not null group by int_col % 10;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select bigint_col % 100, count(*) from alltypesagg where day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*) from alltypesagg where day is not null group by bigint_col % 100;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select float_col, float_col * 2, count(*) from alltypes group by 1, 2;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*) from alltypes group by float_col;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select float_col, count(*) from alltypesagg where float_col is null and day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select double_col, double_col * 2, count(*) from alltypes group by 1, 2;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*) from alltypes group by double_col;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select double_col, count(*) from alltypesagg where double_col is null and day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select date_string_col, count(*) from alltypesagg where day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*) from alltypesagg where day is not null group by date_string_col;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select tinyint_col % 3, smallint_col % 3, count(*) from alltypesagg
      02:07:54 where day = 1 group by 1, 2;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*) from alltypesagg
      02:07:54 where day = 1 group by tinyint_col % 3, smallint_col % 3;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select tinyint_col % 3, smallint_col % 3, count(*) from alltypesagg where day = 1 group by 2, 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select tinyint_col % 2, smallint_col % 2, int_col % 2, bigint_col % 2, date_string_col, count(*)
      02:07:54 from alltypesagg
      02:07:54 where (date_string_col = '01/01/10' or date_string_col = '01/02/10') and day is not null
      02:07:54 group by 1, 2, 3, 4, 5;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*)
      02:07:54 from alltypesagg
      02:07:54 where (date_string_col = '01/01/10' or date_string_col = '01/02/10') and day is not null
      02:07:54 group by tinyint_col % 2, smallint_col % 2, int_col % 2, bigint_col % 2, date_string_col;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), min(tinyint_col), max(tinyint_col), sum(tinyint_col), avg(tinyint_col)
      02:07:54 from alltypesagg
      02:07:54 where tinyint_col = -1 and day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), min(smallint_col), max(smallint_col), sum(smallint_col), avg(smallint_col)
      02:07:54 from alltypesagg
      02:07:54 where smallint_col = -1 and day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), min(int_col), max(int_col), sum(int_col), avg(int_col)
      02:07:54 from alltypesagg
      02:07:54 where int_col = -1 and day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), min(bigint_col), max(bigint_col), sum(bigint_col), avg(bigint_col)
      02:07:54 from alltypesagg
      02:07:54 where bigint_col = -1 and day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), min(float_col), max(float_col), sum(float_col), avg(float_col)
      02:07:54 from alltypesagg
      02:07:54 where float_col < -1.0 and day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select count(*), min(double_col), max(double_col), sum(double_col), avg(double_col)
      02:07:54 from alltypesagg
      02:07:54 where double_col < -1.0 and day is not null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*), max(int_col) from alltypesagg where day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*) from alltypesagg where day is not null group by 1 having max(int_col) > 991;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*) from alltypesagg where day is not null group by 1
      02:07:54 having max(int_col) > 991 and count(*) > 1420;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*) from alltypesagg where day is not null group by 1
      02:07:54 having min(int_col) < 7;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*) from alltypesagg where day is not null group by 1
      02:07:54 having min(int_col) < 7 and count(*) > 1420;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*), sum(int_col) from alltypesagg where day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*), sum(int_col) from alltypesagg where day is not null group by 1
      02:07:54 having sum(int_col) >= 715000;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*), sum(int_col) from alltypesagg where day is not null group by 1
      02:07:54 having sum(int_col) >= 715000 or count(*) > 1420;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*), sum(int_col) from alltypesagg where day is not null group by 1
      02:07:54 having sum(int_col) is null;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*), avg(int_col) from alltypesagg where day is not null group by 1;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*), avg(int_col) from alltypesagg where day is not null group by 1
      02:07:54 having avg(int_col) > 500;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select int_col % 7, count(*), avg(int_col) from alltypesagg where day is not null group by 1
      02:07:54 having avg(int_col) > 500 or count(*) = 10;
      02:07:54 
      02:07:54 -- executing against localhost:21000
      02:07:54 select timestamp_col, count(*) from alltypesagg where day is not null
      02:07:54 group by timestamp_col having timestamp_col < cast('2010-01-01 01:05:20' as timestamp);
      02:07:54 
      02:07:54  TestAggregationQueries.test_aggregation[exec_option: {'disable_codegen': True, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: kudu/none] 
      02:07:54 [gw2] linux2 -- Python 2.7.5 /data/jenkins/workspace/impala-cdh5-2.7.0_5.10.0-exhaustive-rhel7/repos/Impala/bin/../infra/python/env/bin/python
      02:07:54 query_test/test_aggregation.py:202: in test_aggregation
      02:07:54     self.run_test_case('QueryTest/aggregation', vector)
      02:07:54 common/impala_test_suite.py:324: in run_test_case
      02:07:54     result = self.__execute_query(target_impalad_client, query, user=user)
      02:07:54 common/impala_test_suite.py:532: in __execute_query
      02:07:54     return impalad_client.execute(query, user=user)
      02:07:54 common/impala_connection.py:160: in execute
      02:07:54     return self.__beeswax_client.execute(sql_stmt, user=user)
      02:07:54 beeswax/impala_beeswax.py:173: in execute
      02:07:54     handle = self.__execute_query(query_string.strip(), user=user)
      02:07:54 beeswax/impala_beeswax.py:337: in __execute_query
      02:07:54     handle = self.execute_query_async(query_string, user=user)
      02:07:54 beeswax/impala_beeswax.py:333: in execute_query_async
      02:07:54     return self.__do_rpc(lambda: self.imp_service.query(query,))
      02:07:54 beeswax/impala_beeswax.py:458: in __do_rpc
      02:07:54     raise ImpalaBeeswaxException(self.__build_error_message(b), b)
      02:07:54 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      02:07:54 E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
      02:07:54 E    MESSAGE: IllegalStateException: functional_kudu.alltypesagg_idx.timestamp_col < TIMESTAMP '2010-01-01 01:05:20'
      02:07:54 ----------------------------- Captured stderr call -----------------------------
      

        Attachments

          Activity

            People

            • Assignee:
              alex.behm Alexander Behm
              Reporter:
              alex.behm Alexander Behm
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: