Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6107

Error adding columns to Kudu table tbl_with_defaults: The column already exists: i

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Cannot Reproduce
    • Impala 2.11.0
    • None
    • Infrastructure

    Description

      22:47:12 FAIL query_test/test_kudu.py::TestKuduOperations::()::test_kudu_insert[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none]
22:47:12 =================================== FAILURES ===================================
22:47:12  TestKuduOperations.test_kudu_insert[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none] 
22:47:12 [gw2] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
22:47:12 query_test/test_kudu.py:84: in test_kudu_insert
22:47:12     self.run_test_case('QueryTest/kudu_insert', vector, use_db=unique_database)
22:47:12 common/impala_test_suite.py:391: in run_test_case
22:47:12     result = self.__execute_query(target_impalad_client, query, user=user)
22:47:12 common/impala_test_suite.py:600: in __execute_query
22:47:12     return impalad_client.execute(query, user=user)
22:47:12 common/impala_connection.py:160: in execute
22:47:12     return self.__beeswax_client.execute(sql_stmt, user=user)
22:47:12 beeswax/impala_beeswax.py:173: in execute
22:47:12     handle = self.__execute_query(query_string.strip(), user=user)
22:47:12 beeswax/impala_beeswax.py:339: in __execute_query
22:47:12     handle = self.execute_query_async(query_string, user=user)
22:47:12 beeswax/impala_beeswax.py:335: in execute_query_async
22:47:12     return self.__do_rpc(lambda: self.imp_service.query(query,))
22:47:12 beeswax/impala_beeswax.py:460: in __do_rpc
22:47:12     raise ImpalaBeeswaxException(self.__build_error_message(b), b)
22:47:12 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
22:47:12 E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
22:47:12 E    MESSAGE: ImpalaRuntimeException: Error adding columns to Kudu table tbl_with_defaults
22:47:12 E   CAUSED BY: NonRecoverableException: The column already exists: i
22:47:12 ---------------------------- Captured stderr setup -----------------------------
22:47:12 SET sync_ddl=False;
22:47:12 -- executing against localhost:21000
22:47:12 DROP DATABASE IF EXISTS `test_kudu_insert_70eff904` CASCADE;
22:47:12 
22:47:12 SET sync_ddl=False;
22:47:12 -- executing against localhost:21000
22:47:12 CREATE DATABASE `test_kudu_insert_70eff904`;
22:47:12 
22:47:12 MainThread: Created database "test_kudu_insert_70eff904" for test ID "query_test/test_kudu.py::TestKuduOperations::()::test_kudu_insert[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none]"
22:47:12 ----------------------------- Captured stderr call -----------------------------
22:47:12 -- executing against localhost:21000
22:47:12 use test_kudu_insert_70eff904;
22:47:12 
22:47:12 SET batch_size=0;
22:47:12 SET num_nodes=0;
22:47:12 SET disable_codegen_rows_threshold=0;
22:47:12 SET disable_codegen=False;
22:47:12 SET abort_on_error=1;
22:47:12 SET exec_single_node_rows_threshold=0;
22:47:12 -- executing against localhost:21000
22:47:12 create table tdata
22:47:12   (id int primary key, valf float null, vali bigint null, valv string null,
22:47:12    valb boolean null, valt tinyint null, vals smallint null, vald double null,
22:47:12    ts timestamp)
22:47:12   PARTITION BY RANGE (PARTITION VALUES < 10, PARTITION 10 <= VALUES < 30,
22:47:12   PARTITION 30 <= VALUES) STORED AS KUDU;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata values (1, 1, 1, 'one', true, 1, 1, 1,
22:47:12   cast('1987-05-19 00:00:00' as timestamp));
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata values (2, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata values
22:47:12 (3, cast('nan' as float), max_bigint(), '', true, min_tinyint(), max_smallint(),
22:47:12   cast('-inf' as double),
22:47:12   nanoseconds_add(cast('9999-12-31 23:59:59' as timestamp), 999999999));
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata (valb, vald, id) values (true, 0, 4);
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata values
22:47:12 (5, 5.0, 5, 'five', false, NULL, NULL, NULL, NULL),
22:47:12 (6, 16, 60, '', true, 0, -1, -6, cast('2010-12-31 23:59:59' as timestamp)),
22:47:12 (7, NULL, 10, NULL, false, max_tinyint(), -7, 2, cast('1400-01-01 00:00:00' as timestamp));
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata (valv, valf, vali, id) values
22:47:12 ('eight', 88, 888, 8),
22:47:12 (NULL, -9, -99, 9);
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata
22:47:12 select id, float_col, bigint_col, string_col, bool_col, tinyint_col, smallint_col,
22:47:12 double_col, timestamp_col
22:47:12 from functional.alltypes where id = 10;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata (id, vald, valb, vali, ts)
22:47:12 select id, double_col, bool_col, bigint_col, timestamp_col
22:47:12 from functional.alltypes where id = 11;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 delete tdata;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata
22:47:12 select id, float_col, bigint_col, string_col, bool_col, tinyint_col, smallint_col,
22:47:12 double_col, timestamp_col
22:47:12 from functional.alltypes where id < 2;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata (vals, id, valt, vald, ts)
22:47:12 select smallint_col, id, tinyint_col, double_col, timestamp_col
22:47:12 from functional.alltypes where id > 2 and id < 6;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tdata limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata values (320, 2.0, 932, cast('' as string), false, 0, 0, 0, NULL);
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select id, valv, valb from tdata where id = 320;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata values
22:47:12 (666, cast(1.2 as float), 43, cast('z' as string), true, 0, 0, 0, NULL);
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tdata values
22:47:12 (666, cast(1.2 as float), 43, cast('z' as VARCHAR(20)), true, 0, 0, 0, NULL);
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 create table kudu_test_tbl primary key(id)
22:47:12 partition by range(id) (partition values < 100, partition 100 <= values <= 10000)
22:47:12 stored as kudu as
22:47:12 select * from functional_kudu.alltypes where id < 100;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into kudu_test_tbl
22:47:12 select * from functional_kudu.alltypes where id < 100;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into kudu_test_tbl
22:47:12 select * from functional_kudu.alltypes;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into kudu_test_tbl SELECT cast(id + 10000 as int), bool_col, tinyint_col,
22:47:12   smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, string_col,
22:47:12   timestamp_col, year, month
22:47:12 from functional_kudu.alltypes;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 create table allkeytypes (i1 tinyint, i2 smallint, i3 int, i4 bigint, name string,
22:47:12   valt timestamp, valf float, vald double, primary key (i1, i2, i3, i4, name, valt))
22:47:12   partition by hash partitions 3, range
22:47:12   (partition value = (1,1,1,1,'1','2009-01-01 00:01:00'),
22:47:12   partition value = (2,2,2,2,'2','2009-01-01 00:02:00.100000000'),
22:47:12   partition value = (3,3,3,3,'3','2009-01-01 00:03:00.300000000')) stored as kudu;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into allkeytypes select cast(id as tinyint), smallint_col, int_col,
22:47:12   cast (bigint_col/10 as bigint), string_col, timestamp_col, float_col, double_col
22:47:12   from functional.alltypes where id > 1 and id < 10;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 set exec_single_node_rows_threshold=0;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 
22:47:12 insert into allkeytypes values (1,1,1,1,'1','2009-01-01 00:01:00',null,null);
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 SET EXEC_SINGLE_NODE_ROWS_THRESHOLD="100";
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 create table tbl_with_defaults (a int primary key, b int null default 10,
22:47:12   c int not null default 100, d int default 1000, e int null, f int not null,
22:47:12   g string default 'test', h boolean default true) partition by hash (a)
22:47:12   partitions 3 stored as kudu;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tbl_with_defaults (a, f) values (1, 1), (2, 2), (3, 3), (4, 4);
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tbl_with_defaults limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 insert into tbl_with_defaults values (5, 5, 5, 5, 5, 5, 'row', false);
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 select * from tbl_with_defaults limit 1000;
22:47:12 
22:47:12 -- executing against localhost:21000
22:47:12 alter table tbl_with_defaults add columns (i int null, j int not null default 10000);
22:47:12 
22:47:12 ====== 1 failed, 1545 passed, 49 skipped, 36 xfailed in 14509.55 seconds =======

      The HEAD commit was: "IMPALA-4524: Batch ALTER TABLE...ADD PARTITION calls."

      I've seen this on ASAN and DEBUG builds.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-7245 test_kudu.py::test_kudu_insert() fails with "Error adding columns to Kudu table tbl_with_defaults"

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              twmarshall Thomas Tauber-Marshall
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: