Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4584

test_kudu_alter_table fails on exhaustive

    Details

      Description

      Jenkins console snippet:

      16:35:13 =================================== FAILURES ===================================
      16:35:13  TestKuduOperations.test_kudu_alter_table[exec_option: {'disable_codegen': True, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
      16:35:13 query_test/test_kudu.py:63: in test_kudu_alter_table
      16:35:13     self.run_test_case('QueryTest/kudu_alter', vector, use_db=unique_database)
      16:35:13 common/impala_test_suite.py:324: in run_test_case
      16:35:13     result = self.__execute_query(target_impalad_client, query, user=user)
      16:35:13 common/impala_test_suite.py:532: in __execute_query
      16:35:13     return impalad_client.execute(query, user=user)
      16:35:13 common/impala_connection.py:160: in execute
      16:35:13     return self.__beeswax_client.execute(sql_stmt, user=user)
      16:35:13 beeswax/impala_beeswax.py:173: in execute
      16:35:13     handle = self.__execute_query(query_string.strip(), user=user)
      16:35:13 beeswax/impala_beeswax.py:337: in __execute_query
      16:35:13     handle = self.execute_query_async(query_string, user=user)
      16:35:13 beeswax/impala_beeswax.py:333: in execute_query_async
      16:35:13     return self.__do_rpc(lambda: self.imp_service.query(query,))
      16:35:13 beeswax/impala_beeswax.py:458: in __do_rpc
      16:35:13     raise ImpalaBeeswaxException(self.__build_error_message(b), b)
      16:35:13 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      16:35:13 E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
      16:35:13 E    MESSAGE: AnalysisException: Target table 'test_kudu_alter_table_8df0adc5.tbl_to_alter' has fewer columns (3) than the SELECT / VALUES clause returns (5)
      16:35:13 ---------------------------- Captured stderr setup -----------------------------
      16:35:13 -- connecting to: localhost:21000
      16:35:13 SET sync_ddl=False;
      16:35:13 -- executing against localhost:21000
      16:35:13 DROP DATABASE IF EXISTS `test_kudu_alter_table_8df0adc5` CASCADE;
      16:35:13 
      16:35:13 SET sync_ddl=False;
      16:35:13 -- executing against localhost:21000
      16:35:13 CREATE DATABASE `test_kudu_alter_table_8df0adc5`;
      16:35:13 
      16:35:13 MainThread: Created database "test_kudu_alter_table_8df0adc5" for test ID "query_test/test_kudu.py::TestKuduOperations::()::test_kudu_alter_table[exec_option: {'disable_codegen': True, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none]"
      16:35:13 ----------------------------- Captured stderr call -----------------------------
      16:35:13 -- executing against localhost:21000
      16:35:13 use test_kudu_alter_table_8df0adc5;
      16:35:13 
      16:35:13 SET disable_codegen=True;
      16:35:13 SET abort_on_error=1;
      16:35:13 SET exec_single_node_rows_threshold=0;
      16:35:13 SET batch_size=0;
      16:35:13 SET num_nodes=0;
      16:35:13 -- executing against localhost:21000
      16:35:13 create table simple (id int primary key, name string, valf float, vali bigint)
      16:35:13   distribute by hash (id) into 3 buckets stored as kudu;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table simple set tblproperties (
      16:35:13   'kudu.master_addresses' = 'localhost'
      16:35:13 );
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 describe formatted simple;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table simple set tblproperties ('kudu.master_addresses' = '127.0.0.1');
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table simple set tblproperties ('kudu.master_addresses' = 'invalid_host');
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table simple rename to simple_new;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 select count(*) from simple_new;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 create table tbl_to_alter (id int primary key, name string null, vali bigint not null)
      16:35:13   distribute by range (id) (partition 1 < values <= 10) stored as kudu
      16:35:13   tblproperties('kudu.table_name'='tbl_to_alter');
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table tbl_to_alter add range partition 10 < values <= 20;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 insert into tbl_to_alter values (15, 'name', 100);
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 select * from tbl_to_alter limit 1000;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table tbl_to_alter add range partition value = 100;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 insert into tbl_to_alter values (100, 'name1', 1000);
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 select * from tbl_to_alter limit 1000;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table tbl_to_alter add range partition 1000 < values;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table tbl_to_alter add range partition 10 < values <= 30;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table tbl_to_alter add if not exists range partition 10 < values <= 30;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table tbl_to_alter drop range partition value = 100;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 select * from tbl_to_alter;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table tbl_to_alter drop range partition 10 < values <= 20;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table tbl_to_alter drop range partition 1 < values <= 10;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 
      16:35:13 alter table tbl_to_alter drop range partition 1000 < values;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 select count(*), count(id) from tbl_to_alter
      16:35:13   where id = 1 and cast(sin(id) as boolean) = true;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 insert into tbl_to_alter values (1, 'name', 100);
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 alter table tbl_to_alter add range partition 1 < values <= 20;
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 
      16:35:13 alter table tbl_to_alter add columns (new_col1 int not null default 10,
      16:35:13   new_col2 bigint not null default 1000);
      16:35:13 
      16:35:13 -- executing against localhost:21000
      16:35:13 insert into tbl_to_alter values (2, 'test', 100, 1, 100);
      16:35:13 
      16:35:13  generated xml file: /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/logs/ee_tests/results/TEST-impala-serial.xml 
      16:35:13 =========================== short test summary info ============================
      

        Activity

        Hide
        mjacobs Matthew Jacobs added a comment -

        The test fails after 5:53:52 when an INSERT .. SELECT query gets the wrong metadata, it expects 5 columns which should be there but only the original 3 are there, as if the add columns operation weren't successful.

        The issue is that the catalog calls KuduClient.alterTable() which runs asyncronously. The doc for the function states:

           * When the method returns it only indicates that the master accepted the alter
           * command, use {@link KuduClient#isAlterTableDone(String)} to know when the alter finishes.
           * @param name the table's name, if this is a table rename then the old table name must be passed
        
        Show
        mjacobs Matthew Jacobs added a comment - The test fails after 5:53:52 when an INSERT .. SELECT query gets the wrong metadata, it expects 5 columns which should be there but only the original 3 are there, as if the add columns operation weren't successful. The issue is that the catalog calls KuduClient.alterTable() which runs asyncronously. The doc for the function states: * When the method returns it only indicates that the master accepted the alter * command, use {@link KuduClient#isAlterTableDone( String )} to know when the alter finishes. * @param name the table's name, if this is a table rename then the old table name must be passed
        Hide
        dtsirogiannis Dimitris Tsirogiannis added a comment -

        Change-Id: I385bce66691ae9040e72f97557e1bba31009e36b
        Reviewed-on: http://gerrit.cloudera.org:8080/5364
        Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
        Tested-by: Internal Jenkins

        Let's reopen if the issue is not resolved.

        Show
        dtsirogiannis Dimitris Tsirogiannis added a comment - Change-Id: I385bce66691ae9040e72f97557e1bba31009e36b Reviewed-on: http://gerrit.cloudera.org:8080/5364 Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com> Tested-by: Internal Jenkins Let's reopen if the issue is not resolved.

          People

          • Assignee:
            dtsirogiannis Dimitris Tsirogiannis
            Reporter:
            alex.behm Alexander Behm
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development