Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4628

test_kudu_alter_table fails with "Invalid alter step type"

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Backend
    • Labels:

      Description

      Jenkins console log:

      07:10:46 =================================== FAILURES ===================================
      07:10:46  TestKuduOperations.test_kudu_alter_table[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
      07:10:46 query_test/test_kudu.py:63: in test_kudu_alter_table
      07:10:46     self.run_test_case('QueryTest/kudu_alter', vector, use_db=unique_database)
      07:10:46 common/impala_test_suite.py:324: in run_test_case
      07:10:46     result = self.__execute_query(target_impalad_client, query, user=user)
      07:10:46 common/impala_test_suite.py:532: in __execute_query
      07:10:46     return impalad_client.execute(query, user=user)
      07:10:46 common/impala_connection.py:160: in execute
      07:10:46     return self.__beeswax_client.execute(sql_stmt, user=user)
      07:10:46 beeswax/impala_beeswax.py:173: in execute
      07:10:46     handle = self.__execute_query(query_string.strip(), user=user)
      07:10:46 beeswax/impala_beeswax.py:337: in __execute_query
      07:10:46     handle = self.execute_query_async(query_string, user=user)
      07:10:46 beeswax/impala_beeswax.py:333: in execute_query_async
      07:10:46     return self.__do_rpc(lambda: self.imp_service.query(query,))
      07:10:46 beeswax/impala_beeswax.py:458: in __do_rpc
      07:10:46     raise ImpalaBeeswaxException(self.__build_error_message(b), b)
      07:10:46 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      07:10:46 E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
      07:10:46 E    MESSAGE: 
      07:10:46 E   ImpalaRuntimeException: Error renaming column new_col3 to last_name for Kudu table tbl_to_alter
      07:10:46 E   CAUSED BY: NonRecoverableException: Invalid alter step type: type: ALTER_COLUMN 7 { 1 { 1: "new_col3" 2: "last_name" } }
      07:10:46 ---------------------------- Captured stderr setup -----------------------------
      07:10:46 -- connecting to: localhost:21000
      07:10:46 SET sync_ddl=False;
      07:10:46 -- executing against localhost:21000
      07:10:46 DROP DATABASE IF EXISTS `test_kudu_alter_table_ddaaa05e` CASCADE;
      07:10:46 
      07:10:46 SET sync_ddl=False;
      07:10:46 -- executing against localhost:21000
      07:10:46 CREATE DATABASE `test_kudu_alter_table_ddaaa05e`;
      07:10:46 
      07:10:46 MainThread: Created database "test_kudu_alter_table_ddaaa05e" for test ID "query_test/test_kudu.py::TestKuduOperations::()::test_kudu_alter_table[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none]"
      07:10:46 ----------------------------- Captured stderr call -----------------------------
      07:10:46 -- executing against localhost:21000
      07:10:46 use test_kudu_alter_table_ddaaa05e;
      07:10:46 
      07:10:46 SET disable_codegen=False;
      07:10:46 SET abort_on_error=1;
      07:10:46 SET exec_single_node_rows_threshold=0;
      07:10:46 SET batch_size=0;
      07:10:46 SET num_nodes=0;
      07:10:46 -- executing against localhost:21000
      07:10:46 create table simple (id int primary key, name string, valf float, vali bigint)
      07:10:46   partition by hash (id) partitions 3 stored as kudu;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table simple set tblproperties (
      07:10:46   'kudu.master_addresses' = 'localhost'
      07:10:46 );
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 describe formatted simple;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table simple set tblproperties ('kudu.master_addresses' = '127.0.0.1');
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table simple set tblproperties ('kudu.master_addresses' = 'invalid_host');
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table simple rename to simple_new;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select count(*) from simple_new;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 create table tbl_to_alter (id int primary key, name string null, vali bigint not null)
      07:10:46   partition by range (id) (partition 1 < values <= 10) stored as kudu;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter add range partition 10 < values <= 20;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 insert into tbl_to_alter values (15, 'name', 100);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select * from tbl_to_alter limit 1000;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter add range partition value = 100;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 insert into tbl_to_alter values (100, 'name1', 1000);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select * from tbl_to_alter limit 1000;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter add range partition 1000 < values;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter add range partition 10 < values <= 30;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter add if not exists range partition 10 < values <= 30;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter drop range partition value = 100;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select * from tbl_to_alter;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter drop range partition 10 < values <= 20;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter drop range partition 1 < values <= 10;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 
      07:10:46 alter table tbl_to_alter drop range partition 1000 < values;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select count(*), count(id) from tbl_to_alter
      07:10:46   where id = 1 and cast(sin(id) as boolean) = true;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 insert into tbl_to_alter values (1, 'name', 100);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter add range partition 1 < values <= 20;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 
      07:10:46 alter table tbl_to_alter add columns (new_col1 int not null default 10,
      07:10:46   new_col2 bigint not null default 1000);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 insert into tbl_to_alter values (2, 'test', 100, 1, 100);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select * from tbl_to_alter limit 1000;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 insert into tbl_to_alter (id,name,vali) values (3, 'test', 200);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select * from tbl_to_alter limit 1000;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 insert into tbl_to_alter values (9, 'test', 300, null, null);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select * from tbl_to_alter limit 1000;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter add columns (new_col3 string null);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 insert into tbl_to_alter values ((4, 'test', 300, 1, 100, null),
      07:10:46   (5, 'test', 400, 2, 200, 'names'));
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select * from tbl_to_alter limit 1000;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 insert into tbl_to_alter (id, name, vali, new_col1, new_col2)
      07:10:46   values (6, 'test', 500, 3, 300);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select * from tbl_to_alter limit 1000;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter add columns (invalid_col int null default 10);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter add columns (invalid_col int not null);
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter drop column vali;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 select * from tbl_to_alter;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter drop column id;
      07:10:46 
      07:10:46 -- executing against localhost:21000
      07:10:46 alter table tbl_to_alter change column new_col3 last_name string;
      07:10:46 
      07:10:46  generated xml file: /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/logs/ee_tests/results/TEST-impala-serial.xml 
      07:10:46 =========================== short test summary info ============================
      

        Activity

        Hide
        lv Lars Volker added a comment -

        This was resolved by three steps:

        1) Disable the broken tests in this commit

        IMPALA-4628: Disable broken kudu test to unblock GVOs

        Change-Id: I30d45acb26eb3e709a1994a89e8444ca9530d8cc
        Reviewed-on: http://gerrit.cloudera.org:8080/5428
        Reviewed-by: Alex Behm <alex.behm@cloudera.com>
        Tested-by: Alex Behm <alex.behm@cloudera.com>

        2) Wait for fixed JARs to be published by Kudu project

        3) Re-enable test and bump Kudu version in this commit

        Bump Kudu server version to latest master (a70c905006)

        This also re-enabled kudu_alter.test, which was disabled in IMPALA-4628.

        Change-Id: Ie5acdeffea7ed9a68ce0f48d1f68c6c922044704
        Reviewed-on: http://gerrit.cloudera.org:8080/5427
        Reviewed-by: Matthew Jacobs <mj@cloudera.com>
        Tested-by: Internal Jenkins

        Show
        lv Lars Volker added a comment - This was resolved by three steps: 1) Disable the broken tests in this commit IMPALA-4628 : Disable broken kudu test to unblock GVOs Change-Id: I30d45acb26eb3e709a1994a89e8444ca9530d8cc Reviewed-on: http://gerrit.cloudera.org:8080/5428 Reviewed-by: Alex Behm <alex.behm@cloudera.com> Tested-by: Alex Behm <alex.behm@cloudera.com> 2) Wait for fixed JARs to be published by Kudu project 3) Re-enable test and bump Kudu version in this commit Bump Kudu server version to latest master (a70c905006) This also re-enabled kudu_alter.test, which was disabled in IMPALA-4628 . Change-Id: Ie5acdeffea7ed9a68ce0f48d1f68c6c922044704 Reviewed-on: http://gerrit.cloudera.org:8080/5427 Reviewed-by: Matthew Jacobs <mj@cloudera.com> Tested-by: Internal Jenkins
        Hide
        jbapple Jim Apple added a comment -

        This is a bulk comment on all issues with Fix Version 2.8.0 that were resolved on or after 2016-12-09.

        2.8.0 was branched on December 9, with only two changes to master cherry-picked to the 2.8.0 release branch after that:

        https://github.com/apache/incubator-impala/commits/2.8.0

        Issues fixed after December 9 might not be fixed in 2.8.0. If you are the one who marked this issue Resolved, can you check to see if the patch is in 2.8.0 by using the link above? If the patch is not in 2.8.0, can you change the Fix Version to 2.9.0?

        Thank you!

        Show
        jbapple Jim Apple added a comment - This is a bulk comment on all issues with Fix Version 2.8.0 that were resolved on or after 2016-12-09. 2.8.0 was branched on December 9, with only two changes to master cherry-picked to the 2.8.0 release branch after that: https://github.com/apache/incubator-impala/commits/2.8.0 Issues fixed after December 9 might not be fixed in 2.8.0. If you are the one who marked this issue Resolved, can you check to see if the patch is in 2.8.0 by using the link above? If the patch is not in 2.8.0, can you change the Fix Version to 2.9.0? Thank you!
        Hide
        lv Lars Volker added a comment -

        I checked that both changes needed to fix this issue are included in 2.8.0.

        Show
        lv Lars Volker added a comment - I checked that both changes needed to fix this issue are included in 2.8.0.

          People

          • Assignee:
            lv Lars Volker
            Reporter:
            alex.behm Alexander Behm
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development