Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6094

TestDdlStatements::test_alter_table failed on S3 with FileNotFoundException: No such file or directory

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Information Provided
    • Impala 2.11.0
    • None
    • Infrastructure

    Description

      10:39:27 FAIL metadata/test_ddl.py::TestDdlStatements::()::test_alter_table[exec_option: {'sync_ddl': 0, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none-unique_database0]
      10:39:27 =================================== FAILURES ===================================
      10:39:27  TestDdlStatements.test_alter_table[exec_option: {'sync_ddl': 0, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none-unique_database0] 
      10:39:27 [gw1] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test-s3/repos/Impala/bin/../infra/python/env/bin/python
      10:39:27 metadata/test_ddl.py:258: in test_alter_table
      10:39:27     multiple_impalad=self._use_multiple_impalad(vector))
      10:39:27 common/impala_test_suite.py:391: in run_test_case
      10:39:27     result = self.__execute_query(target_impalad_client, query, user=user)
      10:39:27 common/impala_test_suite.py:600: in __execute_query
      10:39:27     return impalad_client.execute(query, user=user)
      10:39:27 common/impala_connection.py:160: in execute
      10:39:27     return self.__beeswax_client.execute(sql_stmt, user=user)
      10:39:27 beeswax/impala_beeswax.py:173: in execute
      10:39:27     handle = self.__execute_query(query_string.strip(), user=user)
      10:39:27 beeswax/impala_beeswax.py:341: in __execute_query
      10:39:27     self.wait_for_completion(handle)
      10:39:27 beeswax/impala_beeswax.py:361: in wait_for_completion
      10:39:27     raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
      10:39:27 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      10:39:27 E    Query aborted:Disk I/O error: Failed to open HDFS file s3a://xxx/test-warehouse/test_alter_table_5bf55c1.db/i1670A_alter_1/x3/e549b60eaa4acde4-c9e71baa00000000_1764541588_data.0.
      10:39:27 E   Error(2): No such file or directory
      10:39:27 E   Root cause: FileNotFoundException: No such file or directory: s3a://xxx/test-warehouse/test_alter_table_5bf55c1.db/i1670A_alter_1/x3/e549b60eaa4acde4-c9e71baa00000000_1764541588_data.0.
      10:39:27 ---------------------------- Captured stderr setup -----------------------------
      10:39:27 SET sync_ddl=True;
      10:39:27 -- executing against localhost:21000
      10:39:27 DROP DATABASE IF EXISTS `test_alter_table_5bf55c1` CASCADE;
      10:39:27 
      10:39:27 SET sync_ddl=True;
      10:39:27 -- executing against localhost:21000
      10:39:27 CREATE DATABASE `test_alter_table_5bf55c1`;
      10:39:27 
      10:39:27 MainThread: Created database "test_alter_table_5bf55c1" for test ID "metadata/test_ddl.py::TestDdlStatements::()::test_alter_table[exec_option: {'sync_ddl': 0, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none-unique_database0]"
      10:39:27 SET sync_ddl=True;
      10:39:27 -- executing against localhost:21000
      10:39:27 DROP DATABASE IF EXISTS `test_alter_table_5bf55c12` CASCADE;
      10:39:27 
      10:39:27 SET sync_ddl=True;
      10:39:27 -- executing against localhost:21000
      10:39:27 CREATE DATABASE `test_alter_table_5bf55c12`;
      10:39:27 
      10:39:27 MainThread: Created database "test_alter_table_5bf55c12" for test ID "metadata/test_ddl.py::TestDdlStatements::()::test_alter_table[exec_option: {'sync_ddl': 0, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none-unique_database0]"
      10:39:27 ----------------------------- Captured stderr call -----------------------------
      10:39:27 -- executing against localhost:21000
      10:39:27 create table test_alter_table_5bf55c1.part_data (i int);
      10:39:27 
      10:39:27 MainThread: Starting new HTTPS connection (1): impala-cdh5-trunk.s3.amazonaws.com
      10:39:27 -- executing against localhost:21000
      10:39:27 use test_alter_table_5bf55c1;
      10:39:27 
      10:39:27 SET sync_ddl=0;
      10:39:27 SET batch_size=0;
      10:39:27 SET num_nodes=0;
      10:39:27 SET disable_codegen_rows_threshold=0;
      10:39:27 SET disable_codegen=False;
      10:39:27 SET abort_on_error=False;
      10:39:27 SET exec_single_node_rows_threshold=0;
      10:39:27 -- executing against localhost:21000
      10:39:27 create external table t1 (i int)
      10:39:27 location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/t1_tmp1';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 describe t1;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t1 add columns (t tinyint, s string comment 'Str Col');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 describe t1;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t1 rename to t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 rename to test_alter_table_5bf55c12.t1_inotherdb;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables in test_alter_table_5bf55c12;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table test_alter_table_5bf55c12.t1_inotherdb rename to t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 describe t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 drop column t;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 describe t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 replace columns (c1 bigint comment 'id col', c2 string, c3 int);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 describe t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 insert overwrite table t2 select 1, '50', 2 from functional.alltypes limit 2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select * from t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 change column c2 int_col int comment 'changed to int col';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 change column c1 id_col bigint;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 change column c3 c3 int comment 'added a comment';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 describe t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select * from t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 add columns (
      10:39:27 x array<int>,
      10:39:27 y map<string,float> comment 'Map Col',
      10:39:27 z struct<f1:boolean,f2:bigint>);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 describe t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 replace columns (
      10:39:27 a int comment 'Int Col',
      10:39:27 b struct<f1:array<int>,f2:map<string,struct<f1:bigint>>>,
      10:39:27 c double);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 describe t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 change column c c array<struct<f1:decimal(10,2)>>;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 describe t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 change column c c varchar(20);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 describe t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t2 alter column a set comment 'alter comment';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 describe t2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create external table jointbl_test like functional.jointbl;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select * from jointbl_test;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table jointbl_test set location 's3a://impala-cdh5-trunk/test-warehouse/jointbl';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 show table stats jointbl_test;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select * from jointbl_test order by test_id limit 3;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table jointbl_test set location 's3a://impala-cdh5-trunk/test-warehouse/jointbl_seq';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 show table stats jointbl_test;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table jointbl_test set fileformat sequencefile;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select * from jointbl_test order by test_id limit 3;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create external table t_part (i int) partitioned by (j int, s string)
      10:39:27 location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/t_part_tmp';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table t_part add partition (j=cast(2-1 as int), s='2012');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table t_part add if not exists partition (j=1, s='2012');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table t_part add if not exists partition (j=1, s='2012/withslash');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table t_part add partition (j=1, s=substring('foo2013bar', 4, 8));
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part add partition (j=100, s='same_location')
      10:39:27 location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/t_part_tmp/j=1/s=2012';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part add partition (j=101, s='different_part_dir')
      10:39:27 location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/part_data/';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 insert overwrite table t_part partition(j=1, s='2012') select 2 from functional.alltypes limit 2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 insert overwrite table t_part partition(j=1, s='2013') select 3 from functional.alltypes limit 3;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 insert overwrite table t_part partition(j=1, s='2012/withslash')
      10:39:27 select 1 from functional.alltypes limit 1;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 refresh t_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part add partition (j=NULL, s='2013');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part add partition (j=NULL, s=NULL);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part drop partition (j=100, s='same_location');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part drop partition (j=101, s='different_part_dir');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 insert overwrite table t_part partition(j=NULL, s=NULL)
      10:39:27 select 4 from functional.alltypes limit 5;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 insert overwrite table t_part partition(j=NULL, s='2013')
      10:39:27 select 5 from functional.alltypes limit 5;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part drop partition (j=NULL, s=NULL);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part drop partition (j=NULL, s=trim('  2013  '));
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part drop partition (j=1, s='2013');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part drop partition (j=1, s='2012/withslash');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 insert into table t_part partition(j=2, s='')
      10:39:27 select 1 from functional.alltypes limit 1;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 insert into table t_part partition(j=2, s)
      10:39:27 select 10, '' from functional.alltypes limit 1;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part where s is NULL;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part drop partition (j=2, s='');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part where s is NULL;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table t_part rename to t_part2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables like 't_part*';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i, j, s from t_part2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create external table alltypes_test like functional.alltypes;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table alltypes_test add partition(month=4, year=2009);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table alltypes_test add partition(month=5, year=2009);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 select int_col from alltypes_test;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table alltypes_test partition(month=4, year=2009)
      10:39:27 set location 's3a://impala-cdh5-trunk/test-warehouse/alltypes_seq_snap/year=2009/month=4';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table alltypes_test partition(month=4, year=2009)
      10:39:27 set fileformat sequencefile;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select int_col, count(*) from alltypes_test
      10:39:27 group by int_col order by 1 limit 100;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table alltypes_test partition(month=cast(1+4 as int), year=cast(100*20+9 as int))
      10:39:27 set location 's3a://impala-cdh5-trunk/test-warehouse/alltypes_rc/year=2009/month=5';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table alltypes_test partition(month=cast(2+3 as int), year=2009)
      10:39:27 set fileformat rcfile;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select int_col, count(*) from alltypes_test
      10:39:27 group by int_col order by 1  limit 100;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show table stats alltypes_test;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table alltypes_test set tblproperties ('numRows'='200');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table alltypes_test partition(year=2009, month=4)
      10:39:27 set tblproperties ('numRows'='30', 'STATS_GENERATED_VIA_STATS_TASK'='true');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show table stats alltypes_test;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 CREATE TABLE imp1016 (string1 string);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 INSERT INTO imp1016 VALUES ('test');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 ALTER TABLE imp1016 ADD COLUMNS (string2 string);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 DESCRIBE imp1016;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 SELECT * FROM imp1016;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 SELECT string1 FROM imp1016;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 SELECT string2 FROM imp1016;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 SELECT COUNT(DISTINCT string1) FROM imp1016;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 SELECT COUNT(DISTINCT string2) FROM imp1016;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 CREATE TABLE imp1016Large (string1 string);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 INSERT INTO imp1016Large SELECT 'test' FROM functional.alltypes LIMIT 2000;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 ALTER TABLE imp1016Large ADD COLUMNS (string2 string);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 DESCRIBE imp1016Large;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 SELECT COUNT(string2) FROM imp1016Large;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 SELECT COUNT(*), COUNT(DISTINCT string1) FROM imp1016Large;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 SELECT COUNT(*), COUNT(DISTINCT string2) FROM imp1016Large;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table test_alter_table_5bf55c1.mv (x int, y string);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into test_alter_table_5bf55c1.mv values(1, 'a'), (2, 'b'), (NULL, NULL);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 compute stats test_alter_table_5bf55c1.mv;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table test_alter_table_5bf55c1.mv rename to test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 invalidate metadata test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables in test_alter_table_5bf55c1 like '*mv*';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables in test_alter_table_5bf55c12 like '*mv*';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show table stats test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show column stats test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 drop table test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables in test_alter_table_5bf55c12 like '*mv*';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table test_alter_table_5bf55c1.mv (x int) partitioned by (y string);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into test_alter_table_5bf55c1.mv partition(y='a') values(1);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into test_alter_table_5bf55c1.mv partition(y='b') values(2);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into test_alter_table_5bf55c1.mv partition(y=NULL) values(NULL);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 compute stats test_alter_table_5bf55c1.mv;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table test_alter_table_5bf55c1.mv rename to test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 invalidate metadata test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables in test_alter_table_5bf55c1 like '*mv*';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables in test_alter_table_5bf55c12 like '*mv*';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show table stats test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show column stats test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 drop table test_alter_table_5bf55c12.mv2;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show tables in test_alter_table_5bf55c12 like '*mv*';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table test_alter_table_5bf55c1.property_length (i int);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table test_alter_table_5bf55c1.property_length
      10:39:27 set serdeproperties(
      10:39:27 'keykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeyk'
      10:39:27 =
      10:39:27 'valuevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevalue'
      10:39:27 );
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 alter table test_alter_table_5bf55c1.property_length
      10:39:27 set tblproperties(
      10:39:27 'keykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeykeyk'
      10:39:27 =
      10:39:27 'valuevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevaluevalue'
      10:39:27 );
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 -- Change table location after a partition has been created in the original table location
      10:39:27 create table test_alter_table_5bf55c1.change_loc (s string) partitioned by (i int);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into test_alter_table_5bf55c1.change_loc partition (i=1) values ('a');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table test_alter_table_5bf55c1.change_loc
      10:39:27   set location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/garbage';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 select * from test_alter_table_5bf55c1.change_loc;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 -- ALTER TABLE .. SET LOCATION works when setting the location of a partition to be the
      10:39:27 -- the same as a partition of a different table with a different number of clustering
      10:39:27 -- columns.
      10:39:27 create table test_alter_table_5bf55c1.table_one_part (s string) partitioned by (i int);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 create table test_alter_table_5bf55c1.table_many_part (s string) partitioned by
      10:39:27   (i0 int, i1 int, i2 int, i3 int, i4 int, i5 int, i6 int, i7 int, i8 int, i9 int);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into test_alter_table_5bf55c1.table_one_part partition (i=0) values ('a');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into test_alter_table_5bf55c1.table_one_part partition (i=1) values ('b');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into test_alter_table_5bf55c1.table_many_part partition
      10:39:27   (i0=0, i1=1, i2=2, i3=3, i4=4, i5=5, i6=6, i7=7, i8=8, i9=9) values ('c');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into test_alter_table_5bf55c1.table_many_part partition
      10:39:27   (i0=10, i1=11, i2=12, i3=13, i4=14, i5=15, i6=16, i7=17, i8=18, i9=19) values ('d');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table test_alter_table_5bf55c1.table_one_part partition (i=1)
      10:39:27   set location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/table_many_part/i0=10/i1=11/i2=12/i3=13/i4=14/i5=15/i6=16/i7=17/i8=18/i9=19';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table test_alter_table_5bf55c1.table_many_part partition
      10:39:27   (i0=10, i1=11, i2=12, i3=13, i4=14, i5=15, i6=16, i7=17, i8=18, i9=19)
      10:39:27   set location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/table_one_part/i=0/';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 select i,s from test_alter_table_5bf55c1.table_one_part order by i;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, s
      10:39:27 from test_alter_table_5bf55c1.table_many_part order by i0;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show partitions test_alter_table_5bf55c1.table_one_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show partitions test_alter_table_5bf55c1.table_many_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 -- Refreshing the tables does not create invalid partition locations
      10:39:27 refresh test_alter_table_5bf55c1.table_one_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 refresh test_alter_table_5bf55c1.table_many_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 select i,s from test_alter_table_5bf55c1.table_one_part order by i;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, s
      10:39:27 from test_alter_table_5bf55c1.table_many_part order by i0;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show partitions test_alter_table_5bf55c1.table_one_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show partitions test_alter_table_5bf55c1.table_many_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 -- Altering the table location does not alter the partition locations
      10:39:27 alter table test_alter_table_5bf55c1.table_one_part set location 's3a://impala-cdh5-trunk/garbage/';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table test_alter_table_5bf55c1.table_many_part set location 's3a://impala-cdh5-trunk/garbage/';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 select i,s from test_alter_table_5bf55c1.table_one_part order by i;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select i0, i1, i2, i3, i4, i5, i6, i7, i8, i9, s
      10:39:27 from test_alter_table_5bf55c1.table_many_part order by i0;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show partitions test_alter_table_5bf55c1.table_one_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show partitions test_alter_table_5bf55c1.table_many_part;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table i1740_alter_1 (i1 integer);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table i1740_alter_1 set tblproperties ('skip.header.line.count'='2');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table i1740_alter_2 (i1 integer);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table i1740_alter_2 set tblproperties ('skip.header.line.count'='-1');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table i1740_alter_3 (i1 integer);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table i1740_alter_3 set tblproperties ('skip.header.line.count'='foo');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table i1740_alter_4 (i1 integer) stored as parquet;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table i1740_alter_4 set tblproperties ('skip.header.line.count'='2');
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table i4155_alter (c1 int)
      10:39:27 partitioned by (p1 string) row format delimited fields terminated by ',';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table i4155_alter set fileformat PARQUET;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into i4155_alter partition(p1="abc") values (1);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 select * from i4155_alter;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table i1670A_alter (s string) partitioned by (i integer);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table i1670A_alter add
      10:39:27 partition (i=1) location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/i1670A_alter/i1'
      10:39:27 partition (i=2) location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/i1670A_alter/i2'
      10:39:27 partition (i=3) uncached;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 show partitions i1670A_alter;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 create table i1670A_alter_1 (a int) partitioned by (x int);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 alter table i1670A_alter_1 add partition (x=1)
      10:39:27 partition (x=2) uncached
      10:39:27 partition (x=3) location 's3a://impala-cdh5-trunk/test-warehouse/test_alter_table_5bf55c1.db/i1670A_alter_1/x3';
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into i1670A_alter_1 partition(x=1) values (1), (2), (3);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into i1670A_alter_1 partition(x=2) values (1), (2), (3), (4);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 insert into i1670A_alter_1 partition(x=3) values (1);
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 
      10:39:27 invalidate metadata i1670A_alter_1;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 show partitions i1670A_alter_1;
      10:39:27 
      10:39:27 -- executing against localhost:21000
      10:39:27 select x, a from i1670A_alter_1 order by x, a;
      10:39:27 
      10:39:27 ======= 1 failed, 1418 passed, 130 skipped, 4 xfailed in 9154.88 seconds =======
      

      Attachments

        Activity

          People

            sailesh Sailesh Mukil
            tarmstrong Tim Armstrong
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: