[IMPALA-6092] Flaky test: query_test/test_udfs.py still happening

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.11.0
• Fix Version/s: Impala 3.0, Impala 2.12.0
• Component/s: Infrastructure
• Labels: flaky
• Target Version: Impala 2.11.0
• Epic Color: ghx-label-4
• Assignee: Vuk Ercegovac
• Reporter: Tim Armstrong
• Created: 20/Oct/17 19:02
• Updated: 12/Apr/18 17:08
• Resolved: 12/Apr/18 17:05

Description

I'm seeing another failure that looks the same as IMPALA-5760. The fix for IMPALA-5760 was to revert an earlier commit, but I'm seeing this on a commit with that change reverted.

I actually just saw this failure on a commit with IMPALA-4795 reverted:

commit 2187d36b36f5b2fc4318e57265cc31ee11d22423
Author: Jim Apple <[e-mail]>
Date:   Wed Oct 11 13:20:49 2017 -0700

    IMPALA-6045: Make build scripts more friendly to Ubuntu 16.04

I'm going to open a new JIRA.

query_test.test_udfs.TestUdfExecution.test_java_udfs[exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'exec_single_node_rows_threshold': 100, 'enable_expr_rewrites': False} | table_format: text/none] (from pytest)
Failing for the past 1 build (Since Failed#493 )
Took 11 sec.
add description
Error Message

query_test/test_udfs.py:321: in test_java_udfs     self.run_test_case('QueryTest/java-udf', vector, use_db=unique_database) common/impala_test_suite.py:391: in run_test_case     result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:600: in __execute_query     return impalad_client.execute(query, user=user) common/impala_connection.py:160: in execute     return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:173: in execute     handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:341: in __execute_query     self.wait_for_completion(handle) beeswax/impala_beeswax.py:361: in wait_for_completion     raise ImpalaBeeswaxException("Query aborted:" + error_log, None) E   ImpalaBeeswaxException: ImpalaBeeswaxException: E    Query aborted:ImpalaRuntimeException: Unable to find class. E   CAUSED BY: ClassNotFoundException: org.apache.impala.TestUdf

Stacktrace

query_test/test_udfs.py:321: in test_java_udfs
    self.run_test_case('QueryTest/java-udf', vector, use_db=unique_database)
common/impala_test_suite.py:391: in run_test_case
    result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:600: in __execute_query
    return impalad_client.execute(query, user=user)
common/impala_connection.py:160: in execute
    return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:173: in execute
    handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:341: in __execute_query
    self.wait_for_completion(handle)
beeswax/impala_beeswax.py:361: in wait_for_completion
    raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E   ImpalaBeeswaxException: ImpalaBeeswaxException:
E    Query aborted:ImpalaRuntimeException: Unable to find class.
E   CAUSED BY: ClassNotFoundException: org.apache.impala.TestUdf

Standard Error

-- connecting to: localhost:21000
SET sync_ddl=False;
-- executing against localhost:21000
DROP DATABASE IF EXISTS `test_java_udfs_df1420f` CASCADE;

SET sync_ddl=False;
-- executing against localhost:21000
CREATE DATABASE `test_java_udfs_df1420f`;

MainThread: Created database "test_java_udfs_df1420f" for test ID "query_test/test_udfs.py::TestUdfExecution::()::test_java_udfs[exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'exec_single_node_rows_threshold': 100, 'enable_expr_rewrites': False} | table_format: text/none]"
-- executing against localhost:21000
use test_java_udfs_df1420f;

SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET exec_single_node_rows_threshold=100;
SET enable_expr_rewrites=False;
-- executing against localhost:21000
create function hive_pi() returns double
location '/test-warehouse/hive-exec.jar'
symbol='org.apache.hadoop.hive.ql.udf.UDFPI';

-- executing against localhost:21000


create function hive_round(double) returns double
location '/test-warehouse/hive-exec.jar'
symbol='org.apache.hadoop.hive.ql.udf.UDFRound';

-- executing against localhost:21000


create function hive_floor(double) returns bigint
location '/test-warehouse/hive-exec.jar'
symbol='org.apache.hadoop.hive.ql.udf.UDFFloor';

-- executing against localhost:21000


create function hive_mod(int, int) returns int
location '/test-warehouse/hive-exec.jar'
symbol='org.apache.hadoop.hive.ql.udf.UDFPosMod';

-- executing against localhost:21000


create function hive_bin(bigint) returns string
location '/test-warehouse/hive-exec.jar'
symbol='org.apache.hadoop.hive.ql.udf.UDFBin';

-- executing against localhost:21000


create function hive_lower(string) returns string
location '/test-warehouse/hive-exec.jar'
symbol='org.apache.hadoop.hive.ql.udf.UDFLower';

-- executing against localhost:21000


create function identity_anytype
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(boolean) returns boolean
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(tinyint) returns tinyint
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(smallint) returns smallint
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(int) returns int
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(bigint) returns bigint
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(float) returns float
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(double) returns double
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(string) returns string
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(string, string) returns string
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function identity(string, string, string) returns string
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function hive_add(int, int) returns int
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function hive_add(smallint, smallint) returns smallint
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function hive_add(float, float) returns float
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function hive_add(double, double) returns double
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function hive_add(boolean, boolean) returns boolean
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdf';

-- executing against localhost:21000


create function throws_exception() returns boolean
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.TestUdfException';

-- executing against localhost:21000


create function replace_string(string) returns string
location '/test-warehouse/impala-hive-udfs.jar'
symbol='org.apache.impala.ReplaceStringUdf';

-- executing against localhost:21000
use test_java_udfs_df1420f;

SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET exec_single_node_rows_threshold=100;
SET enable_expr_rewrites=False;
-- executing against localhost:21000
select hive_pi();

-- executing against localhost:21000
select hive_bin(100);

-- executing against localhost:21000
select min(hive_pi()) from functional.alltypesagg;

-- executing against localhost:21000
select identity(true), identity(cast(NULL as boolean));

-- executing against localhost:21000
select identity(cast(10 as tinyint)), identity(cast(NULL as tinyint));

-- executing against localhost:21000
select identity(cast(10 as smallint)), identity(cast(NULL as smallint));

-- executing against localhost:21000
select identity(cast(10 as int)), identity(cast(NULL as int));

-- executing against localhost:21000
select identity(cast(10 as bigint)), identity(cast(NULL as bigint));

-- executing against localhost:21000
select identity(cast(10.0 as float)), identity(cast(NULL as float));

-- executing against localhost:21000
select identity(cast(10.0 as double)), identity(cast(NULL as double));

-- executing against localhost:21000
select identity("why hello there"),
       identity("why", " hello there"),
       identity("why", " hello", " there"),
       identity(cast(NULL as string)),
       identity(cast(NULL as string), cast(NULL as string)),
       identity(cast(NULL as string), cast(NULL as string), cast(NULL as string));

-- executing against localhost:21000
select length(identity("0123456789")),
       length(identity("0123456789", "0123456789")),
       length(identity("0123456789", "0123456789", "0123456789"));

-- executing against localhost:21000
select throws_exception();

-- executing against localhost:21000
select throws_exception() from functional.alltypestiny;

-- executing against localhost:21000
select hive_add(cast(1 as int), cast(2 as int));

-- executing against localhost:21000
select hive_add(hive_add(cast(1 as int), cast(2 as int)), cast(2 as int));

-- executing against localhost:21000
select hive_add(cast(hive_add(cast(1 as int), cast(2 as int)) - hive_add(cast(2 as int), cast(1 as int)) as int), cast(2 as int));

-- executing against localhost:21000
select hive_add(cast(1 as smallint), cast(2 as smallint));

-- executing against localhost:21000
select hive_add(cast(1.0 as float), cast(2.0 as float));

-- executing against localhost:21000
select hive_add(cast(1.0 as double), cast(2.0 as double));

-- executing against localhost:21000
select hive_add(cast(1 as boolean), cast(0 as boolean));

-- executing against localhost:21000
select identity_anytype(true);

-- executing against localhost:21000
select identity_anytype(cast(10 as tinyint));

-- executing against localhost:21000
select identity_anytype(cast(10 as smallint));

-- executing against localhost:21000
select identity_anytype(cast(10 as int));

-- executing against localhost:21000
select identity_anytype(cast(10 as bigint));

-- executing against localhost:21000
select identity_anytype(cast(10.0 as float));

-- executing against localhost:21000
select identity_anytype(cast(10.0 as double));

-- executing against localhost:21000
select identity_anytype("a", "b");

-- executing against localhost:21000
select identity_anytype("a", "b", "c");

-- executing against localhost:21000
select * from
(select max(int_col) from functional.alltypesagg
 where identity(bool_col) union all
(select max(int_col) from functional.alltypesagg
 where identity(tinyint_col) > 1 union all
(select max(int_col) from functional.alltypesagg
 where identity(smallint_col) > 1 union all
(select max(int_col) from functional.alltypesagg
 where identity(int_col) > 1 union all
(select max(int_col) from functional.alltypesagg
 where identity(bigint_col) > 1 union all
(select max(int_col) from functional.alltypesagg
 where identity(float_col) > 1.0 union all
(select max(int_col) from functional.alltypesagg
 where identity(double_col) > 1.0 union all
(select max(int_col) from functional.alltypesagg
 where identity(string_col) > '1' union all
(select max(int_col) from functional.alltypesagg
 where not identity(bool_col) union all
(select max(int_col) from functional.alltypesagg
 where identity(tinyint_col) > 2 union all
(select max(int_col) from functional.alltypesagg
 where identity(smallint_col) > 2 union all
(select max(int_col) from functional.alltypesagg
 where identity(int_col) > 2 union all
(select max(int_col) from functional.alltypesagg
 where identity(bigint_col) > 2 union all
(select max(int_col) from functional.alltypesagg
 where identity(float_col) > 2.0 union all
(select max(int_col) from functional.alltypesagg
 where identity(double_col) > 2.0 union all
(select max(int_col) from functional.alltypesagg
 where identity(string_col) > '2'
)))))))))))))))) v;

Issue Links

• is related to: IMPALA-6215 Race between lib_cache and java udf class loading
• relates to: IMPALA-5760 Flaky test: query_test/test_udfs.py