[IMPALA-4585] test_udfs.py fails on S3 and local filesystem builds

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 2.8.0
• Component/s: Infrastructure
• Labels: broken-build
• Target Version: Impala 2.8.0
• Assignee: Taras Bobrovytsky
• Reporter: Alexander Behm
• Created: 05/Dec/16 07:04
• Updated: 08/Dec/16 02:50
• Resolved: 08/Dec/16 02:50

Description

Looks like a test infra problem with replacing the special _HDFS_FILENAME_ marker.

Michael, might this be related to your recent fix to the test framework?

Jenkins console snippet:

04:05:38 =================================== FAILURES ===================================
04:05:38  TestUdfs.test_udf_errors[exec_option: {'disable_codegen': True, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
04:05:38 [gw2] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
04:05:38 query_test/test_udfs.py:79: in test_udf_errors
04:05:38     self.run_test_case('QueryTest/udf-errors', vector, use_db=unique_database)
04:05:38 common/impala_test_suite.py:327: in run_test_case
04:05:38     self.__verify_exceptions(test_section['CATCH'], str(e), use_db)
04:05:38 common/impala_test_suite.py:218: in __verify_exceptions
04:05:38     (expected_str, actual_str)
04:05:38 E   AssertionError: Unexpected exception string. Expected: Could not load binary: file:/tmp/test-warehouse/not-a-real-file.so
04:05:38 E   Not found in actual: ImpalaBeeswaxException: INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'> MESSAGE: AnalysisException: Could not load binary: __HDFS_FILENAME__(2): No such file or directory
04:05:38 ---------------------------- Captured stderr setup -----------------------------
04:05:38 -- connecting to: localhost:21000
04:05:38 SET sync_ddl=False;
04:05:38 -- executing against localhost:21000
04:05:38 DROP DATABASE IF EXISTS `test_udf_errors_cace194a` CASCADE;
04:05:38 
04:05:38 SET sync_ddl=False;
04:05:38 -- executing against localhost:21000
04:05:38 CREATE DATABASE `test_udf_errors_cace194a`;
04:05:38 
04:05:38 MainThread: Created database "test_udf_errors_cace194a" for test ID "query_test/test_udfs.py::TestUdfs::()::test_udf_errors[exec_option: {'disable_codegen': True, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none]"
04:05:38 ----------------------------- Captured stderr call -----------------------------
04:05:38 -- executing against localhost:21000
04:05:38 use test_udf_errors_cace194a;
04:05:38 
04:05:38 SET disable_codegen=1;
04:05:38 SET abort_on_error=1;
04:05:38 SET exec_single_node_rows_threshold=0;
04:05:38 SET batch_size=0;
04:05:38 SET num_nodes=0;
04:05:38 -- executing against localhost:21000
04:05:38 create function if not exists hive_pi() returns double
04:05:38 location 'file:/tmp/test-warehouse/hive-exec.jar'
04:05:38 symbol='org.apache.hadoop.hive.ql.udf.UDFPI';
04:05:38 
04:05:38 -- executing against localhost:21000
04:05:38 create function if not exists foo() returns double
04:05:38 location 'file:/tmp/test-warehouse/not-a-real-file.so'
04:05:38 symbol='FnDoesNotExist';
04:05:38 
04:05:38  TestUdfs.test_udf_errors[exec_option: {'disable_codegen': True, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 100, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
04:05:38 [gw2] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
04:05:38 query_test/test_udfs.py:79: in test_udf_errors
04:05:38     self.run_test_case('QueryTest/udf-errors', vector, use_db=unique_database)
04:05:38 common/impala_test_suite.py:327: in run_test_case
04:05:38     self.__verify_exceptions(test_section['CATCH'], str(e), use_db)
04:05:38 common/impala_test_suite.py:218: in __verify_exceptions
04:05:38     (expected_str, actual_str)
04:05:38 E   AssertionError: Unexpected exception string. Expected: Could not load binary: file:/tmp/test-warehouse/not-a-real-file.so
04:05:38 E   Not found in actual: ImpalaBeeswaxException: INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'> MESSAGE: AnalysisException: Could not load binary: __HDFS_FILENAME__(2): No such file or directory
04:05:38 ---------------------------- Captured stderr setup -----------------------------
04:05:38 SET sync_ddl=False;
04:05:38 -- executing against localhost:21000
04:05:38 DROP DATABASE IF EXISTS `test_udf_errors_39fb7221` CASCADE;
04:05:38 
04:05:38 SET sync_ddl=False;
04:05:38 -- executing against localhost:21000
04:05:38 CREATE DATABASE `test_udf_errors_39fb7221`;
04:05:38 
04:05:38 MainThread: Created database "test_udf_errors_39fb7221" for test ID "query_test/test_udfs.py::TestUdfs::()::test_udf_errors[exec_option: {'disable_codegen': True, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 100, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none]"
04:05:38 ----------------------------- Captured stderr call -----------------------------
04:05:38 -- executing against localhost:21000
04:05:38 use test_udf_errors_39fb7221;
04:05:38 
04:05:38 SET disable_codegen=1;
04:05:38 SET abort_on_error=1;
04:05:38 SET exec_single_node_rows_threshold=100;
04:05:38 SET batch_size=0;
04:05:38 SET num_nodes=0;
04:05:38 -- executing against localhost:21000
04:05:38 create function if not exists hive_pi() returns double
04:05:38 location 'file:/tmp/test-warehouse/hive-exec.jar'
04:05:38 symbol='org.apache.hadoop.hive.ql.udf.UDFPI';
04:05:38 
04:05:38 -- executing against localhost:21000
04:05:38 create function if not exists foo() returns double
04:05:38 location 'file:/tmp/test-warehouse/not-a-real-file.so'
04:05:38 symbol='FnDoesNotExist';
04:05:38 
04:05:38  TestUdfs.test_udf_errors[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
04:05:38 [gw1] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
04:05:38 query_test/test_udfs.py:79: in test_udf_errors
04:05:38     self.run_test_case('QueryTest/udf-errors', vector, use_db=unique_database)
04:05:38 common/impala_test_suite.py:327: in run_test_case
04:05:38     self.__verify_exceptions(test_section['CATCH'], str(e), use_db)
04:05:38 common/impala_test_suite.py:218: in __verify_exceptions
04:05:38     (expected_str, actual_str)
04:05:38 E   AssertionError: Unexpected exception string. Expected: Could not load binary: file:/tmp/test-warehouse/not-a-real-file.so
04:05:38 E   Not found in actual: ImpalaBeeswaxException: INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'> MESSAGE: AnalysisException: Could not load binary: __HDFS_FILENAME__(2): No such file or directory
04:05:38 ---------------------------- Captured stderr setup -----------------------------
04:05:38 SET sync_ddl=False;
04:05:38 -- executing against localhost:21000
04:05:38 DROP DATABASE IF EXISTS `test_udf_errors_a0ea005b` CASCADE;
04:05:38 
04:05:38 SET sync_ddl=False;
04:05:38 -- executing against localhost:21000
04:05:38 CREATE DATABASE `test_udf_errors_a0ea005b`;
04:05:38 
04:05:38 MainThread: Created database "test_udf_errors_a0ea005b" for test ID "query_test/test_udfs.py::TestUdfs::()::test_udf_errors[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none]"
04:05:38 ----------------------------- Captured stderr call -----------------------------
04:05:38 -- executing against localhost:21000
04:05:38 use test_udf_errors_a0ea005b;
04:05:38 
04:05:38 SET disable_codegen=1;
04:05:38 SET abort_on_error=1;
04:05:38 SET exec_single_node_rows_threshold=0;
04:05:38 SET batch_size=0;
04:05:38 SET num_nodes=0;
04:05:38 -- executing against localhost:21000
04:05:38 create function if not exists hive_pi() returns double
04:05:38 location 'file:/tmp/test-warehouse/hive-exec.jar'
04:05:38 symbol='org.apache.hadoop.hive.ql.udf.UDFPI';
04:05:38 
04:05:38 -- executing against localhost:21000
04:05:38 create function if not exists foo() returns double
04:05:38 location 'file:/tmp/test-warehouse/not-a-real-file.so'
04:05:38 symbol='FnDoesNotExist';
04:05:38 
04:05:38  TestUdfs.test_udf_errors[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 100, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
04:05:38 [gw1] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
04:05:38 query_test/test_udfs.py:79: in test_udf_errors
04:05:38     self.run_test_case('QueryTest/udf-errors', vector, use_db=unique_database)
04:05:38 common/impala_test_suite.py:327: in run_test_case
04:05:38     self.__verify_exceptions(test_section['CATCH'], str(e), use_db)
04:05:38 common/impala_test_suite.py:218: in __verify_exceptions
04:05:38     (expected_str, actual_str)
04:05:38 E   AssertionError: Unexpected exception string. Expected: Could not load binary: file:/tmp/test-warehouse/not-a-real-file.so
04:05:38 E   Not found in actual: ImpalaBeeswaxException: INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'> MESSAGE: AnalysisException: Could not load binary: __HDFS_FILENAME__(2): No such file or directory
04:05:38 ---------------------------- Captured stderr setup -----------------------------
04:05:38 SET sync_ddl=False;
04:05:38 -- executing against localhost:21000
04:05:38 DROP DATABASE IF EXISTS `test_udf_errors_e4993d5` CASCADE;
04:05:38 
04:05:38 SET sync_ddl=False;
04:05:38 -- executing against localhost:21000
04:05:38 CREATE DATABASE `test_udf_errors_e4993d5`;
04:05:38 
04:05:38 MainThread: Created database "test_udf_errors_e4993d5" for test ID "query_test/test_udfs.py::TestUdfs::()::test_udf_errors[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 100, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none]"
04:05:38 ----------------------------- Captured stderr call -----------------------------
04:05:38 -- executing against localhost:21000
04:05:38 use test_udf_errors_e4993d5;
04:05:38 
04:05:38 SET disable_codegen=1;
04:05:38 SET abort_on_error=1;
04:05:38 SET exec_single_node_rows_threshold=100;
04:05:38 SET batch_size=0;
04:05:38 SET num_nodes=0;
04:05:38 -- executing against localhost:21000
04:05:38 create function if not exists hive_pi() returns double
04:05:38 location 'file:/tmp/test-warehouse/hive-exec.jar'
04:05:38 symbol='org.apache.hadoop.hive.ql.udf.UDFPI';
04:05:38 
04:05:38 -- executing against localhost:21000
04:05:38 create function if not exists foo() returns double
04:05:38 location 'file:/tmp/test-warehouse/not-a-real-file.so'
04:05:38 symbol='FnDoesNotExist';
04:05:38 
04:05:38  generated xml file: /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/logs/ee_tests/results/TEST-impala-parallel.xml 
04:05:38 =========================== short test summary info ============================