Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4564

TestMtDopParquet.test_parquet should be skipped in legacy AGG/JOIN builds

    Details

      Description

      The newly added test in commit 290db20dbc364c102c40ea2022535b896bf1c45c needs to be skipped in legacy AGG/JOIN builds.

      18:05:39 =================================== FAILURES ===================================
      18:05:39  TestMtDopParquet.test_parquet[mt_dop: 1 | exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: parquet/none] 
      18:05:39 [gw3] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
      18:05:39 query_test/test_mt_dop.py:90: in test_parquet
      18:05:39     self.run_test_case('QueryTest/mt-dop-parquet', vector)
      18:05:39 common/impala_test_suite.py:321: in run_test_case
      18:05:39     result = self.__execute_query(target_impalad_client, query, user=user)
      18:05:39 common/impala_test_suite.py:525: in __execute_query
      18:05:39     return impalad_client.execute(query, user=user)
      18:05:39 common/impala_connection.py:160: in execute
      18:05:39     return self.__beeswax_client.execute(sql_stmt, user=user)
      18:05:39 beeswax/impala_beeswax.py:173: in execute
      18:05:39     handle = self.__execute_query(query_string.strip(), user=user)
      18:05:39 beeswax/impala_beeswax.py:339: in __execute_query
      18:05:39     self.wait_for_completion(handle)
      18:05:39 beeswax/impala_beeswax.py:359: in wait_for_completion
      18:05:39     raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
      18:05:39 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      18:05:39 E    Query aborted:
      18:05:39 E   Query referencing nested types is not supported because the --enable_partitioned_hash_join and/or --enable_partitioned_aggregation Impala Daemon start-up flags are set to false.
      18:05:39 E   To enable nested types support please set those flags to true (they are enabled by default).
      18:05:39 ---------------------------- Captured stderr setup -----------------------------
      18:05:39 -- connecting to: localhost:21000
      18:05:39 ----------------------------- Captured stderr call -----------------------------
      18:05:39 -- executing against localhost:21000
      18:05:39 use functional_parquet;
      18:05:39 
      18:05:39 SET disable_codegen=False;
      18:05:39 SET batch_size=0;
      18:05:39 SET num_nodes=0;
      18:05:39 SET mt_dop=1;
      18:05:39 SET abort_on_error=1;
      18:05:39 SET exec_single_node_rows_threshold=0;
      18:05:39 -- executing against localhost:21000
      18:05:39 select * from functional_parquet.bad_metadata_len;
      18:05:39 
      18:05:39 -- executing against localhost:21000
      18:05:39 select id, cnt
      18:05:39 from functional_parquet.complextypestbl t,
      18:05:39   (select count(item) cnt from t.int_array) v
      18:05:39 order by id
      18:05:39 limit 10;
      18:05:39 
      18:05:39  TestMtDopParquet.test_parquet[mt_dop: 2 | exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: parquet/none] 
      18:05:39 [gw3] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
      18:05:39 query_test/test_mt_dop.py:90: in test_parquet
      18:05:39     self.run_test_case('QueryTest/mt-dop-parquet', vector)
      18:05:39 common/impala_test_suite.py:321: in run_test_case
      18:05:39     result = self.__execute_query(target_impalad_client, query, user=user)
      18:05:39 common/impala_test_suite.py:525: in __execute_query
      18:05:39     return impalad_client.execute(query, user=user)
      18:05:39 common/impala_connection.py:160: in execute
      18:05:39     return self.__beeswax_client.execute(sql_stmt, user=user)
      18:05:39 beeswax/impala_beeswax.py:173: in execute
      18:05:39     handle = self.__execute_query(query_string.strip(), user=user)
      18:05:39 beeswax/impala_beeswax.py:339: in __execute_query
      18:05:39     self.wait_for_completion(handle)
      18:05:39 beeswax/impala_beeswax.py:359: in wait_for_completion
      18:05:39     raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
      18:05:39 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      18:05:39 E    Query aborted:
      18:05:39 E   Query referencing nested types is not supported because the --enable_partitioned_hash_join and/or --enable_partitioned_aggregation Impala Daemon start-up flags are set to false.
      18:05:39 E   To enable nested types support please set those flags to true (they are enabled by default).
      18:05:39 ----------------------------- Captured stderr call -----------------------------
      18:05:39 -- executing against localhost:21000
      18:05:39 use functional_parquet;
      18:05:39 
      18:05:39 SET disable_codegen=False;
      18:05:39 SET batch_size=0;
      18:05:39 SET num_nodes=0;
      18:05:39 SET mt_dop=2;
      18:05:39 SET abort_on_error=1;
      18:05:39 SET exec_single_node_rows_threshold=0;
      18:05:39 -- executing against localhost:21000
      18:05:39 select * from functional_parquet.bad_metadata_len;
      18:05:39 
      18:05:39 -- executing against localhost:21000
      18:05:39 select id, cnt
      18:05:39 from functional_parquet.complextypestbl t,
      18:05:39   (select count(item) cnt from t.int_array) v
      18:05:39 order by id
      18:05:39 limit 10;
      18:05:39 
      18:05:39  TestMtDopParquet.test_parquet[mt_dop: 8 | exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: parquet/none] 
      18:05:39 [gw3] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
      18:05:39 query_test/test_mt_dop.py:90: in test_parquet
      18:05:39     self.run_test_case('QueryTest/mt-dop-parquet', vector)
      18:05:39 common/impala_test_suite.py:321: in run_test_case
      18:05:39     result = self.__execute_query(target_impalad_client, query, user=user)
      18:05:39 common/impala_test_suite.py:525: in __execute_query
      18:05:39     return impalad_client.execute(query, user=user)
      18:05:39 common/impala_connection.py:160: in execute
      18:05:39     return self.__beeswax_client.execute(sql_stmt, user=user)
      18:05:39 beeswax/impala_beeswax.py:173: in execute
      18:05:39     handle = self.__execute_query(query_string.strip(), user=user)
      18:05:39 beeswax/impala_beeswax.py:339: in __execute_query
      18:05:39     self.wait_for_completion(handle)
      18:05:39 beeswax/impala_beeswax.py:359: in wait_for_completion
      18:05:39     raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
      18:05:39 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      18:05:39 E    Query aborted:
      18:05:39 E   Query referencing nested types is not supported because the --enable_partitioned_hash_join and/or --enable_partitioned_aggregation Impala Daemon start-up flags are set to false.
      18:05:39 E   To enable nested types support please set those flags to true (they are enabled by default).
      18:05:39 ----------------------------- Captured stderr call -----------------------------
      18:05:39 -- executing against localhost:21000
      18:05:39 use functional_parquet;
      18:05:39 
      18:05:39 SET disable_codegen=False;
      18:05:39 SET batch_size=0;
      18:05:39 SET num_nodes=0;
      18:05:39 SET mt_dop=8;
      18:05:39 SET abort_on_error=1;
      18:05:39 SET exec_single_node_rows_threshold=0;
      18:05:39 -- executing against localhost:21000
      18:05:39 select * from functional_parquet.bad_metadata_len;
      18:05:39 
      18:05:39 -- executing against localhost:21000
      18:05:39 select id, cnt
      18:05:39 from functional_parquet.complextypestbl t,
      18:05:39   (select count(item) cnt from t.int_array) v
      18:05:39 order by id
      18:05:39 limit 10;
      18:05:39 
      18:05:39  generated xml file: /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/logs/ee_tests/results/TEST-impala-parallel.xml 
      18:05:39 =========================== short test summary info ============================
      

        Activity

        Hide
        tarmstrong Tim Armstrong added a comment -

        IMPALA-4564,IMPALA-4565: mt_dop fixes for old aggs and joins

        Fix a test bug where we need to skip nested types tests for the old aggs
        and joins.

        Fix a product bug where *eos is not initialised by the MT scan node.
        This causes incorrect results when the calling ExecNode does not
        initialise the eos variable, e.g. the sort node and the old agg and join
        nodes.

        Testing:
        Added a test that reproduces the incorrect results with the sort node
        when run under ASAN

        Tested the mt_dop tests locally with old aggs and joins to ensure they
        pass.

        Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313
        Reviewed-on: http://gerrit.cloudera.org:8080/5302
        Reviewed-by: Alex Behm <alex.behm@cloudera.com>
        Tested-by: Impala Public Jenkins

        Show
        tarmstrong Tim Armstrong added a comment - IMPALA-4564 , IMPALA-4565 : mt_dop fixes for old aggs and joins Fix a test bug where we need to skip nested types tests for the old aggs and joins. Fix a product bug where *eos is not initialised by the MT scan node. This causes incorrect results when the calling ExecNode does not initialise the eos variable, e.g. the sort node and the old agg and join nodes. Testing: Added a test that reproduces the incorrect results with the sort node when run under ASAN Tested the mt_dop tests locally with old aggs and joins to ensure they pass. Change-Id: I48c50c8aa0c23710eb099fba252bc3c0cb74b313 Reviewed-on: http://gerrit.cloudera.org:8080/5302 Reviewed-by: Alex Behm <alex.behm@cloudera.com> Tested-by: Impala Public Jenkins

          People

          • Assignee:
            tarmstrong Tim Armstrong
            Reporter:
            kwho Michael Ho
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development