Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2192

Wrong results on TPCH-Q11 during nightly data load

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.3.0
    • Fix Version/s: Impala 2.3.0
    • Component/s: None

      Description

      Build:
      http://sandbox.jenkins.cloudera.com/job/impala-CDH5-nightly-data-load/584

      Relevant console snippet:

      ...
      03:48:38.836 [gw1] PASSED query_test/test_udfs.py::TestUdfs::test_drop_function_while_running[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
      03:48:39.403 [gw0] PASSED query_test/test_aggregation.py::TestAggregation::test_aggregation[data_type: double | agg_func: max | exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: parquet/none] 
      03:48:44.370 [gw4] PASSED query_test/test_insert_behaviour.py::TestInsertBehaviour::test_load_permissions 
      03:48:47.938 [gw8] PASSED query_test/test_tpch_queries.py::TestTpchQuery::test_tpch_q8[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/gzip/block] 
      03:48:49.972 [gw7] PASSED statestore/test_statestore.py::TestStatestore::test_topic_persistence 
      03:49:07.942 [gw5] PASSED query_test/test_udfs.py::TestUdfs::test_udf_update_via_drop[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
      03:49:21.561 [gw3] PASSED statestore/test_statestore.py::TestStatestore::test_hung_heartbeat 
      03:49:21.820 
      03:49:21.820 =================================== FAILURES ===================================
      03:49:21.820  TestTpchQuery.test_tpch_q11[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] 
      03:49:21.821 [gw6] linux2 -- Python 2.6.6 /data/2/jenkins/workspace/impala-CDH5-nightly-data-load/repos/Impala/bin/../infra/python/env/bin/python
      03:49:21.822 query_test/test_tpch_queries.py:57: in test_tpch_q11
      03:49:21.822     self.run_test_case('tpch-q11', vector)
      03:49:21.822 common/impala_test_suite.py:253: in run_test_case
      03:49:21.822     pytest.config.option.update_results)
      03:49:21.822 common/test_result_verifier.py:345: in verify_raw_results
      03:49:21.822     VERIFIER_MAP[verifier](expected, actual)
      03:49:21.822 common/test_result_verifier.py:197: in verify_query_result_is_equal
      03:49:21.822     assert expected_results == actual_results
      03:49:21.822 E   assert Comparing QueryTestResults (expected vs actual):
      03:49:21.822 E     Detailed information truncated, use "-vv" to show
      03:49:21.824 ---------------------------- Captured stderr setup -----------------------------
      03:49:21.825 -- connecting to: localhost:21000
      03:49:21.826 ----------------------------- Captured stderr call -----------------------------
      03:49:21.826 -- executing against localhost:21000
      03:49:21.826 use tpch;
      03:49:21.826 
      03:49:21.826 SET disable_codegen=False;
      03:49:21.826 SET abort_on_error=1;
      03:49:21.826 SET exec_single_node_rows_threshold=0;
      03:49:21.826 SET batch_size=0;
      03:49:21.826 SET num_nodes=0;
      03:49:21.826 MainThread: Query Name: 
      03:49:21.826 TPCH-Q11
      03:49:21.826 
      03:49:21.826 -- executing against localhost:21000
      03:49:21.826 select
      03:49:21.826   *
      03:49:21.826 from (
      03:49:21.826   select
      03:49:21.826     ps_partkey,
      03:49:21.826     sum(ps_supplycost * ps_availqty) as value
      03:49:21.826   from
      03:49:21.826     partsupp,
      03:49:21.826     supplier,
      03:49:21.826     nation
      03:49:21.826   where
      03:49:21.826     ps_suppkey = s_suppkey
      03:49:21.826     and s_nationkey = n_nationkey
      03:49:21.826     and n_name = 'GERMANY'
      03:49:21.826   group by
      03:49:21.826     ps_partkey
      03:49:21.826 ) as inner_query
      03:49:21.826 where
      03:49:21.826   value > (
      03:49:21.826     select
      03:49:21.826       sum(ps_supplycost * ps_availqty) * 0.0001
      03:49:21.826     from
      03:49:21.826       partsupp,
      03:49:21.826       supplier,
      03:49:21.826       nation
      03:49:21.826     where
      03:49:21.826       ps_suppkey = s_suppkey
      03:49:21.826       and s_nationkey = n_nationkey
      03:49:21.826       and n_name = 'GERMANY'
      03:49:21.826   )
      03:49:21.826 order by
      03:49:21.826   value desc;
      03:49:21.826 
      03:49:21.826 MainThread: Comparing QueryTestResults (expected vs actual):
      03:49:21.826 129760,17538456.86 != 192960,1475555058456027064291.57
      03:49:21.826 166726,16503353.92 != 177195,1327427703544139364815.41
      03:49:21.826 191287,16474801.97 != 10,1844674407377905958.26
      03:49:21.826 161758,16101755.54 != 129760,17538456.86
      03:49:21.826 34452,15983844.72 != 166726,16503353.92
      03:49:21.826 139035,15907078.34 != 191287,16474801.97
      03:49:21.826 9403,15451755.62 != 161758,16101755.54
      03:49:21.826 154358,15212937.88 != 34452,15983844.72
      03:49:21.826 38823,15064802.86 != 139035,15907078.34
      03:49:21.826 85606,15053957.15 != 9403,15451755.62
      03:49:21.826 33354,14408297.40 != 154358,15212937.88
      03:49:21.826 154747,14407580.68 != 38823,15064802.86
      03:49:21.826 82865,14235489.78 != 85606,15053957.15
      03:49:21.826 76094,14094247.04 != 33354,14408297.40
      03:49:21.826 222,13937777.74 != 154747,14407580.68
      03:49:21.826 121271,13908336.00 != 82865,14235489.78
      03:49:21.826 55221,13716120.47 != 76094,14094247.04
      03:49:21.826 22819,13666434.28 != 222,13937777.74
      03:49:21.826 76281,13646853.68 != 121271,13908336.00
      03:49:21.826 85298,13581154.93 != 55221,13716120.47
      03:49:21.826 85158,13554904.00 != 22819,13666434.28
      03:49:21.826 139684,13535538.72 != 76281,13646853.68
      03:49:21.826 31034,13498025.25 != 85298,13581154.93
      03:49:21.826 87305,13482847.04 != 85158,13554904.00
      03:49:21.826 10181,13445148.75 != 139684,13535538.72
      ...
      

      I strongly suspect this is what ASAN is telling us in IMPALA-2191, but I'll leave this issue separate just to be sure.

      Dimitris, based on the timing and the query it looks like your nested-loop join patch may have something to do with this. Feel free to re-assign if you think otherwise.

        Attachments

          Activity

            People

            • Assignee:
              dtsirogiannis Dimitris Tsirogiannis
              Reporter:
              alex.behm Alexander Behm
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: