Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9355

TestExchangeMemUsage.test_exchange_mem_usage_scaling doesn't hit the memory limit

    XMLWordPrintableJSON

Details

    Description

      The EE test test_exchange_mem_usage_scaling failed because the query at https://github.com/apache/impala/blame/master/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test#L7-L15 does not hit the specified memory limit (170m) at https://github.com/apache/impala/blame/master/testdata/workloads/functional-query/queries/QueryTest/exchange-mem-scaling.test#L7. We may need to further reduce the specified limit. In what follows the error message is also given. Recall that the same issue occurred at https://issues.apache.org/jira/browse/IMPALA-7873 but was resolved.

      FAIL query_test/test_mem_usage_scaling.py::TestExchangeMemUsage::()::test_exchange_mem_usage_scaling[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
      =================================== FAILURES ===================================
       TestExchangeMemUsage.test_exchange_mem_usage_scaling[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] 
      [gw3] linux2 -- Python 2.7.12 /home/ubuntu/Impala/bin/../infra/python/env/bin/python
      query_test/test_mem_usage_scaling.py:386: in test_exchange_mem_usage_scaling
          self.run_test_case('QueryTest/exchange-mem-scaling', vector)
      common/impala_test_suite.py:674: in run_test_case
          expected_str, query)
      E   AssertionError: Expected exception: Memory limit exceeded
      E   
      E   when running:
      E   
      E   set mem_limit=170m;
      E   set num_scanner_threads=1;
      E   select *
      E   from tpch_parquet.lineitem l1
      E     join tpch_parquet.lineitem l2 on l1.l_orderkey = l2.l_orderkey and
      E         l1.l_partkey = l2.l_partkey and l1.l_suppkey = l2.l_suppkey
      E         and l1.l_linenumber = l2.l_linenumber
      E   order by l1.l_orderkey desc, l1.l_partkey, l1.l_suppkey, l1.l_linenumber
      E   limit 5
      

      tarmstrong@cloudera.com and joemcdonnell reviewed the patch at https://gerrit.cloudera.org/c/11965/. Assign this JIRA to joemcdonnell for now. Please re-assign the JIRA to others as appropriate. Thanks!

       

      Attachments

        Activity

          People

            sql_forever Qifan Chen
            fangyurao Fang-Yu Rao
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: