Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1933

Scanner sometimes returns wrong error in very low memory

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Duplicate
    • Impala 2.1, Impala 2.2
    • Impala 2.3.0
    • None

    Description

      In very low mem situations the parquet-scanner will fail with the wrong error. That causes the mem_usage_scaling test to fail sometimes, example below:

      FAILED TESTS
      ================================================
      1 tests failed.
      REGRESSION:  query_test.test_mem_usage_scaling.TestTpchMemLimitError.test_low_mem_limit_q21[mem_limit: 100 | exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: parquet/none]
      
      Error Message:
      test failure
      
      Stack Trace:
      query_test/test_mem_usage_scaling.py:155: in test_low_mem_limit_q21
          self.low_memory_limit_test(vector, 'tpch-q21', self.MIN_MEM_FOR_TPCH['Q21']);
      query_test/test_mem_usage_scaling.py:119: in low_memory_limit_test
          assert TestTpchMemLimitError.EXPECTED_ERROR_MSG in str(e)
      E   assert 'Memory limit exceeded' in 'ImpalaBeeswaxException:\n Query aborted:\nMetadata states that in group hdfs://localhost:20500/test-warehouse/tpch.or...parquet/a345e61c42eae84f-e2322ebea5a08dbe_467083517_data.0.parq(0) there are 318497 rows, but only 0 rows were read.\n'
      E    +  where 'Memory limit exceeded' = TestTpchMemLimitError.EXPECTED_ERROR_MSG
      E    +  and   'ImpalaBeeswaxException:\n Query aborted:\nMetadata states that in group hdfs://localhost:20500/test-warehouse/tpch.or...parquet/a345e61c42eae84f-e2322ebea5a08dbe_467083517_data.0.parq(0) there are 318497 rows, but only 0 rows were read.\n' = str(ImpalaBeeswaxException())
      

      When run with ASAN enabled, it may cause impalad to crash.

      Attachments

        Issue Links

          Activity

            People

              skye Skye Wanderman-Milne
              ippokratis Ippokratis Pandis
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: