[IMPALA-5847] Some query options do not work as expected in .test files

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: Impala 3.2.0
• Component/s: Infrastructure
• Target Version: Product Backlog
• Epic Color: ghx-label-9
• Assignee: Thomas Tauber-Marshall
• Reporter: Alexander Behm
• Created: 24/Aug/17 22:49
• Updated: 22/Jan/19 18:58
• Resolved: 22/Jan/19 18:58

Description

We often use "set" in .test files to alter query options. Theoretically, a "set" command should change the session-level query options and in most cases a single .test file is executed from the same Impala session. However, for some options using "set" within a query section does not seem to work. For example, "num_nodes" does not work as expected as shown below.

PyTest:

import pytest
from tests.common.impala_test_suite import ImpalaTestSuite

class TestStringQueries(ImpalaTestSuite):
  @classmethod
  def get_workload(cls):
    return 'functional-query'

  def test_set_bug(self, vector):
    self.run_test_case('QueryTest/set_bug', vector)

Corresponding .test file:

====
---- QUERY
set num_nodes=1;
select count(*) from functional.alltypes;
select count(*) from functional.alltypes;
select count(*) from functional.alltypes;
---- RESULTS
7300
---- TYPES
BIGINT
====

After running the test above, I validated that the 3 queries were run from the same session, and that the queries run a distributed plan. The "num_nodes" option was definitely not picked up. I am not sure which query options are affected. In several .test files setting other query options does seem to work as expected.

I suspect that the test framework might keep its own list of default query options which get submitted together with the query, so the session-level options are overridden on a per-request basis. For example, if I change the pytest to remove the "num_nodes" dictionary entry, then the test works as expected.

PyTest workaround:

import pytest
from tests.common.impala_test_suite import ImpalaTestSuite

class TestStringQueries(ImpalaTestSuite):
  @classmethod
  def get_workload(cls):
    return 'functional-query'

  def test_set_bug(self, vector):
    # Workaround SET bug
    vector.get_value('exec_option').pop('num_nodes', None)
    self.run_test_case('QueryTest/set_bug', vector)

Issue Links

• is duplicated by: IMPALA-6940 num_nodes=1 is not applied in end-to-end test