Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5847

Some query options do not work as expected in .test files

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Impala 3.2.0
    • Component/s: Infrastructure
    • Labels:
      None

      Description

      We often use "set" in .test files to alter query options. Theoretically, a "set" command should change the session-level query options and in most cases a single .test file is executed from the same Impala session. However, for some options using "set" within a query section does not seem to work. For example, "num_nodes" does not work as expected as shown below.

      PyTest:

      import pytest
      from tests.common.impala_test_suite import ImpalaTestSuite
      
      class TestStringQueries(ImpalaTestSuite):
        @classmethod
        def get_workload(cls):
          return 'functional-query'
      
        def test_set_bug(self, vector):
          self.run_test_case('QueryTest/set_bug', vector)
      

      Corresponding .test file:

      ====
      ---- QUERY
      set num_nodes=1;
      select count(*) from functional.alltypes;
      select count(*) from functional.alltypes;
      select count(*) from functional.alltypes;
      ---- RESULTS
      7300
      ---- TYPES
      BIGINT
      ====
      

      After running the test above, I validated that the 3 queries were run from the same session, and that the queries run a distributed plan. The "num_nodes" option was definitely not picked up. I am not sure which query options are affected. In several .test files setting other query options does seem to work as expected.

      I suspect that the test framework might keep its own list of default query options which get submitted together with the query, so the session-level options are overridden on a per-request basis. For example, if I change the pytest to remove the "num_nodes" dictionary entry, then the test works as expected.

      PyTest workaround:

      import pytest
      from tests.common.impala_test_suite import ImpalaTestSuite
      
      class TestStringQueries(ImpalaTestSuite):
        @classmethod
        def get_workload(cls):
          return 'functional-query'
      
        def test_set_bug(self, vector):
          # Workaround SET bug
          vector.get_value('exec_option').pop('num_nodes', None)
          self.run_test_case('QueryTest/set_bug', vector)
      

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              twmarshall Thomas Tauber-Marshall
              Reporter:
              alex.behm Alexander Behm

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment