Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5330

Impala tests never use or set a secondary FS, so TestMultipleFilesystems is always skipped

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Infrastructure
    • Labels:
    • Epic Color:
      ghx-label-3

      Description

      This issue came up while investigating how can set up Impala + ADLS testing on remote clusters. Since testing Impala with ADLS is analogous to testing Impala with s3, I used that testing as a starting point.

      When we change the TARGET_FILESYSTEM to 's3', the end result is to overwrite the defaultFS setting in core-site.xml to the s3 location. There's not a point (or doesn't appear to be a point) where we ever treat s3 as a true secondary filesystem.

      Further more, if we look at our test suite that is supposed to check multiple filesystems simultaneously, we can see there's a pytest SkipIf decorator set to skip the test if a secondary FS is not set.

      @SkipIf.no_secondary_fs
      class TestMultipleFilesystems(ImpalaTestSuite)
      

      From what I can see by going through old Jenkins console logs, this test never gets executed. It's skipped during exhaustive runs with HDFS as the default, and is also skipped during s3 and local FS test runs as well.

      04:27:57 query_test/test_multiple_filesystems.py::TestMultipleFilesystems::test_multiple_filesystems[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] SKIPPED
      

        Activity

        There are no comments yet on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            dknupp David Knupp
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development