Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6334

test_compute_stats_tablesample failing on Isilon builds

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.12.0
    • Fix Version/s: Impala 2.12.0
    • Component/s: None
    • Labels:
    • Epic Color:
      ghx-label-4

      Description

      MainThread: Comparing QueryTestResults (expected vs actual):
      Expected:

      3660,3660,12,regex:.*B,'NOT CACHED','NOT CACHED','TEXT','false','hdfs://10.17.95.12:8020/test-warehouse/test_compute_stats_tablesample_16dd5daf.db/alltypesnopart'
      

      Actual:

      3661,3661,12,'238.68KB','NOT CACHED','NOT CACHED','TEXT','false','hdfs://10.17.95.12:8020/test-warehouse/test_compute_stats_tablesample_16dd5daf.db/alltypesnopart'
      

      Stacktrace

      self = <test_stats_extrapolation.TestStatsExtrapolation object at 0x3aa1890>
      vector = <tests.common.test_vector.ImpalaTestVector object at 0x31fd810>
      unique_database = 'test_compute_stats_tablesample_16dd5daf'
      
          @CustomClusterTestSuite.with_args(impalad_args=('--enable_stats_extrapolation=true'))
          def test_compute_stats_tablesample(self, vector, unique_database):
            # Create a partitioned and unpartitioned text table. Use the existing files from
            # functional.alltypes as data because those have a known, stable file size. This
            # test is sensitive to changes in file sizes across test runs because the sampling
            # is file based. Creating test tables with INSERT does not guarantee that the same
            # file sample is selected across test runs, even with REPEATABLE.
          
            # Create partitioned test table. External to avoid dropping files from alltypes.
            part_test_tbl = unique_database + ".alltypes"
            self.client.execute(
              "create external table %s like functional.alltypes" % part_test_tbl)
            alltypes_loc = self._get_table_location("functional.alltypes", vector)
            for m in xrange(1, 13):
              part_loc = path.join(alltypes_loc, "year=2009/month=%s" % m)
              self.client.execute(
                "alter table %s add partition (year=2009,month=%s) location '%s'"
                % (part_test_tbl, m, part_loc))
          
            # Create unpartitioned test table.
            nopart_test_tbl = unique_database + ".alltypesnopart"
            self.client.execute("drop table if exists %s" % nopart_test_tbl)
            self.client.execute(
              "create table %s like functional.alltypesnopart" % nopart_test_tbl)
            nopart_test_tbl_loc = self._get_table_location(nopart_test_tbl, vector)
            # Remove NameNode prefix and first '/' because PyWebHdfs expects that
            if nopart_test_tbl_loc.startswith(NAMENODE):
              nopart_test_tbl_loc = nopart_test_tbl_loc[len(NAMENODE)+1:]
            for m in xrange(1, 13):
              src_part_loc = alltypes_loc + "/year=2009/month=%s" % m
              # Remove NameNode prefix and first '/' because PyWebHdfs expects that
              if src_part_loc.startswith(NAMENODE): src_part_loc = src_part_loc[len(NAMENODE)+1:]
              file_names = self.filesystem_client.ls(src_part_loc)
              for f in file_names:
                self.filesystem_client.copy(path.join(src_part_loc, f),
                                            path.join(nopart_test_tbl_loc, f))
            self.client.execute("refresh %s" % nopart_test_tbl)
          
      >     self.run_test_case('QueryTest/compute-stats-tablesample', vector, unique_database)
      
      custom_cluster/test_stats_extrapolation.py:84: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      common/impala_test_suite.py:424: in run_test_case
          self.__verify_results_and_errors(vector, test_section, result, use_db)
      common/impala_test_suite.py:297: in __verify_results_and_errors
          replace_filenames_with_placeholder)
      common/test_result_verifier.py:404: in verify_raw_results
          VERIFIER_MAP[verifier](expected, actual)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      expected_results = <tests.common.test_result_verifier.QueryTestResult object at 0x3a84850>
      actual_results = <tests.common.test_result_verifier.QueryTestResult object at 0x3aa0c10>
      
          def verify_query_result_is_equal(expected_results, actual_results):
            assert_args_not_none(expected_results, actual_results)
      >     assert expected_results == actual_results
      E     assert Comparing QueryTestResults (expected vs actual):
      E       3660,3660,12,regex:.*B,'NOT CACHED','NOT CACHED','TEXT','false','hdfs://10.17.95.12:8020/test-warehouse/test_compute_stats_tablesample_16dd5daf.db/alltypesnopart' != 3661,3661,12,'238.68KB','NOT CACHED','NOT CACHED','TEXT','false','hdfs://10.17.95.12:8020/test-warehouse/test_compute_stats_tablesample_16dd5daf.db/alltypesnopart'
      
      common/test_result_verifier.py:231: AssertionError
      

        Attachments

          Activity

            People

            • Assignee:
              alex.behm Alexander Behm
              Reporter:
              dknupp David Knupp
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: