Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-11646

IMPALA-11562 seems to break test_unsupported_text_compression in s3 builds

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 4.2.0
    • Impala 4.2.0
    • None
    • ghx-label-5

    Description

      It seems that the recently merged IMPALA-11562 may have inadvertently broken the end-to-end test of test_unsupported_text_compression.

      Error Message

      metadata/test_partition_metadata.py:214: in test_unsupported_text_compression     assert len(show_files_result.data) == 5, "Expected one file per partition dir" E   AssertionError: Expected one file per partition dir E   assert 2 == 5 E    +  where 2 = len(['s3a://impala-test-uswest2-2/test-warehouse/alltypes/year=2009/month=1/090101.txt\t19.95KB\tyear=2009/month=1', 's3a://impala-test-uswest2-2/test-warehouse/alltypes_text_gzip/year=2009/month=2/000000_0.gz\t3.00KB\tyear=2009/month=2']) E    +    where ['s3a://impala-test-uswest2-2/test-warehouse/alltypes/year=2009/month=1/090101.txt\t19.95KB\tyear=2009/month=1', 's3a://impala-test-uswest2-2/test-warehouse/alltypes_text_gzip/year=2009/month=2/000000_0.gz\t3.00KB\tyear=2009/month=2'] = <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at 0x7f94f94fab10>.data
      

      Stacktrace

      metadata/test_partition_metadata.py:214: in test_unsupported_text_compression
          assert len(show_files_result.data) == 5, "Expected one file per partition dir"
      E   AssertionError: Expected one file per partition dir
      E   assert 2 == 5
      E    +  where 2 = len(['s3a://impala-test-uswest2-2/test-warehouse/alltypes/year=2009/month=1/090101.txt\t19.95KB\tyear=2009/month=1', 's3a://impala-test-uswest2-2/test-warehouse/alltypes_text_gzip/year=2009/month=2/000000_0.gz\t3.00KB\tyear=2009/month=2'])
      E    +    where ['s3a://impala-test-uswest2-2/test-warehouse/alltypes/year=2009/month=1/090101.txt\t19.95KB\tyear=2009/month=1', 's3a://impala-test-uswest2-2/test-warehouse/alltypes_text_gzip/year=2009/month=2/000000_0.gz\t3.00KB\tyear=2009/month=2'] = <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at 0x7f94f94fab10>.data
      

      Attachments

        Issue Links

          Activity

            fangyurao Fang-Yu Rao added a comment -

            Hi MikaelSmith, assigned this to you since you were the author of IMPALA-11562. Please reassign the JIRA as you see appropriate. Thanks!

            fangyurao Fang-Yu Rao added a comment - Hi MikaelSmith , assigned this to you since you were the author of IMPALA-11562 . Please reassign the JIRA as you see appropriate. Thanks!
            MikaelSmith Michael Smith added a comment -

            That's surprising. I don't see any obvious reason those changes would have impacted this test at all.

            MikaelSmith Michael Smith added a comment - That's surprising. I don't see any obvious reason those changes would have impacted this test at all.

            Commit 4638a0e585a76e4a0b0aebf40eb7e4c34e2944d1 in impala's branch refs/heads/master from Michael Smith
            [ https://gitbox.apache.org/repos/asf?p=impala.git;h=4638a0e58 ]

            IMPALA-11646 IMPALA-11562: Fix test_unsupported_text_compression in s3

            Fixes test_unsupported_text_compression for S3 tests where hdfs_util.py
            was changed to expect a leading slash (and the pure HDFS implementation
            removes the leading slash). Skips removing the leading slash since HDFS
            now adds it and other filesystems expect the leading slash.

            Change-Id: I74724b2d5af0bd879d92d5e5febe92d01c972525
            Reviewed-on: http://gerrit.cloudera.org:8080/19119
            Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com>
            Reviewed-by: Fang-Yu Rao <fangyu.rao@cloudera.com>
            Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>

            jira-bot ASF subversion and git services added a comment - Commit 4638a0e585a76e4a0b0aebf40eb7e4c34e2944d1 in impala's branch refs/heads/master from Michael Smith [ https://gitbox.apache.org/repos/asf?p=impala.git;h=4638a0e58 ] IMPALA-11646 IMPALA-11562 : Fix test_unsupported_text_compression in s3 Fixes test_unsupported_text_compression for S3 tests where hdfs_util.py was changed to expect a leading slash (and the pure HDFS implementation removes the leading slash). Skips removing the leading slash since HDFS now adds it and other filesystems expect the leading slash. Change-Id: I74724b2d5af0bd879d92d5e5febe92d01c972525 Reviewed-on: http://gerrit.cloudera.org:8080/19119 Reviewed-by: Joe McDonnell <joemcdonnell@cloudera.com> Reviewed-by: Fang-Yu Rao <fangyu.rao@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>

            People

              MikaelSmith Michael Smith
              fangyurao Fang-Yu Rao
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: