Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7099

test_unsupported_text_compression fails: Expected one file per partition dir

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 2.13.0, Impala 3.1.0
    • None
    • None
    • ghx-label-7

    Description

      Seen in an S3 build at e9bd917a218b5e1717fced983f70f64850c6e02f:

      15:17:10  TestPartitionMetadataUncompressedTextOnly.test_unsupported_text_compression[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: text/none] 
      15:17:10 [gw2] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/bin/../infra/python/env/bin/python
      15:17:10 metadata/test_partition_metadata.py:203: in test_unsupported_text_compression
      15:17:10     assert len(show_files_result.data) == 4, "Expected one file per partition dir"
      15:17:10 E   AssertionError: Expected one file per partition dir
      15:17:10 E   assert 2 == 4
      15:17:10 E    +  where 2 = len(['s3a://impala-cdh5-s3-test/test-warehouse/alltypes/year=2009/month=1/090101.txt\t19.95KB\tyear=2009/month=1', 's3a://impala-cdh5-s3-test/test-warehouse/alltypes_text_lzo/year=2009/month=2/000023_0.lzo\t4.84KB\tyear=2009/month=2'])
      15:17:10 E    +    where ['s3a://impala-cdh5-s3-test/test-warehouse/alltypes/year=2009/month=1/090101.txt\t19.95KB\tyear=2009/month=1', 's3a://impala-cdh5-s3-test/test-warehouse/alltypes_text_lzo/year=2009/month=2/000023_0.lzo\t4.84KB\tyear=2009/month=2'] = <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at 0x6bcd790>.data
      15:17:10 ---------------------------- Captured stderr setup -----------------------------
      15:17:10 -- connecting to: localhost:21000
      15:17:10 SET sync_ddl=False;
      15:17:10 -- executing against localhost:21000
      15:17:10 DROP DATABASE IF EXISTS `test_unsupported_text_compression_695d360a` CASCADE;
      15:17:10 
      15:17:10 SET sync_ddl=False;
      15:17:10 -- executing against localhost:21000
      15:17:10 CREATE DATABASE `test_unsupported_text_compression_695d360a`;
      15:17:10 
      15:17:10 MainThread: Created database "test_unsupported_text_compression_695d360a" for test ID "metadata/test_partition_metadata.py::TestPartitionMetadataUncompressedTextOnly::()::test_unsupported_text_compression[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: text/none]"
      15:17:10 ----------------------------- Captured stderr call -----------------------------
      15:17:10 MainThread: Starting new HTTPS connection (1): impala-cdh5-s3-test.s3.amazonaws.com
      15:17:10 -- executing against localhost:21000
      15:17:10 create external table test_unsupported_text_compression_695d360a.multi_text_compression like functional.alltypes location 's3a://impala-cdh5-s3-test/test-warehouse/test_unsupported_text_compression_695d360a.db/multi_text_compression';
      15:17:10 
      15:17:10 -- executing against localhost:21000
      15:17:10 use functional;
      15:17:10 
      15:17:10 -- executing against localhost:21000
      15:17:10 describe formatted functional.alltypes;
      15:17:10 
      15:17:10 -- executing against localhost:21000
      15:17:10 alter table test_unsupported_text_compression_695d360a.multi_text_compression add partition (year=2009, month=1) location 's3a://impala-cdh5-s3-test/test-warehouse/alltypes/year=2009/month=1';
      15:17:10 
      15:17:10 -- executing against localhost:21000
      15:17:10 use functional;
      15:17:10 
      15:17:10 -- executing against localhost:21000
      15:17:10 describe formatted functional_text_lzo.alltypes;
      15:17:10 
      15:17:10 -- executing against localhost:21000
      15:17:10 alter table test_unsupported_text_compression_695d360a.multi_text_compression add partition (year=2009, month=2) location 's3a://impala-cdh5-s3-test/test-warehouse/alltypes_text_lzo/year=2009/month=2';
      15:17:10 
      15:17:10 MainThread: Resetting dropped connection: impala-cdh5-s3-test.s3.amazonaws.com
      15:17:10 -- executing against localhost:21000
      15:17:10 alter table test_unsupported_text_compression_695d360a.multi_text_compression add partition (year=2009, month=3) location 's3a://impala-cdh5-s3-test/test-warehouse/test_unsupported_text_compression_695d360a.db/multi_text_compression/year=2009/month=3';
      15:17:10 
      15:17:10 MainThread: Resetting dropped connection: impala-cdh5-s3-test.s3.amazonaws.com
      15:17:10 -- executing against localhost:21000
      15:17:10 alter table test_unsupported_text_compression_695d360a.multi_text_compression add partition (year=2009, month=4) location 's3a://impala-cdh5-s3-test/test-warehouse/test_unsupported_text_compression_695d360a.db/multi_text_compression/year=2009/month=4';
      15:17:10 
      15:17:10 -- executing against localhost:21000
      15:17:10 show files in test_unsupported_text_compression_695d360a.multi_text_compression;
      15:17:10 
      

      Assigning to Dan because you touched the test recently in IMPALA-7068, though it looks like a different issue

      Attachments

        Issue Links

          Activity

            People

              dhecht Daniel Hecht
              twmarshall Thomas Tauber-Marshall
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: