[IMPALA-7236] Erasure coding dataload broken by IMPALA-7102

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 3.1.0
• Fix Version/s: Impala 3.1.0
• Component/s: Infrastructure
• Labels: broken-build
• Target Version: Impala 3.1.0
• Epic Color: ghx-label-3
• Assignee: Tianyi Wang
• Reporter: Joe McDonnell
• Created: 02/Jul/18 22:58
• Updated: 07/Mar/19 23:49
• Resolved: 05/Jul/18 18:09

Description

Loading nested data now fails on erasure coding:

16:58:45 Loading nested data (logging to /data/jenkins/workspace/impala-asf-master-core-erasure-coding/repos/Impala/logs/data_loading/load-nested.log)... 
17:01:22     FAILED (Took: 2 min 37 sec)
17:01:22     '/data/jenkins/workspace/impala-asf-master-core-erasure-coding/repos/Impala/testdata/bin/load_nested.py' failed. Tail of log:
17:01:22 2018-06-30 16:59:54,241 INFO:load_nested[277]:Executing: 
17:01:22 
17:01:22         CREATE TABLE customer
17:01:22         STORED AS PARQUET
17:01:22         TBLPROPERTIES('parquet.compression'='SNAPPY')
17:01:22         AS SELECT * FROM tmp_customer
17:01:22 2018-06-30 17:01:00,454 INFO:load_nested[277]:Executing: 
17:01:22 
17:01:22         CREATE TABLE region
17:01:22         STORED AS PARQUET
17:01:22         TBLPROPERTIES('parquet.compression'='SNAPPY')
17:01:22         AS SELECT * FROM tmp_region
17:01:22 2018-06-30 17:01:01,744 INFO:load_nested[277]:Executing: 
17:01:22 
17:01:22         CREATE TABLE supplier
17:01:22         STORED AS PARQUET
17:01:22         TBLPROPERTIES('parquet.compression'='SNAPPY')
17:01:22         AS SELECT * FROM tmp_supplier
17:01:22 2018-06-30 17:01:06,033 INFO:load_nested[296]:Executing: 
17:01:22         DROP TABLE tmp_orders_string
17:01:22 2018-06-30 17:01:06,097 INFO:load_nested[296]:Executing: 
17:01:22         DROP TABLE tmp_customer_string
17:01:22 2018-06-30 17:01:06,127 INFO:load_nested[296]:Executing: 
17:01:22         DROP TABLE tmp_customer
17:01:22 2018-06-30 17:01:10,067 INFO:load_nested[296]:Executing: 
17:01:22 
17:01:22         DROP TABLE tmp_region_string
17:01:22 2018-06-30 17:01:10,097 INFO:load_nested[296]:Executing: 
17:01:22         DROP TABLE tmp_region
17:01:22 2018-06-30 17:01:14,071 INFO:load_nested[296]:Executing: 
17:01:22 
17:01:22         DROP TABLE tmp_supplier
17:01:22 2018-06-30 17:01:18,075 INFO:load_nested[296]:Executing: 
17:01:22         DROP TABLE tmp_supplier_string
17:01:22 Traceback (most recent call last):
17:01:22   File "/data/jenkins/workspace/impala-asf-master-core-erasure-coding/repos/Impala/testdata/bin/load_nested.py", line 331, in <module>
17:01:22     load()
17:01:22   File "/data/jenkins/workspace/impala-asf-master-core-erasure-coding/repos/Impala/testdata/bin/load_nested.py", line 303, in load
17:01:22     impala.compute_stats()
17:01:22   File "/data/jenkins/workspace/impala-asf-master-core-erasure-coding/repos/Impala/tests/comparison/db_connection.py", line 760, in compute_stats
17:01:22     self.execute("COMPUTE STATS %s" % table_name)
17:01:22   File "/data/jenkins/workspace/impala-asf-master-core-erasure-coding/repos/Impala/tests/comparison/db_connection.py", line 206, in execute
17:01:22     return self._cursor.execute(sql, *args, **kwargs)
17:01:22   File "/data/jenkins/workspace/impala-asf-master-core-erasure-coding/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/hiveserver2.py", line 304, in execute
17:01:22     self._wait_to_finish()  # make execute synchronous
17:01:22   File "/data/jenkins/workspace/impala-asf-master-core-erasure-coding/repos/Impala/infra/python/env/lib/python2.7/site-packages/impala/hiveserver2.py", line 380, in _wait_to_finish
17:01:22     raise OperationalError(resp.errorMessage)
17:01:22 impala.error.OperationalError: ImpalaRuntimeException: Scanning of HDFS erasure-coded file (hdfs://localhost:20500/test-warehouse/tpch_nested_parquet.db/customer/000000_0) is not supported
17:01:22