[IMPALA-7490] Uninitialized variable in data-load.py causes misleading error messages

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 3.0, Impala 2.12.0
• Fix Version/s: Impala 2.12.0, Impala 3.1.0
• Component/s: Infrastructure
• Epic Color: ghx-label-1
• Assignee: Quanlong Huang
• Reporter: Quanlong Huang
• Created: 26/Aug/18 02:35
• Updated: 21/Sep/18 06:52
• Resolved: 21/Sep/18 06:52

Description

I found the following messages in a failed data-loading.

19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-hbase-none-none.sql
19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-rc-snap-block.sql
19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-text-none-none.sql
19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-avro-snap-block.sql
19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-kudu-none-none.sql
19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-orc-def-block.sql
19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-seq-snap-block.sql
19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-parquet-none-none.sql
Traceback (most recent call last):
  File "bin/load-data.py", line 495, in <module>
    if __name__ == "__main__": main()
  File "bin/load-data.py", line 459, in main
    impala_exec_query_files_parallel(thread_pool, impala_create_files)
  File "bin/load-data.py", line 297, in impala_exec_query_files_parallel
    exec_query_files_parallel(thread_pool, query_files, 'impala')
  File "bin/load-data.py", line 291, in exec_query_files_parallel
    for result in thread_pool.imap_unordered(execution_function, query_files):
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 659, in next
    raise value
UnboundLocalError: local variable 'query' referenced before assignment

The error is misleading and actually don't tell anything (e.g. log file locations) for debug. The real cause for my error is that my Impala cluster failed to launch thus causes bin/load-data.py to fail.

The 'query' variable is used in the Exception clause inside 'exec_impala_query_from_file'. We should judge whether it's initialized.