Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7490

Uninitialized variable in data-load.py causes misleading error messages

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Epic Color:
      ghx-label-1

      Description

      I found the following messages in a failed data-loading.

      19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-hbase-none-none.sql
      19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-rc-snap-block.sql
      19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-text-none-none.sql
      19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-avro-snap-block.sql
      19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-kudu-none-none.sql
      19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-orc-def-block.sql
      19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-seq-snap-block.sql
      19:12:38 Beginning execution of impala SQL: /mnt/volume1/impala-orc/incubator-impala/logs/data_loading/sql/functional/create-functional-query-core-impala-generated-parquet-none-none.sql
      Traceback (most recent call last):
        File "bin/load-data.py", line 495, in <module>
          if __name__ == "__main__": main()
        File "bin/load-data.py", line 459, in main
          impala_exec_query_files_parallel(thread_pool, impala_create_files)
        File "bin/load-data.py", line 297, in impala_exec_query_files_parallel
          exec_query_files_parallel(thread_pool, query_files, 'impala')
        File "bin/load-data.py", line 291, in exec_query_files_parallel
          for result in thread_pool.imap_unordered(execution_function, query_files):
        File "/usr/lib/python2.7/multiprocessing/pool.py", line 659, in next
          raise value
      UnboundLocalError: local variable 'query' referenced before assignment
      

      The error is misleading and actually don't tell anything (e.g. log file locations) for debug. The real cause for my error is that my Impala cluster failed to launch thus causes bin/load-data.py to fail.

      The 'query' variable is used in the Exception clause inside 'exec_impala_query_from_file'. We should judge whether it's initialized.

        Attachments

          Activity

            People

            • Assignee:
              stigahuang Quanlong Huang
              Reporter:
              stigahuang Quanlong Huang

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment