Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10462

TestIcebergTable::test_create_iceberg_tables fails with ClassNotFoundException on newer Hive/Iceberg

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • Impala 4.0.0
    • Impala 4.0.0
    • Frontend
    • None
    • ghx-label-1

    Description

      When testing against a newer version of Hive/Iceberg, TestIcebergTable.test_create_iceberg_tables fails with a ClassNotFoundException forĀ org.apache.hadoop.hive.common.type.Date:

      23:35:39  TestIcebergTable.test_create_iceberg_tables[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] 
      23:35:39 [gw5] linux2 -- Python 2.7.16 /data/jenkins/workspace/impala-private-basic-parameterized/repos/Impala/bin/../infra/python/env-gcc7.5.0/bin/python
      23:35:39 query_test/test_iceberg.py:47: in test_create_iceberg_tables
      23:35:39     self.run_test_case('QueryTest/iceberg-create', vector, use_db=unique_database)
      23:35:39 common/impala_test_suite.py:662: in run_test_case
      23:35:39     result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
      23:35:39 common/impala_test_suite.py:600: in __exec_in_impala
      23:35:39     result = self.__execute_query(target_impalad_client, query, user=user)
      23:35:39 common/impala_test_suite.py:920: in __execute_query
      23:35:39     return impalad_client.execute(query, user=user)
      23:35:39 common/impala_connection.py:205: in execute
      23:35:39     return self.__beeswax_client.execute(sql_stmt, user=user)
      23:35:39 beeswax/impala_beeswax.py:187: in execute
      23:35:39     handle = self.__execute_query(query_string.strip(), user=user)
      23:35:39 beeswax/impala_beeswax.py:363: in __execute_query
      23:35:39     handle = self.execute_query_async(query_string, user=user)
      23:35:39 beeswax/impala_beeswax.py:357: in execute_query_async
      23:35:39     handle = self.__do_rpc(lambda: self.imp_service.query(query,))
      23:35:39 beeswax/impala_beeswax.py:520: in __do_rpc
      23:35:39     raise ImpalaBeeswaxException(self.__build_error_message(b), b)
      23:35:39 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      23:35:39 E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
      23:35:39 E    MESSAGE: NoClassDefFoundError: org/apache/hadoop/hive/common/type/Date
      23:35:39 E   CAUSED BY: ClassNotFoundException: org.apache.hadoop.hive.common.type.Date

      Maybe org.apache.hadoop.hive.common.type.* needs to be added to the impala-minimal-hive-exec shaded jar.

      https://github.com/apache/impala/blob/master/java/shaded-deps/hive-exec/pom.xml#L34

      Attachments

        Activity

          People

            joemcdonnell Joe McDonnell
            joemcdonnell Joe McDonnell
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: