Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9136

IllegalStateException in calling Table.toThrift() without holding the table lock

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 3.2.0, Impala 3.3.0
    • Fix Version/s: Impala 3.4.0
    • Component/s: Catalog
    • Labels:
      None
    • Epic Color:
      ghx-label-1

      Description

      In CatalogServiceCatalog.addTableToCatalogDelta(), we are calling tbl.getUniqueName() without holding the table lock, which could finally results in an IllegalStateException:

      I1107 17:34:27.058262  6381 CatalogServiceCatalog.java:1118] Table test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_14_part is skipping topic update 423
      I1107 17:34:27.058709  6381 catalog-server.cc:641] Collected update: 1:TABLE:test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_15_part, version=422, original size=2299, compressed size=1219
      I1107 17:34:27.061233  6381 jni-util.cc:288] java.lang.IllegalStateException: Table.toThrift() called without holding the table lock: test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_12_part org.apache.impala.catalog.HdfsTable
              at org.apache.impala.catalog.Table.toThrift(Table.java:448)
              at org.apache.impala.catalog.HdfsTable.toThrift(HdfsTable.java:1408)
              at org.apache.impala.catalog.Table.setTCatalogObject(Table.java:487)
              at org.apache.impala.catalog.CatalogObjectImpl.toTCatalogObject(CatalogObjectImpl.java:51)
              at org.apache.impala.catalog.CatalogObjectImpl.getUniqueName(CatalogObjectImpl.java:45)
              at org.apache.impala.catalog.CatalogServiceCatalog.addTableToCatalogDelta(CatalogServiceCatalog.java:1084)
              at org.apache.impala.catalog.CatalogServiceCatalog.addDatabaseToCatalogDelta(CatalogServiceCatalog.java:961)
              at org.apache.impala.catalog.CatalogServiceCatalog.getCatalogDelta(CatalogServiceCatalog.java:710)
              at org.apache.impala.service.JniCatalog.getCatalogDelta(JniCatalog.java:148)
      I1107 17:34:27.129712  7356 HdfsTable.java:934] Reloading metadata for all partition(s) of test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_16 (INSERT)
      I1107 17:34:27.132397  7356 HdfsTable.java:630] Loaded file and block metadata for test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_16 partitions:
      I1107 17:34:27.132424  7356 HdfsTable.java:965] Incrementally loaded table metadata for: test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_16
      I1107 17:34:27.157740  6381 status.cc:126] IllegalStateException: Table.toThrift() called without holding the table lock: test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_12_part org.apache.impala.catalog.HdfsTable
          @          0x1c5ae50  impala::Status::Status()
          @          0x24f7ad2  impala::JniUtil::GetJniExceptionMsg()
          @          0x1c414af  impala::JniCall::Call<>()
          @          0x1c3fe27  impala::JniUtil::CallJniMethod<>()
          @          0x1c3e06d  impala::Catalog::GetCatalogDelta()
          @          0x1c14344  impala::CatalogServer::GatherCatalogUpdatesThread()
          @          0x1c2e14e  boost::_mfi::mf0<>::operator()()
          @          0x1c2da93  boost::_bi::list1<>::operator()<>()
          @          0x1c2d4a6  boost::_bi::bind_t<>::operator()()
          @          0x1c2cd91  boost::detail::function::void_function_obj_invoker0<>::invoke()
          @          0x1ffb6e9  boost::function0<>::operator()()
          @          0x2573dea  impala::Thread::SuperviseThread()
          @          0x257c16e  boost::_bi::list5<>::operator()<>()
          @          0x257c092  boost::_bi::bind_t<>::operator()()
          @          0x257c055  boost::detail::thread_data<>::run()
          @          0x3d61599  thread_proxy
          @     0x7f1ce6ca46b9  start_thread
          @     0x7f1ce343f41c  clone
      E1107 17:34:27.157758  6381 catalog-server.cc:379] IllegalStateException: Table.toThrift() called without holding the table lock: test_ddls_with_invalidate_metadata_sync_ddl_f41e97e6.test_12_part org.apache.impala.catalog.HdfsTable
      I1107 17:34:27.158537  6389 catalog-server.cc:337] A catalog update with 4 entries is assembled. Catalog version: 419 Last sent catalog version: 419
      
      

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                stigahuang Quanlong Huang
                Reporter:
                stigahuang Quanlong Huang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: