[IMPALA-4948] Concurrent refresh against the same table fails with ConcurrentModificationException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: Impala 2.9.0
• Component/s: Catalog
• Labels: catalog-server scalability
• Assignee: Alexander Behm
• Reporter: Mostafa Mokhtar
• Created: 17/Feb/17 02:44
• Updated: 24/Feb/17 16:06
• Resolved: 24/Feb/17 06:22

Description

Concurrent refresh statements against the same table fail with exception below

I0216 18:34:13.343709 158318 catalog-server.cc:316] Publishing update: CATALOG:2b3e096917a145d3:82177a82fe8dce0f@35450
I0216 18:34:20.475253 193290 jni-util.cc:169] java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922)
	at java.util.HashMap$EntryIterator.next(HashMap.java:962)
	at java.util.HashMap$EntryIterator.next(HashMap.java:960)
	at org.apache.impala.thrift.THdfsPartition$THdfsPartitionStandardScheme.write(THdfsPartition.java:1832)
	at org.apache.impala.thrift.THdfsPartition$THdfsPartitionStandardScheme.write(THdfsPartition.java:1546)
	at org.apache.impala.thrift.THdfsPartition.write(THdfsPartition.java:1389)
	at org.apache.impala.thrift.THdfsTable$THdfsTableStandardScheme.write(THdfsTable.java:1243)
	at org.apache.impala.thrift.THdfsTable$THdfsTableStandardScheme.write(THdfsTable.java:1071)
	at org.apache.impala.thrift.THdfsTable.write(THdfsTable.java:940)
	at org.apache.impala.thrift.TTable$TTableStandardScheme.write(TTable.java:1628)
	at org.apache.impala.thrift.TTable$TTableStandardScheme.write(TTable.java:1399)
	at org.apache.impala.thrift.TTable.write(TTable.java:1208)
	at org.apache.impala.thrift.TCatalogObject$TCatalogObjectStandardScheme.write(TCatalogObject.java:1241)
	at org.apache.impala.thrift.TCatalogObject$TCatalogObjectStandardScheme.write(TCatalogObject.java:1098)
	at org.apache.impala.thrift.TCatalogObject.write(TCatalogObject.java:938)
	at org.apache.impala.thrift.TCatalogUpdateResult$TCatalogUpdateResultStandardScheme.write(TCatalogUpdateResult.java:995)
	at org.apache.impala.thrift.TCatalogUpdateResult$TCatalogUpdateResultStandardScheme.write(TCatalogUpdateResult.java:867)
	at org.apache.impala.thrift.TCatalogUpdateResult.write(TCatalogUpdateResult.java:748)
	at org.apache.impala.thrift.TResetMetadataResponse$TResetMetadataResponseStandardScheme.write(TResetMetadataResponse.java:359)
	at org.apache.impala.thrift.TResetMetadataResponse$TResetMetadataResponseStandardScheme.write(TResetMetadataResponse.java:321)
	at org.apache.impala.thrift.TResetMetadataResponse.write(TResetMetadataResponse.java:269)
	at org.apache.thrift.TSerializer.serialize(TSerializer.java:79)
	at org.apache.impala.service.JniCatalog.resetMetadata(JniCatalog.java:156)
I0216 18:34:20.488674 193290 status.cc:114] ConcurrentModificationException: null
    @           0x837a89  impala::Status::Status()
    @           0xb80420  impala::JniUtil::GetJniExceptionMsg()
    @           0x82557b  impala::Catalog::ResetMetadata()
    @           0x818e31  CatalogServiceThriftIf::ResetMetadata()
    @           0x8ee85d  impala::CatalogServiceProcessor::process_ResetMetadata()
    @           0x8ebc09  impala::CatalogServiceProcessor::dispatchCall()
    @           0x803f2c  apache::thrift::TDispatchProcessor::process()
    @           0x9f858f  apache::thrift::server::TAcceptQueueServer::Task::run()
    @           0x9f2bb9  impala::ThriftThread::RunRunnable()
    @           0x9f3612  boost::detail::function::void_function_obj_invoker0<>::invoke()
    @           0xbd39c9  impala::Thread::SuperviseThread()
    @           0xbd43a4  boost::detail::thread_data<>::run()
    @           0xe19f1a  thread_proxy
    @       0x348e407aa1  (unknown)
    @       0x348e0e893d  (unknown)
E0216 18:34:20.488878 193290 catalog-server.cc:79] ConcurrentModificationException: null

Issue Links

• is related to: IMPALA-4902 Concurrent DDL may fail with a ConcurrentModificationException