[SOLR-7113] Multiple calls to UpdateLog#init is not thread safe with respect to the HDFS FileSystem client object usage.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.1, 6.0
• Assignee: Mark Miller
• Reporter: Vamsee K. Yarlagadda
• Created: 14/Feb/15 22:00
• Updated: 14/Jun/16 11:01
• Resolved: 02/Apr/15 16:14

Description

I notice this issue while trying to do some heavy indexing into Solr. (700K docs per minute)

Solr log errors

15:42:47
ERROR
HdfsTransactionLog
Exception closing tlog.
java.io.IOException: Filesystem closed
	at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:765)
	at org.apache.hadoop.hdfs.DFSOutputStream.flushOrSync(DFSOutputStream.java:1898)
	at org.apache.hadoop.hdfs.DFSOutputStream.hflush(DFSOutputStream.java:1859)
	at org.apache.hadoop.fs.FSDataOutputStream.hflush(FSDataOutputStream.java:130)
	at org.apache.solr.update.HdfsTransactionLog.close(HdfsTransactionLog.java:303)
	at org.apache.solr.update.TransactionLog.decref(TransactionLog.java:504)
	at org.apache.solr.update.UpdateLog.addOldLog(UpdateLog.java:335)
	at org.apache.solr.update.UpdateLog.postCommit(UpdateLog.java:628)
	at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:600)
	at org.apache.solr.update.CommitTracker.run(CommitTracker.java:216)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
15:42:47
ERROR
CommitTracker
auto commit error...:org.apache.solr.common.SolrException: java.io.IOException: Filesystem closed
auto commit error...:org.apache.solr.common.SolrException: java.io.IOException: Filesystem closed

Issue Links

• is related to: SOLR-7115 UpdateLog can miss closing transaction log objects.