[FLUME-3056] TestApplication hangs indefinitely

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.8.1, 1.9.0
• Component/s: Configuration, Test
• Assignee: Andras Beni
• Reporter: Andras Beni
• Created: 20/Feb/17 13:26
• Updated: 09/Mar/18 12:20
• Resolved: 09/Mar/18 11:08

Description

Unit test hangs indefinitely when TestApplication.testFLUME1854() becomes blocked in the following, deadlock-like situation:
Application waits for PollingPropertiesFileConfigurationProvider to stop while PollingPropertiesFileConfigurationProvider tries to notify Application of configuration change.

"conf-file-poller-0" #17750 prio=5 os_prio=31 tid=0x00007fdeb7972000 nid=0x638f waiting for monitor entry [0x000070000eb36000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.apache.flume.node.Application.handleConfigurationEvent(Application.java:87)
	- waiting to lock <0x000000077a130178> (a org.apache.flume.node.Application)
	at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68)
	at com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45)
	- locked <0x000000077a1301b0> (a com.google.common.eventbus.SynchronizedEventHandler)
	at com.google.common.eventbus.EventBus.dispatch(EventBus.java:313)
	at com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296)
	at com.google.common.eventbus.EventBus.post(EventBus.java:264)
	at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:141)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
	- <0x000000077a130250> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"main" #1 prio=5 os_prio=31 tid=0x00007fdeb9000000 nid=0x1b03 waiting on condition [0x000070000d779000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c253e9f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1465)
	at java.util.concurrent.Executors$DelegatedExecutorService.awaitTermination(Executors.java:675)
	at org.apache.flume.node.PollingPropertiesFileConfigurationProvider.stop(PollingPropertiesFileConfigurationProvider.java:88)
	at org.apache.flume.lifecycle.LifecycleSupervisor.stop(LifecycleSupervisor.java:104)
	- locked <0x000000077a138158> (a org.apache.flume.lifecycle.LifecycleSupervisor)
	at org.apache.flume.node.Application.stop(Application.java:92)
	- locked <0x000000077a130178> (a org.apache.flume.node.Application)
	at org.apache.flume.node.TestApplication.testFLUME1854(TestApplication.java:155)

Issue Links

• incorporates: FLUME-2786 It will enter a deadlock state when modify the conf file before I stop flume-ng
• incorporates: FLUME-3117 Application can be dead loop when call System.exit() in methodconfigure
• is related to: FLUME-1854 Application class can deadlock if stopped immediately after start
• links to: GitHub Pull Request #108