[BEAM-3119] direct-metrics-counter-committer threads are leaking

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.5.0
• Component/s: runner-direct
• Assignee: Romain Manni-Bucau
• Reporter: Etienne Chauchot
• Created: 30/Oct/17 14:12
• Updated: 16/May/20 13:49
• Resolved: 30/Apr/18 14:16

Description

When I run ElasticsearchIOTests using ESv5, there is a thread leak control mechanism (com.carrotsearch.randomizedtesting.ThreadLeakControl). It waits for 5s for non-terminated threads at the end of a test. It detects leaked direct-metrics-counter-committer thread.

com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=296, name=direct-metrics-counter-committer, state=TIMED_WAITING, group=TGRP-ElasticsearchIOTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

	at __randomizedtesting.SeedInfo.seed([59E504CA1B0DD6A8]:0)

I tried to increase the timeout to 30s (by patching randomizedtesting-runner-2.5.0.jar) but still gets a zombie thread.

To reproduce, just comment

@ThreadLeakScope(ThreadLeakScope.Scope.NONE)

in

beam/sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTest.java

and run

testRead()

Issue Links

• links to: GitHub Pull Request #4965