[DIRMINA-678] NioProcessor 100% CPU usage on Linux (epoll selector bug) - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 2.0.0-M4
Fix Version/s: 2.0.3
Component/s: Core
Labels:
None
Environment:
CentOS 5.x, 32/64-bit, 32/64-bit Sun JDK 1.6.0_12, also _11/_10/_09 and Sun JDK 1.7.0 b50, Kernel 2.6.18-92.1.22.el5 and also older versions,

Description

It's the same bug as described at http://jira.codehaus.org/browse/JETTY-937 , but affecting MINA in the very similar way.

NioProcessor threads start to eat 100% resources per CPU. After 10-30 minutes of running depending on the load (sometimes after several hours) one of the NioProcessor starts to consume all the available CPU resources probably spinning in the epoll select loop. Later, more threads can be affected by the same issue, thus 100% loading all the available CPU cores.

Sample trace:

NioProcessor-10 [RUNNABLE] CPU time: 5:15
sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int)
sun.nio.ch.EPollArrayWrapper.poll(long)
sun.nio.ch.EPollSelectorImpl.doSelect(long)
sun.nio.ch.SelectorImpl.lockAndDoSelect(long)
sun.nio.ch.SelectorImpl.select(long)
org.apache.mina.transport.socket.nio.NioProcessor.select(long)
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run()
org.apache.mina.util.NamePreservingRunnable.run()
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker)
java.util.concurrent.ThreadPoolExecutor$Worker.run()
java.lang.Thread.run()

It seems to affect any NIO based Java server applications running in the specified environment.
Some projects provide workarounds for similar JDK bugs, probably MINA can also think about a workaround.

As far as I know, there are at least 3 users who experience this issue with Jetty and all of them are running CentOS (some distribution default setting is a trigger?). As for MINA, I'm not aware of similar reports yet.