[THRIFT-5230] Fix connection leak and CancelledKeyException when handling Epoll bug

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.13.0
• Fix Version/s: 0.21.0
• Component/s: Java - Library

Environment

java version "1.8.0_161"

• Assignee: Jens Geyer
• Reporter: zengji
• Created: 07/Jun/20 14:51
• Updated: 21/Sep/24 13:30
• Resolved: 01/Aug/24 20:29

Description

1. When Epoll bug occurs, the TThreadedSelectorServer.rebuildSelector rebuilds only the channel has events, the idle connection was ignored and caused connection leak

 

for (SelectionKey key : oldSelector.selectedKeys()) {
  if (!key.isValid() && key.readyOps() == 0)
    continue;
  SelectableChannel channel = key.channel();
  Object attachment = key.attachment();

  try {
    if (attachment == null) {
      channel.register(newSelector, key.readyOps());
    } else {
      channel.register(newSelector, key.readyOps(), attachment);
    }
  } catch (ClosedChannelException e) {
    LOGGER.error("Register new selector key error.", e);
  }
}

selector = newSelector;
try {
  oldSelector.close();
} catch (IOException e) {
  LOGGER.error("Close old selector error.", e);
}

2. When re-register the channel to new selector, the interested ops should same as before, not only the readyOps

 

3. In the same code block, the channel will be registered to a new selector and the previous selector will be closed, but the FrameBuffer is still holding the previous selector causing the FrameBuffer in a wrong state. When the FrameBuffer is trying to processing the channel, it may occur a CancelledKeyException.This issue (CancelledKeyException) has been reported before:https://issues.apache.org/jira/browse/THRIFT-4847

Issue Links

• is related to: THRIFT-4251 Java Epoll Selector Bug
• is related to: THRIFT-4847 CancelledKeyException causes TThreadedSelectorServer to fail.
• links to: GitHub Pull Request #2171