Uploaded image for project: 'HttpComponents HttpCore'
  1. HttpComponents HttpCore
  2. HTTPCORE-631

HttpAsyncClient 5.0 file descriptor leak

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 5.0
    • Fix Version/s: 5.0.1
    • Component/s: HttpCore NIO
    • Labels:
      None

      Description

      It seems that HttpAsyncClient doesn't close all its file descriptors with the close() operation. Consider the following test:

          @Test
          public void leakApache() throws Exception {
              for (int i = 0; i < 10000; i++) {
                  CloseableHttpAsyncClient client = null;
                  try {
                      client = HttpAsyncClientBuilder.create().build();
                      System.out.println(i);
                  } catch (Throwable e) {
                      e.printStackTrace();
                  } finally {
                      if (client != null)
                          client.close();
                  }
                  Thread.sleep(10);
              }
      
              Thread.sleep(300000);
          }
       

      After around 280 iterations I start seeing this error message (java.io.IOException: Too many open files):

      java.lang.IllegalStateException: Unexpected failure opening I/O selector
       at org.apache.hc.core5.reactor.AbstractSingleCoreIOReactor.<init>(AbstractSingleCoreIOReactor.java:60)
       at org.apache.hc.core5.reactor.SingleCoreIOReactor.<init>(SingleCoreIOReactor.java:81)
       at org.apache.hc.core5.reactor.DefaultConnectingIOReactor.<init>(DefaultConnectingIOReactor.java:71)
       at org.apache.hc.client5.http.impl.async.HttpAsyncClientBuilder.build(HttpAsyncClientBuilder.java:924)
       at com.mimecast.micro.rest.BenTest.leakApache(BenTest.java:77)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
       at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
       at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
       at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
       at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
       at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
       at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
       at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
       at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
       at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
       at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
       at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
       at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
       at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
       at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
       at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
       at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
       at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
       Caused by: java.io.IOException: Too many open files
       at sun.nio.ch.IOUtil.makePipe(Native Method)
       at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
       at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
       at java.nio.channels.Selector.open(Selector.java:227)
       at org.apache.hc.core5.reactor.AbstractSingleCoreIOReactor.<init>(AbstractSingleCoreIOReactor.java:58)
       ... 26 more 

      If I run lsof i see around 10k files. These files do not get released until the whole test closes.

      Is there something I'm missing?

      Thanks,
      Ben

        Attachments

        1. apacheleaktest.tar
          2.20 MB
          Ben Tindall

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              bentindall Ben Tindall
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: