Uploaded image for project: 'Apache Knox'
  1. Apache Knox
  2. KNOX-1139

Tthread pool size in tests are not enough in a 32 core machine if gateway port mapping enabled

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: Tests
    • Labels:
      None

      Description

      Currently Jetty calculates acceptor and selector count as a fraction of processor count (core/8 and core/2 respectively) and will maximize them as 4. That means 4 acceptor and 4 select in a 32+ core machine for each ServerConnector. In Server.doStart jetty will check if maxThreadPoolSize is large enough to store a request thread and all acceptors and selectors thread for each connector together.
      Currently in tests thread pool max size fixed as 16. In GatewayServer.start if gateway port mapping is enabled it will initialize two connectors in jetty which on a 32+ core machine will create 4+4 acceptor and 4+4 selector. With the one extra request thread it will exceed the 16 thread pool limit and org.apache.hadoop.gateway.GatewayPortMappingFuncTest will throw the following exception:

      java.lang.IllegalStateException: Insufficient threads: max=16 < needed(acceptors=8 + selectors=8 + request=1)
      	at org.apache.hadoop.gateway.GatewayPortMappingFuncTest.setup(GatewayPortMappingFuncTest.java:96)
      

      This is only a unit test problem since the default value for thread pool size in GatewayConfigImpl is 254.

      Probably it would be a better method to calculate thread pool count from core count or max thread count in tests or just use the default value from GatewayConfigImpl.

        Attachments

        1. KNOX-1139.patch
          1.0 kB
          Attila Csoma

          Activity

            People

            • Assignee:
              csomaati Attila Csoma
              Reporter:
              csomaati Attila Csoma
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: