XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.0, 3.2.2, 3.4.0
    • Component/s: capacityscheduler
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Validation API creates a new CapacityScheduler and missed to close after the validation. Every CapacityScheduler#init opens MutableCSConfigurationProvider which opens ZKConfigurationStore and creates a ZK Connection.

      ZK LOGS

      -03-12 16:45:51,881 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [2 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:52,449 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:52,710 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:52,876 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [4 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:53,068 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [2 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:53,391 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [2 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:54,008 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:54,287 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:54,483 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [4 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      

      And there is an another bug in ZKConfigurationStore which has not handled close() of ZKCuratorManager.

        Attachments

        1. YARN-10194-001.patch
          7 kB
          Prabhu Joseph
        2. YARN-10194-002.patch
          7 kB
          Prabhu Joseph
        3. YARN-10194-003.patch
          4 kB
          Prabhu Joseph
        4. YARN-10194-004.patch
          4 kB
          Prabhu Joseph
        5. YARN-10194-005.patch
          7 kB
          Prabhu Joseph
        6. YARN-10194-branch-3.2.001.patch
          7 kB
          Prabhu Joseph

          Activity

            People

            • Assignee:
              prabhujoseph Prabhu Joseph
              Reporter:
              akhilpb Akhil PB
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: