Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22172

Worker hangs when the external shuffle service port is already in use

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      When the external shuffle service port is already in use, Worker throws the below BindException and hangs forever, I think the exception should be handled gracefully.

      17/09/29 11:16:30 INFO ExternalShuffleService: Starting shuffle service on port 7337 (auth enabled = false)
      17/09/29 11:16:30 ERROR Inbox: Ignoring error
      java.net.BindException: Address already in use
              at sun.nio.ch.Net.bind0(Native Method)
              at sun.nio.ch.Net.bind(Net.java:433)
              at sun.nio.ch.Net.bind(Net.java:425)
              at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
              at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128)
              at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:500)
              at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1218)
              at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:495)
              at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:480)
              at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:965)
              at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:209)
              at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:355)
              at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
      
      

        Attachments

          Activity

            People

            • Assignee:
              devaraj.k Devaraj K
              Reporter:
              devaraj.k Devaraj K
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: