Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31631

Fix test flakiness caused by MiniKdc which throws "address in use" BindException

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 3.1.0
    • Fix Version/s: 3.0.0
    • Component/s: Tests
    • Labels:
      None

      Description

      [info] org.apache.spark.sql.kafka010.KafkaDelegationTokenSuite *** ABORTED *** (15 seconds, 426 milliseconds)
      [info]   java.net.BindException: Address already in use
      [info]   at sun.nio.ch.Net.bind0(Native Method)
      [info]   at sun.nio.ch.Net.bind(Net.java:433)
      [info]   at sun.nio.ch.Net.bind(Net.java:425)
      [info]   at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
      [info]   at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
      [info]   at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:198)
      [info]   at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:51)
      [info]   at org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:547)
      [info]   at org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$400(AbstractPollingIoAcceptor.java:68)
      [info]   at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:422)
      [info]   at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
      [info]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      [info]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      [info]   at java.lang.Thread.run(Thread.java:748)
      

      This is an issue fixed in hadoop 2.8.0
      https://issues.apache.org/jira/browse/HADOOP-12656

      We may need apply the approach from HBASE first before we drop Hadoop 2.7.x

      https://issues.apache.org/jira/browse/HBASE-14734

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Qin Yao Kent Yao
                Reporter:
                Qin Yao Kent Yao
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: