[SOLR-13871] JettySolrRunner should stop trying to re-use ports on re-start

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Chris M. Hostetter
• Created: 24/Oct/19 19:56
• Updated: 02/Nov/20 19:07

Description

JettySolrRunner currently has special logic in it's start() method that will cause it (by default) to try to rebind to the port it was previously assigned if it's restarting and configured with port '0'

ie: the first time it starts the OS assigns the port, after that it tries to re-use that same port.

This is a bad idea in general, and leads to (very slow) BindException failures in a lot of jenkins tests where nodes are restarted.

Example...

   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ShardSplitTest -Dtests.method=testSplitWithChaosMonkey -Dtests.seed=9097C20E8E9ACC68 -Dtes
ts.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test
-data/enwiki.random.lines.txt -Dtests.locale=so-SO -Dtests.timezone=Etc/GMT+9 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   81.2s J1 | ShardSplitTest.testSplitWithChaosMonkey <<<
   [junit4]    > Throwable #1: java.net.BindException: Address already in use
   [junit4]    >        at __randomizedtesting.SeedInfo.seed([9097C20E8E9ACC68:1BB011DFCF9C67EC]:0)
   [junit4]    >        at java.base/sun.nio.ch.Net.bind0(Native Method)
   [junit4]    >        at java.base/sun.nio.ch.Net.bind(Net.java:461)
   [junit4]    >        at java.base/sun.nio.ch.Net.bind(Net.java:453)
   [junit4]    >        at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
   [junit4]    >        at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
   [junit4]    >        at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
   [junit4]    >        at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:308)
   [junit4]    >        at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
   [junit4]    >        at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
   [junit4]    >        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
   [junit4]    >        at org.eclipse.jetty.server.Server.doStart(Server.java:396)
   [junit4]    >        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
   [junit4]    >        at org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:569)
   [junit4]    >        at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:508)
   [junit4]    >        at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:476)
   [junit4]    >        at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:499)

Ideally JettySolrRunner's default behavior should b to just trust it's config – binding to a random port (even on restart, even if diff from it's previous port) if configured with '0'.

Callers – including tests – should eliminate any assumptions in their code that the port will be consistent for the life of a JettySolrRunner (ie: accept that the URL may change anytime stop/start is called)

Issue Links

• relates to: SOLR-14979 'nodeName' should be configurable as a startup option/system property