Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8599

Errors in construction of SolrZooKeeper cause Solr to go into an inconsistent state

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.5.1, 6.0
    • Component/s: SolrCloud
    • Labels:
      None

      Description

      We originally saw this happen due to a DNS exception (see stack trace below). Although any exception thrown in the constructor of SolrZooKeeper or the parent class, ZooKeeper, will cause DefaultConnectionStrategy to fail to update the zookeeper client. Once it gets into this state, it will not try to connect again until the process is restarted. The node itself will also respond successfully to query requests, but not to update requests.

      Two things should be address here:
      1) Fix the error handling and issue some number of retries
      2) If we are stuck in a state like this stop responding to all requests

      2016-01-23 13:49:20.222 ERROR ConnectionManager [main-EventThread] - :java.net.UnknownHostException: HOSTNAME: unknown error
      at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
      at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
      at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
      at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
      at java.net.InetAddress.getAllByName(InetAddress.java:1192)
      at java.net.InetAddress.getAllByName(InetAddress.java:1126)
      at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61)
      at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)
      at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
      at org.apache.solr.common.cloud.SolrZooKeeper.<init>(SolrZooKeeper.java:41)
      at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:53)
      at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:132)
      at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
      at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
      2016-01-23 13:49:20.222 INFO ConnectionManager [main-EventThread] - Connected:false
      2016-01-23 13:49:20.222 INFO ClientCnxn [main-EventThread] - EventThread shut down
      
      1. SOLR-8599.patch
        4 kB
        Keith Laban
      2. SOLR-8599.patch
        8 kB
        Dennis Gove
      3. SOLR-8599.patch
        5 kB
        Keith Laban
      4. SOLR-8599.patch
        7 kB
        Keith Laban

        Activity

        Hide
        k317h Keith Laban added a comment -

        Added a patch to address issue #1. In this patch I had to remove final clause for ZkServerAddress and add a public setter method in order to test the issue

        Show
        k317h Keith Laban added a comment - Added a patch to address issue #1. In this patch I had to remove final clause for ZkServerAddress and add a public setter method in order to test the issue
        Hide
        dpgove Dennis Gove added a comment - - edited

        This fails the forbidden api precommit check.

        [forbidden-apis] Forbidden method invocation: java.util.concurrent.Executors#newSingleThreadScheduledExecutor() [Spawns threads with vague names; use a custom thread factory (Lucene's NamedThreadFactory, Solr's DefaultSolrThreadFactory) and name threads so that you can tell (by its name) which executor it is associated with]
        [forbidden-apis]   in org.apache.solr.cloud.ConnectionManagerTest (ConnectionManagerTest.java:119)
        
        Show
        dpgove Dennis Gove added a comment - - edited This fails the forbidden api precommit check. [forbidden-apis] Forbidden method invocation: java.util.concurrent.Executors#newSingleThreadScheduledExecutor() [Spawns threads with vague names; use a custom thread factory (Lucene's NamedThreadFactory, Solr's DefaultSolrThreadFactory) and name threads so that you can tell (by its name) which executor it is associated with] [forbidden-apis] in org.apache.solr.cloud.ConnectionManagerTest (ConnectionManagerTest.java:119)
        Hide
        k317h Keith Laban added a comment -

        added usage of DefaultSolrThreadFactory to test

        Show
        k317h Keith Laban added a comment - added usage of DefaultSolrThreadFactory to test
        Hide
        dpgove Dennis Gove added a comment -

        I have somewhat of an interesting situation at hand here.

        As part of this patch a test is added to ConnectionManagerTest which forces a DNS failure on the zookeeper connection by attempting to connect to "BADADDRESS" and then fixing it after 5 seconds. This shows that the change Keith put in ConnectionManager will continually try to make a connection until it can. It's a good test and it exercises the bug and fix perfectly.

        However, the test depends on my ISP. I've run the test under 5 scenarios and only 3 of them pass.

        1. Connected to my corporate network
        In this scenario the test passes perfectly as it should.

        2. Connected to no network (ie, wifi card turned off)
        In this scenario the test passes perfectly as it should.

        3. Connected to my home network backed by Verizon FIOS
        In this scenario the test hangs and upon further investigation I found that it is in an "infinite" loop in ConnectionManager::waitForConnected. This appears to be an infinite loop because while there is a timeout the timeout is Long.MAX_VALUE. The problem here is that the loop waits until it is either connected or closed. Neither of those conditions are ever hit. But why? We're trying to hit http://BADADDRESS and clearly that is a DNS lookup failure. Oh no no no, not according to Verizon. See, Verizon instead says "Oh, you must've typed something in wrong so instead of returning to you a DNS failure let me return to you a redirect to a search page - you clearly want this search page". It appears that because of this redirection a connection is never made nor is it ever closed. Hence, loop forever.

        4. Connected to my personal wifi hotspot backed by T-Mobile
        Same issue as seen with Verizon FIOS, though a T-Mobile specific search page.

        5. Connected to a hotspot through my iPhone backed by Verizon Wireless
        In this scenario the test passes perfectly as it should.

        Note that this difference is only seen when a DNS lookup failure is in play. If I change the bad address to "http://BADADDRESS" then it fails instead because "//BADADDRESSIS" is said to be an invalid path string. Technically this is testing a slightly different case but I'm comfortable calling it the same test because the issue being corrected is a failure to make a connection during the construction of SolrZooKeeper and a malformed url fails just the same.

        Show
        dpgove Dennis Gove added a comment - I have somewhat of an interesting situation at hand here. As part of this patch a test is added to ConnectionManagerTest which forces a DNS failure on the zookeeper connection by attempting to connect to "BADADDRESS" and then fixing it after 5 seconds. This shows that the change Keith put in ConnectionManager will continually try to make a connection until it can. It's a good test and it exercises the bug and fix perfectly. However, the test depends on my ISP. I've run the test under 5 scenarios and only 3 of them pass. 1. Connected to my corporate network In this scenario the test passes perfectly as it should. 2. Connected to no network (ie, wifi card turned off) In this scenario the test passes perfectly as it should. 3. Connected to my home network backed by Verizon FIOS In this scenario the test hangs and upon further investigation I found that it is in an "infinite" loop in ConnectionManager::waitForConnected. This appears to be an infinite loop because while there is a timeout the timeout is Long.MAX_VALUE. The problem here is that the loop waits until it is either connected or closed. Neither of those conditions are ever hit. But why? We're trying to hit http://BADADDRESS and clearly that is a DNS lookup failure. Oh no no no, not according to Verizon. See, Verizon instead says "Oh, you must've typed something in wrong so instead of returning to you a DNS failure let me return to you a redirect to a search page - you clearly want this search page". It appears that because of this redirection a connection is never made nor is it ever closed. Hence, loop forever. 4. Connected to my personal wifi hotspot backed by T-Mobile Same issue as seen with Verizon FIOS, though a T-Mobile specific search page. 5. Connected to a hotspot through my iPhone backed by Verizon Wireless In this scenario the test passes perfectly as it should. Note that this difference is only seen when a DNS lookup failure is in play. If I change the bad address to "http://BADADDRESS" then it fails instead because "//BADADDRESSIS" is said to be an invalid path string. Technically this is testing a slightly different case but I'm comfortable calling it the same test because the issue being corrected is a failure to make a connection during the construction of SolrZooKeeper and a malformed url fails just the same.
        Hide
        markrmiller@gmail.com Mark Miller added a comment -

        Yeah, we try to avoid that in tests due to dns hijacking. I think there is an ip6 address that works for this? Uwe Schindler, am I remembering right?

        Show
        markrmiller@gmail.com Mark Miller added a comment - Yeah, we try to avoid that in tests due to dns hijacking. I think there is an ip6 address that works for this? Uwe Schindler , am I remembering right?
        Hide
        thetaphi Uwe Schindler added a comment - - edited

        Hi,
        please don't write tests that rely on DNS failures. Every local provider does something else, but not all return a correct "not found". Unless we have a "mock DNS server", we cannot use any DNS in Solr tests. For similar reasons we also do not use "localhost" as hostname, but instead use 127.0.0.1 as IP address.

        To have some broken address that always fails to connect to, use an IPv6 address that is standardized to be routed nowhere and already refused by the Hosts's IPv6 stack. According to spec one possible address is: http://[ff01::114]:anyport/

        As example look at SolrExceptionTest.java

        FYI, ff01:: is the address range for local Multicast addresses, but only ff01::1 (all local nodes) and ff01::2 (all routers) are allowed in IPv6 networks. ff01::114 is plain invalid and refused by host's TCP stack for using in TCP connections. So it is guaranteed to fail.

        Show
        thetaphi Uwe Schindler added a comment - - edited Hi, please don't write tests that rely on DNS failures. Every local provider does something else, but not all return a correct "not found". Unless we have a "mock DNS server", we cannot use any DNS in Solr tests. For similar reasons we also do not use "localhost" as hostname, but instead use 127.0.0.1 as IP address. To have some broken address that always fails to connect to, use an IPv6 address that is standardized to be routed nowhere and already refused by the Hosts's IPv6 stack. According to spec one possible address is: http://[ff01::114]:anyport/ As example look at SolrExceptionTest.java FYI, ff01:: is the address range for local Multicast addresses, but only ff01::1 (all local nodes) and ff01::2 (all routers) are allowed in IPv6 networks. ff01::114 is plain invalid and refused by host's TCP stack for using in TCP connections. So it is guaranteed to fail.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 2c0a5e30364d83dc82383075a5f7c65200022494 in lucene-solr's branch refs/heads/master from Dennis Gove
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2c0a5e3 ]

        SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection can be made

        Show
        jira-bot ASF subversion and git services added a comment - Commit 2c0a5e30364d83dc82383075a5f7c65200022494 in lucene-solr's branch refs/heads/master from Dennis Gove [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2c0a5e3 ] SOLR-8599 : After a failed connection during construction of SolrZkClient attempt to retry until a connection can be made
        Hide
        dpgove Dennis Gove added a comment -

        Thanks. I'll keep that in mind going forward - and I agree, depending on a DNS failure was ripe for issues down the road. As the test was changed to instead rely on an invalid url I think what's in there will be good. New attachment forthcoming.

        Show
        dpgove Dennis Gove added a comment - Thanks. I'll keep that in mind going forward - and I agree, depending on a DNS failure was ripe for issues down the road. As the test was changed to instead rely on an invalid url I think what's in there will be good. New attachment forthcoming.
        Hide
        dpgove Dennis Gove added a comment - - edited

        New patch with slightly updated test. This now causes an exception due to an invalid url instead of a DNS failure. It exposes the same issue being fixed and the test shows that the patch does in fact resolve it. This patch is what was committed in 2c0a5e30364d83dc82383075a5f7c65200022494.

        Show
        dpgove Dennis Gove added a comment - - edited New patch with slightly updated test. This now causes an exception due to an invalid url instead of a DNS failure. It exposes the same issue being fixed and the test shows that the patch does in fact resolve it. This patch is what was committed in 2c0a5e30364d83dc82383075a5f7c65200022494.
        Hide
        thetaphi Uwe Schindler added a comment -

        Hi,
        why not make the setter of the zkAddress package private? Test is in same package, so it does not need to be fully public!

        Show
        thetaphi Uwe Schindler added a comment - Hi, why not make the setter of the zkAddress package private? Test is in same package, so it does not need to be fully public!
        Hide
        thetaphi Uwe Schindler added a comment -

        In addition there is a bug in the test: It sets the "correct" server address in another thread but there is no synchronization. This may work many times but fails easily, especially on highly parallel CPU architectures! Making the zkServerAddress volatile would be a fix, but as its for testing only, it is not a good idea.

        Show
        thetaphi Uwe Schindler added a comment - In addition there is a bug in the test: It sets the "correct" server address in another thread but there is no synchronization. This may work many times but fails easily, especially on highly parallel CPU architectures! Making the zkServerAddress volatile would be a fix, but as its for testing only, it is not a good idea.
        Hide
        thetaphi Uwe Schindler added a comment -

        Hi, I would go a completely other route to test this:

        • make the zkAddress final again. The change here will not work correctly in Multi-CPU environments. Also it is always a bad idea to "suddenly" change accessibility of fields
        • instead of trying to connect to a wronmg address, we should emulate what is really the issue: the same address does not resolve or work. The correct way to "emulate" this failure is to intercept the connection to zookeeper. All socket connections in Java are handled through a SocketFactory. During the test you can exchange the default SocketFactory by a custom one that throws IOException, if the connect address is the one in the test and delegate otherwise (to not break other stuff like HTTP connections,...). After a few seconds restore the original socket factory (and also in finally!!!). By this we emulate the real failure and we dont need to use "bad"/"invalid" adresses. You can also throw the real execption like java.net.UnknownHostException.

        Beware: This may not work, if Zookeeper caches the socketfactory internally, but that is unlikely.

        Show
        thetaphi Uwe Schindler added a comment - Hi, I would go a completely other route to test this: make the zkAddress final again. The change here will not work correctly in Multi-CPU environments. Also it is always a bad idea to "suddenly" change accessibility of fields instead of trying to connect to a wronmg address, we should emulate what is really the issue: the same address does not resolve or work. The correct way to "emulate" this failure is to intercept the connection to zookeeper. All socket connections in Java are handled through a SocketFactory. During the test you can exchange the default SocketFactory by a custom one that throws IOException, if the connect address is the one in the test and delegate otherwise (to not break other stuff like HTTP connections,...). After a few seconds restore the original socket factory (and also in finally!!!). By this we emulate the real failure and we dont need to use "bad"/"invalid" adresses. You can also throw the real execption like java.net.UnknownHostException . Beware: This may not work, if Zookeeper caches the socketfactory internally, but that is unlikely.
        Hide
        thetaphi Uwe Schindler added a comment -

        2nd: Do we really need to wait 5 !!! seconds in this test? This is horrible for people running the test! At least you should mark it with @Slow so it only runs nightly.

        Show
        thetaphi Uwe Schindler added a comment - 2nd: Do we really need to wait 5 !!! seconds in this test? This is horrible for people running the test! At least you should mark it with @Slow so it only runs nightly.
        Hide
        thetaphi Uwe Schindler added a comment -

        Ah, the whole test is marked as slow.

        Show
        thetaphi Uwe Schindler added a comment - Ah, the whole test is marked as slow.
        Hide
        k317h Keith Laban added a comment - - edited

        uploaded a new patch against master addressing Uwes concerns. I see that this was already merged in master, but that seems to be the only branch its currently on so hopefully its ok to put this out there.

        Instead of scheduling a thread to change the address I was able to mock the DefaultConnectionStrategy to throw an exception in the reconnect call the first time its called to verify that it is being retried. With this change I also changed the server address to be final again and removed the public setter method seeing how they were no longer needed

        Note: The new patch doesn't have the changes which introduced the retry logic in ConnectionManager since that part is already merged in on the master branch and branch_6x/branch_6_0. I can put in a separate patch containing both for backport to branch_5x if needed

        Show
        k317h Keith Laban added a comment - - edited uploaded a new patch against master addressing Uwes concerns. I see that this was already merged in master, but that seems to be the only branch its currently on so hopefully its ok to put this out there. Instead of scheduling a thread to change the address I was able to mock the DefaultConnectionStrategy to throw an exception in the reconnect call the first time its called to verify that it is being retried. With this change I also changed the server address to be final again and removed the public setter method seeing how they were no longer needed Note: The new patch doesn't have the changes which introduced the retry logic in ConnectionManager since that part is already merged in on the master branch and branch_6x/branch_6_0. I can put in a separate patch containing both for backport to branch_5x if needed
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit e3b785a906d6f93e04f2cb45c436516158af0425 in lucene-solr's branch refs/heads/master from Dennis Gove
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e3b785a ]

        SOLR-8599: Improved the tests for this issue to avoid changing a variable to non-final

        Show
        jira-bot ASF subversion and git services added a comment - Commit e3b785a906d6f93e04f2cb45c436516158af0425 in lucene-solr's branch refs/heads/master from Dennis Gove [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e3b785a ] SOLR-8599 : Improved the tests for this issue to avoid changing a variable to non-final
        Hide
        anshumg Anshum Gupta added a comment - - edited

        I see this in the CHANGE log for 6.0 but there's no Fix version. I guess this was just a mistake. Please correct me if I'm missing something here.
        I'm also back porting this for 5.5.1.

        Show
        anshumg Anshum Gupta added a comment - - edited I see this in the CHANGE log for 6.0 but there's no Fix version. I guess this was just a mistake. Please correct me if I'm missing something here. I'm also back porting this for 5.5.1.
        Hide
        k317h Keith Laban added a comment -

        Anshum Gupta There were two separate commits for this ticket, but they didn't all land on 6x or 6.0 but are both on master:

        commit e3b785a906d6f93e04f2cb45c436516158af0425
        Author: Dennis Gove <dpgove@gmail.com>
        Date:   Sun Mar 20 11:13:56 2016 -0400
        
            SOLR-8599: Improved the tests for this issue to avoid changing a variable to non-final
        
        commit 2c0a5e30364d83dc82383075a5f7c65200022494
        Author: Dennis Gove <dpgove@gmail.com>
        Date:   Wed Feb 10 15:02:18 2016 -0500
        
            SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection can be made
        

        however only the first commit found its way to 6x and 6.0 so please port that second commit and remember to port both for 5.5.1, thanks

        Show
        k317h Keith Laban added a comment - Anshum Gupta There were two separate commits for this ticket, but they didn't all land on 6x or 6.0 but are both on master: commit e3b785a906d6f93e04f2cb45c436516158af0425 Author: Dennis Gove <dpgove@gmail.com> Date: Sun Mar 20 11:13:56 2016 -0400 SOLR-8599: Improved the tests for this issue to avoid changing a variable to non- final commit 2c0a5e30364d83dc82383075a5f7c65200022494 Author: Dennis Gove <dpgove@gmail.com> Date: Wed Feb 10 15:02:18 2016 -0500 SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection can be made however only the first commit found its way to 6x and 6.0 so please port that second commit and remember to port both for 5.5.1, thanks
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit d9875832f4798e5f732f4ae5627c7b306ccafa9c in lucene-solr's branch refs/heads/branch_5x from Dennis Gove
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d987583 ]

        SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection can be made

        Show
        jira-bot ASF subversion and git services added a comment - Commit d9875832f4798e5f732f4ae5627c7b306ccafa9c in lucene-solr's branch refs/heads/branch_5x from Dennis Gove [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d987583 ] SOLR-8599 : After a failed connection during construction of SolrZkClient attempt to retry until a connection can be made
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 20e2caba9615e19f84fbcc59a950fb197385592e in lucene-solr's branch refs/heads/branch_5x from Dennis Gove
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=20e2cab ]

        SOLR-8599: Improved the tests for this issue to avoid changing a variable to non-final

        Show
        jira-bot ASF subversion and git services added a comment - Commit 20e2caba9615e19f84fbcc59a950fb197385592e in lucene-solr's branch refs/heads/branch_5x from Dennis Gove [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=20e2cab ] SOLR-8599 : Improved the tests for this issue to avoid changing a variable to non-final
        Hide
        anshumg Anshum Gupta added a comment -

        Thanks Keith. We need to track these better
        I'll commit the other one to 6x too. I got both of them to 5x and I'm about to commit these to 5.5.

        Show
        anshumg Anshum Gupta added a comment - Thanks Keith. We need to track these better I'll commit the other one to 6x too. I got both of them to 5x and I'm about to commit these to 5.5.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 853e1b99b10ccce4029fd77ba88df17dbc77ce3d in lucene-solr's branch refs/heads/branch_5_5 from Dennis Gove
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=853e1b9 ]

        SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection can be made

        Show
        jira-bot ASF subversion and git services added a comment - Commit 853e1b99b10ccce4029fd77ba88df17dbc77ce3d in lucene-solr's branch refs/heads/branch_5_5 from Dennis Gove [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=853e1b9 ] SOLR-8599 : After a failed connection during construction of SolrZkClient attempt to retry until a connection can be made
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 983abb1ca14f7ee42678a03f9d754af8e05e8288 in lucene-solr's branch refs/heads/branch_5_5 from Dennis Gove
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=983abb1 ]

        SOLR-8599: Improved the tests for this issue to avoid changing a variable to non-final

        Show
        jira-bot ASF subversion and git services added a comment - Commit 983abb1ca14f7ee42678a03f9d754af8e05e8288 in lucene-solr's branch refs/heads/branch_5_5 from Dennis Gove [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=983abb1 ] SOLR-8599 : Improved the tests for this issue to avoid changing a variable to non-final
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 78176e23bcac5c6e4accd8989dc931ec6cedb188 in lucene-solr's branch refs/heads/branch_6x from Dennis Gove
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=78176e2 ]

        SOLR-8599: Improved the tests for this issue to avoid changing a variable to non-final

        Show
        jira-bot ASF subversion and git services added a comment - Commit 78176e23bcac5c6e4accd8989dc931ec6cedb188 in lucene-solr's branch refs/heads/branch_6x from Dennis Gove [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=78176e2 ] SOLR-8599 : Improved the tests for this issue to avoid changing a variable to non-final
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 8d24d72ab64d435a5e6bdca11b5e79c22f0057ef in lucene-solr's branch refs/heads/branch_6_0 from Dennis Gove
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8d24d72 ]

        SOLR-8599: Improved the tests for this issue to avoid changing a variable to non-final

        Show
        jira-bot ASF subversion and git services added a comment - Commit 8d24d72ab64d435a5e6bdca11b5e79c22f0057ef in lucene-solr's branch refs/heads/branch_6_0 from Dennis Gove [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8d24d72 ] SOLR-8599 : Improved the tests for this issue to avoid changing a variable to non-final
        Hide
        varunthacker Varun Thacker added a comment -

        Hi Dennis,

        In ConnectionManager do we intend to wait for 1 or 5 seconds? Just wanted to clarify and maybe fix it in another Jira

                  log.info("Could not connect due to error, sleeping for 5s and trying agian");
                  waitSleep(1000);
        
        Show
        varunthacker Varun Thacker added a comment - Hi Dennis, In ConnectionManager do we intend to wait for 1 or 5 seconds? Just wanted to clarify and maybe fix it in another Jira log.info( "Could not connect due to error, sleeping for 5s and trying agian" ); waitSleep(1000);

          People

          • Assignee:
            dpgove Dennis Gove
            Reporter:
            k317h Keith Laban
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development