Solr
  1. Solr
  2. SOLR-6549

bin/solr script should support a -s option to set the -Dsolr.solr.home property

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.10.2, 5.0
    • Component/s: scripts and tools
    • Labels:
      None

      Description

      The bin/solr script supports a -d parameter for specifying the directory containing the webapp, resources, etc, lib ... In most cases, these binaries are reusable (and will eventually be in a server directory SOLR-3619) even if you want to have multiple solr.solr.home directories on the same server. In other words, it is more common/better to do:

      bin/solr start -d server -s home1
      bin/solr start -d server -s home2
      

      than to do:

      bin/solr start -d server1
      bin/solr start -d server2
      

      Basically, the start script needs to support a -s option that allows you to share binaries but have different Solr home directories for running multiple Solr instances on the same host.

        Activity

        Hide
        ASF subversion and git services added a comment -

        Commit 1630550 from Timothy Potter in branch 'dev/trunk'
        [ https://svn.apache.org/r1630550 ]

        SOLR-6549: add a -s option to set the -Dsolr.solr.home property, thus allowing multiple Solr nodes on the same host to share the same server directory -d but with different Solr home directories

        Show
        ASF subversion and git services added a comment - Commit 1630550 from Timothy Potter in branch 'dev/trunk' [ https://svn.apache.org/r1630550 ] SOLR-6549 : add a -s option to set the -Dsolr.solr.home property, thus allowing multiple Solr nodes on the same host to share the same server directory -d but with different Solr home directories
        Hide
        ASF subversion and git services added a comment -

        Commit 1630551 from Timothy Potter in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1630551 ]

        SOLR-6549: add a -s option to set the -Dsolr.solr.home property, thus allowing multiple Solr nodes on the same host to share the same server directory -d but with different Solr home directories

        Show
        ASF subversion and git services added a comment - Commit 1630551 from Timothy Potter in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1630551 ] SOLR-6549 : add a -s option to set the -Dsolr.solr.home property, thus allowing multiple Solr nodes on the same host to share the same server directory -d but with different Solr home directories
        Hide
        ASF subversion and git services added a comment -

        Commit 1630552 from Timothy Potter in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1630552 ]

        SOLR-6549: fix ignores on solr/example/logs directory

        Show
        ASF subversion and git services added a comment - Commit 1630552 from Timothy Potter in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1630552 ] SOLR-6549 : fix ignores on solr/example/logs directory
        Hide
        ASF subversion and git services added a comment -

        Commit 1631528 from Timothy Potter in branch 'dev/branches/lucene_solr_4_10'
        [ https://svn.apache.org/r1631528 ]

        SOLR-6509, SOLR-6486, SOLR-6549, SOLR-6529: backport recent fixes / improvements to the bin/solr scripts for inclusion in 4.10.2 release.

        Show
        ASF subversion and git services added a comment - Commit 1631528 from Timothy Potter in branch 'dev/branches/lucene_solr_4_10' [ https://svn.apache.org/r1631528 ] SOLR-6509 , SOLR-6486 , SOLR-6549 , SOLR-6529 : backport recent fixes / improvements to the bin/solr scripts for inclusion in 4.10.2 release.
        Hide
        Shawn Heisey added a comment -

        Paraphrasing some thoughts that I expressed on IRC:

        If the -s option is used, the argument could be tested. If it's not a directory, a startup warning can be logged, or the script could even abort. If solr.xml does not exist in the directory indicated, a warning could be printed. A lack of solr.xml should abort, because the user might have solr.xml in zookeeper, or they might be intentionally trying to start in single-core mode. (Related: single-core mode probably should be nuked from 5.0, if it's not already)

        Perhaps a -q option could be implemented to suppress startup warnings. If the script sources /etc/default/solr (or some other filename), options could be picked up from that file.

        I see that after I said these things and started transferring my thoughts here, Timothy Potter replied on IRC, saying that the script already does check for the existence of both the solr home and solr.xml, then disappeared. I admit that I did not look at the script before speaking. Have to stop doing that! I think the existing checks can be improved, will think about it.

        Show
        Shawn Heisey added a comment - Paraphrasing some thoughts that I expressed on IRC: If the -s option is used, the argument could be tested. If it's not a directory, a startup warning can be logged, or the script could even abort. If solr.xml does not exist in the directory indicated, a warning could be printed. A lack of solr.xml should abort, because the user might have solr.xml in zookeeper, or they might be intentionally trying to start in single-core mode. (Related: single-core mode probably should be nuked from 5.0, if it's not already) Perhaps a -q option could be implemented to suppress startup warnings. If the script sources /etc/default/solr (or some other filename), options could be picked up from that file. I see that after I said these things and started transferring my thoughts here, Timothy Potter replied on IRC, saying that the script already does check for the existence of both the solr home and solr.xml, then disappeared. I admit that I did not look at the script before speaking. Have to stop doing that! I think the existing checks can be improved, will think about it.
        Hide
        Shawn Heisey added a comment -

        Scott Blum popped up on IRC asking about GC tuning options, and noticed that CMSInitiatingOccupancyFraction was at 50%, and he was going to try it at 70.

        I noted that the CMS parameters on my solr wiki page were at 70, and that the initial GC tuning parameters were heavily influenced by that wiki page.

        Scott went digging deeper, and found that solr.in.sh was initially using 70, then it was changed to 50 by the initial commits for this issue.

        We were wondering whether the change was intentional, and if it was, what the motivation was. The GC change was not mentioned in the commit message.

        Show
        Shawn Heisey added a comment - Scott Blum popped up on IRC asking about GC tuning options, and noticed that CMSInitiatingOccupancyFraction was at 50%, and he was going to try it at 70. I noted that the CMS parameters on my solr wiki page were at 70, and that the initial GC tuning parameters were heavily influenced by that wiki page. Scott went digging deeper, and found that solr.in.sh was initially using 70, then it was changed to 50 by the initial commits for this issue. We were wondering whether the change was intentional, and if it was, what the motivation was. The GC change was not mentioned in the commit message.
        Hide
        Timothy Potter added a comment -

        The change was intentional. I found that having the CMS start collecting sooner rather than later reduced pause times in my testing. However, if you're seeing something different, I'd love to see some empirical evidence that using 50 vs. 70 is causing problems. Keep in mind this is the threshold when concurrent collections start, although there is some stop-the-world phases of CMS, so I can see how starting too early (or too often) can lead to issues. Of course, initial GC tuning params will never be correct for all workloads, so if 70 works better for you, then use 70. The default is 68 if not specified. If we want to remove those options, that's fine as well, but be sure to remove -XX:+UseCMSInitiatingOccupancyOnly to let the JVM pick the correct value.

        Show
        Timothy Potter added a comment - The change was intentional. I found that having the CMS start collecting sooner rather than later reduced pause times in my testing. However, if you're seeing something different, I'd love to see some empirical evidence that using 50 vs. 70 is causing problems. Keep in mind this is the threshold when concurrent collections start, although there is some stop-the-world phases of CMS, so I can see how starting too early (or too often) can lead to issues. Of course, initial GC tuning params will never be correct for all workloads, so if 70 works better for you, then use 70. The default is 68 if not specified. If we want to remove those options, that's fine as well, but be sure to remove -XX:+UseCMSInitiatingOccupancyOnly to let the JVM pick the correct value.

          People

          • Assignee:
            Timothy Potter
            Reporter:
            Timothy Potter
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development