Solr
  1. Solr
  2. SOLR-5543

solr.xml duplicat eentries after SWAP 4.6

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.6
    • Fix Version/s: 4.6.1, 4.7, 5.0
    • Component/s: None
    • Labels:
      None

      Description

      We are having issues with SWAP CoreAdmin in 4.6.

      Using legacy solr.xml we issue a COreodmin SWAP, and we want it persistent. It has been running flawless since 4.5. Now it creates duplicate lines in solr.xml.

      Even the example multi core schema in doesn't work with persistent="true" - it creates duplicate lines in solr.xml.

      <cores adminPath="/admin/cores">
      <core name="autosuggest" loadOnStartup="true" instanceDir="autosuggest" transient="false"/>

      <core name="citystateprovider" loadOnStartup="true" instanceDir="citystateprovider" transient="false"/>

      <core name="collection1" loadOnStartup="true" instanceDir="collection1" transient="false"/>

      <core name="facility" loadOnStartup="true" instanceDir="facility" transient="false"/>

      <core name="inactiveproviders" loadOnStartup="true" instanceDir="inactiveproviders" transient="false"/>

      <core name="linesvcgeo" instanceDir="linesvcgeo" loadOnStartup="true" transient="false"/>

      <core name="linesvcgeofull" instanceDir="linesvcgeofull" loadOnStartup="true" transient="false"/>

      <core name="locationgeo" loadOnStartup="true" instanceDir="locationgeo" transient="false"/>

      <core name="market" loadOnStartup="true" instanceDir="market" transient="false"/>

      <core name="portalprovider" loadOnStartup="true" instanceDir="portalprovider" transient="false"/>

      <core name="practice" loadOnStartup="true" instanceDir="practice" transient="false"/>

      <core name="provider" loadOnStartup="true" instanceDir="provider" transient="false"/>

      <core name="providersearch" loadOnStartup="true" instanceDir="providersearch" transient="false"/>

      <core name="tridioncomponents" loadOnStartup="true" instanceDir="tridioncomponents" transient="false"/>

      <core name="linesvcgeo" instanceDir="linesvcgeo" loadOnStartup="true" transient="false"/>

      <core name="linesvcgeofull" instanceDir="linesvcgeofull" loadOnStartup="true" transient="false"/>
      </cores>

      1. SOLR-5543.patch
        8 kB
        Alan Woodward

        Activity

        Hide
        Alan Woodward added a comment -

        This patch adds an explicit #swap(CoreContainer, CoreDescriptor, CoreDescriptor) method to CoreLocator. CoreContainer.swap() was just using persist, which ended up adding two new entries to the end of the cores list but never clearing them out. This wasn't being picked up in tests because the SolrXMLCoreLocator deduplicated them in discovery (by a happy accident).

        Show
        Alan Woodward added a comment - This patch adds an explicit #swap(CoreContainer, CoreDescriptor, CoreDescriptor) method to CoreLocator. CoreContainer.swap() was just using persist, which ended up adding two new entries to the end of the cores list but never clearing them out. This wasn't being picked up in tests because the SolrXMLCoreLocator deduplicated them in discovery (by a happy accident).
        Hide
        ASF subversion and git services added a comment -

        Commit 1550969 from Alan Woodward in branch 'dev/trunk'
        [ https://svn.apache.org/r1550969 ]

        SOLR-5543: Core Swaps result in duplicate entries in solr.xml

        Show
        ASF subversion and git services added a comment - Commit 1550969 from Alan Woodward in branch 'dev/trunk' [ https://svn.apache.org/r1550969 ] SOLR-5543 : Core Swaps result in duplicate entries in solr.xml
        Hide
        ASF subversion and git services added a comment -

        Commit 1550970 from Alan Woodward in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1550970 ]

        SOLR-5543: Core Swaps result in duplicate entries in solr.xml

        Show
        ASF subversion and git services added a comment - Commit 1550970 from Alan Woodward in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1550970 ] SOLR-5543 : Core Swaps result in duplicate entries in solr.xml
        Hide
        Alan Woodward added a comment -

        Thanks Bill!

        Show
        Alan Woodward added a comment - Thanks Bill!
        Hide
        Shawn Heisey added a comment -

        Alan Woodward ... I hear we're doing a 4.6.1 in the near future. IMHO this is a strong candidate for backporting.

        Interesting detail, which I don't think needs to change your patch at all: My 'full index rebuild' process does seven core swaps. It appears that the duplication introduced by a core swap is eliminated by a subsequent core swap – there were only extra entries for the last core swapped.

        Show
        Shawn Heisey added a comment - Alan Woodward ... I hear we're doing a 4.6.1 in the near future. IMHO this is a strong candidate for backporting. Interesting detail, which I don't think needs to change your patch at all: My 'full index rebuild' process does seven core swaps. It appears that the duplication introduced by a core swap is eliminated by a subsequent core swap – there were only extra entries for the last core swapped.
        Hide
        Alan Woodward added a comment -

        It doesn't seem to actually make a difference to the running application, because the ConfigSolr object deduplicates the entries internally, which is also why it only writes out the latest swap (the current set, plus the two extra core definitions). But it makes it confusing for people who are editing solr.xml offline.

        Show
        Alan Woodward added a comment - It doesn't seem to actually make a difference to the running application, because the ConfigSolr object deduplicates the entries internally, which is also why it only writes out the latest swap (the current set, plus the two extra core definitions). But it makes it confusing for people who are editing solr.xml offline.
        Hide
        Shawn Heisey added a comment -

        The reason I think it's critical to include in 4.6.1 is this: Although everything works perfectly after the problem appears, as soon as you restart Solr, the duplicate entries cause exceptions and cores don't start up properly. That was my experience when I upgraded my dev server to the released 4.6.0, rebuilt the index (resulting in seven core swaps), and then later restarted Solr for some config changes. It's not standard procedure to restart Solr after rebuilding, but it does occasionally happen. It happens a LOT on my dev server, where I try out new configs.

        If I were to do a PERSIST CoreAdmin action after each swap, would it effectively fix the problem even on an unpatched 4.6.0?

        TL;DR: I do have the ability to instead do a "persist all" after all of the swaps, because I separately create and track per-server objects for CoreAdmin SolrJ calls. Although I end up with 28 HttpSolrServer objects for updates and queries with my production build application, I only create one CoreAdmin server object for each host/port combination, instead of an additional 28 objects. I don't know what the incremental size of HttpSolrServer objects is, but I do try to be careful about memory.

        Show
        Shawn Heisey added a comment - The reason I think it's critical to include in 4.6.1 is this: Although everything works perfectly after the problem appears, as soon as you restart Solr, the duplicate entries cause exceptions and cores don't start up properly. That was my experience when I upgraded my dev server to the released 4.6.0, rebuilt the index (resulting in seven core swaps), and then later restarted Solr for some config changes. It's not standard procedure to restart Solr after rebuilding, but it does occasionally happen. It happens a LOT on my dev server, where I try out new configs. If I were to do a PERSIST CoreAdmin action after each swap, would it effectively fix the problem even on an unpatched 4.6.0? TL;DR: I do have the ability to instead do a "persist all" after all of the swaps, because I separately create and track per-server objects for CoreAdmin SolrJ calls. Although I end up with 28 HttpSolrServer objects for updates and queries with my production build application, I only create one CoreAdmin server object for each host/port combination, instead of an additional 28 objects. I don't know what the incremental size of HttpSolrServer objects is, but I do try to be careful about memory.
        Hide
        Shawn Heisey added a comment -

        With a verbal OK from Mark Miller via IRC, I am backporting the fix for this issue to 4.6.1.

        Both precommit and tests in solr/ are passing. The commits for trunk and branch_4x have no code changes, I'm just moving the CHANGES.txt entry.

        Show
        Shawn Heisey added a comment - With a verbal OK from Mark Miller via IRC, I am backporting the fix for this issue to 4.6.1. Both precommit and tests in solr/ are passing. The commits for trunk and branch_4x have no code changes, I'm just moving the CHANGES.txt entry.
        Hide
        ASF subversion and git services added a comment -

        Commit 1556965 from Shawn Heisey in branch 'dev/branches/lucene_solr_4_6'
        [ https://svn.apache.org/r1556965 ]

        SOLR-5543: Backport to 4.6 branch, for 4.6.1 release.

        Show
        ASF subversion and git services added a comment - Commit 1556965 from Shawn Heisey in branch 'dev/branches/lucene_solr_4_6' [ https://svn.apache.org/r1556965 ] SOLR-5543 : Backport to 4.6 branch, for 4.6.1 release.
        Hide
        ASF subversion and git services added a comment -

        Commit 1556968 from Shawn Heisey in branch 'dev/trunk'
        [ https://svn.apache.org/r1556968 ]

        SOLR-5543: move changes entry from 4.7.0 to 4.6.1.

        Show
        ASF subversion and git services added a comment - Commit 1556968 from Shawn Heisey in branch 'dev/trunk' [ https://svn.apache.org/r1556968 ] SOLR-5543 : move changes entry from 4.7.0 to 4.6.1.
        Hide
        ASF subversion and git services added a comment -

        Commit 1556969 from Shawn Heisey in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1556969 ]

        SOLR-5543: Move changes entry from 4.7.0 to 4.6.1 (merge trunk r1556968)

        Show
        ASF subversion and git services added a comment - Commit 1556969 from Shawn Heisey in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1556969 ] SOLR-5543 : Move changes entry from 4.7.0 to 4.6.1 (merge trunk r1556968)

          People

          • Assignee:
            Alan Woodward
            Reporter:
            Bill Bell
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development