Solr
  1. Solr
  2. SOLR-4749

Refactor the new solr.xml and core discovery code.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.4, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      I think we can clean up a variety of things in the latest code.

      1. SOLR-4749.patch
        124 kB
        Mark Miller

        Issue Links

          Activity

          Hide
          Yonik Seeley added a comment -

          One issue I noticed after a quick glance is a race introduced into CoreContainer by some of the recent changes.

          The getCore() code now increments the reference count (core.open()) outside of the sync block, which means it could be closed in the meantime.

              SolrCore core = coreMaps.getCoreFromAnyList(name);
          
              if (core != null) {
                core.open();
                return core;
              }
          
          Show
          Yonik Seeley added a comment - One issue I noticed after a quick glance is a race introduced into CoreContainer by some of the recent changes. The getCore() code now increments the reference count (core.open()) outside of the sync block, which means it could be closed in the meantime. SolrCore core = coreMaps.getCoreFromAnyList(name); if (core != null ) { core.open(); return core; }
          Hide
          Yonik Seeley added a comment -

          The publishCoresAsDown() code holds the same lock as used for everything else now, and publishes to ZK with the lock held, effectively preventing any other core operations for the duration. It's tough to determine if this has real practical downsides as it's only called during shutdown, but perhaps we should move the publish outside the synchronization just in case (unless it's really needed of course).

          Show
          Yonik Seeley added a comment - The publishCoresAsDown() code holds the same lock as used for everything else now, and publishes to ZK with the lock held, effectively preventing any other core operations for the duration. It's tough to determine if this has real practical downsides as it's only called during shutdown, but perhaps we should move the publish outside the synchronization just in case (unless it's really needed of course).
          Hide
          Mark Miller added a comment -

          As far as I can tell, a bunch of the persistence stuff looks broke too - looks like it saves correctly for cores level stuff, but not core level stuff.

          Show
          Mark Miller added a comment - As far as I can tell, a bunch of the persistence stuff looks broke too - looks like it saves correctly for cores level stuff, but not core level stuff.
          Hide
          Mark Miller added a comment -

          To help get a handle on some of this stuff, I've started a large refactoring. I'll post my first patch soon and then quickly commit it before any conflicts are built up.

          Once I know exactly what the current persistent issues are, I'll likely open a new JIRA issue to track that, but fix it here.

          Show
          Mark Miller added a comment - To help get a handle on some of this stuff, I've started a large refactoring. I'll post my first patch soon and then quickly commit it before any conflicts are built up. Once I know exactly what the current persistent issues are, I'll likely open a new JIRA issue to track that, but fix it here.
          Hide
          Mark Miller added a comment -

          Here is pass1 that I will commit shortly. At a minimum, a pass2 is coming.

          Show
          Mark Miller added a comment - Here is pass1 that I will commit shortly. At a minimum, a pass2 is coming.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470674

          SOLR-4749: Clean up and refactor CoreContainer code around solr.xml and SolrCore management.

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470674 SOLR-4749 : Clean up and refactor CoreContainer code around solr.xml and SolrCore management.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470678

          SOLR-4749: Remove nocommit

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470678 SOLR-4749 : Remove nocommit
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470681

          SOLR-4749: Clean up and refactor CoreContainer code around solr.xml and SolrCore management.

          Show
          Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470681 SOLR-4749 : Clean up and refactor CoreContainer code around solr.xml and SolrCore management.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470709

          SOLR-4749: Clean up and refactor CoreContainer code around solr.xml and SolrCore management.

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470709 SOLR-4749 : Clean up and refactor CoreContainer code around solr.xml and SolrCore management.
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470711

          SOLR-4749: Clean up and refactor CoreContainer code around solr.xml and SolrCore management.

          Show
          Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470711 SOLR-4749 : Clean up and refactor CoreContainer code around solr.xml and SolrCore management.
          Hide
          Mark Miller added a comment -

          That was pass 2 - pass 3 coming in a moment. On top of further refactoring in this area, I started isolating the zk code in a new class to help slim down CoreContainer.

          Show
          Mark Miller added a comment - That was pass 2 - pass 3 coming in a moment. On top of further refactoring in this area, I started isolating the zk code in a new class to help slim down CoreContainer.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470758

          SOLR-4749: Clean up and refactor CoreContainer code around solr.xml and SolrCore management.

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470758 SOLR-4749 : Clean up and refactor CoreContainer code around solr.xml and SolrCore management.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470766

          SOLR-4749: Fix a few issues.

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470766 SOLR-4749 : Fix a few issues.
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470768

          SOLR-4749: Clean up and refactor CoreContainer code around solr.xml and SolrCore management.

          Show
          Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470768 SOLR-4749 : Clean up and refactor CoreContainer code around solr.xml and SolrCore management.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470772

          SOLR-4749: Record solrcloud attribs even when not in zk mode.

          Show
          Commit Tag Bot added a comment - [trunk commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470772 SOLR-4749 : Record solrcloud attribs even when not in zk mode.
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470773

          SOLR-4749: Fix a few issues.

          Show
          Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470773 SOLR-4749 : Fix a few issues.
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] markrmiller
          http://svn.apache.org/viewvc?view=revision&revision=1470774

          SOLR-4749: Record solrcloud attribs even when not in zk mode.

          Show
          Commit Tag Bot added a comment - [branch_4x commit] markrmiller http://svn.apache.org/viewvc?view=revision&revision=1470774 SOLR-4749 : Record solrcloud attribs even when not in zk mode.
          Hide
          Yonik Seeley added a comment -

          This may be related: a user reported core reference counting issues:
          http://markmail.org/message/akrrj5o24prasm6e

          2965529:SEVERE: Too many close [count:-1] on
          org.apache.solr.core.SolrCore@7722b49b. Please report this exception to
          solr-user@lucene.apache.org
          

          When I tried running TestLazyCores, I also saw these errors.

          Show
          Yonik Seeley added a comment - This may be related: a user reported core reference counting issues: http://markmail.org/message/akrrj5o24prasm6e 2965529:SEVERE: Too many close [count:-1] on org.apache.solr.core.SolrCore@7722b49b. Please report this exception to solr-user@lucene.apache.org When I tried running TestLazyCores, I also saw these errors.
          Hide
          Mark Miller added a comment -

          I filed SOLR-4753 for the ref count issue.

          Show
          Mark Miller added a comment - I filed SOLR-4753 for the ref count issue.
          Hide
          Mark Miller added a comment -

          As far as a general attack on this, I'm done. I'll open specific JIRA issues for any further changes.

          Show
          Mark Miller added a comment - As far as a general attack on this, I'm done. I'll open specific JIRA issues for any further changes.
          Hide
          Steve Rowe added a comment -

          Bulk close resolved 4.4 issues

          Show
          Steve Rowe added a comment - Bulk close resolved 4.4 issues

            People

            • Assignee:
              Mark Miller
              Reporter:
              Mark Miller
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development