Solr
  1. Solr
  2. SOLR-5918

ant clean does not remove ZooKeeper data

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 5.1, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      From the solr/ directory when I run 'ant clean' it cleans up all the necessary compiled files etc.

      This also removes the indexes rightly so, but fails to delete the ZooKeeper data.

      1. SOLR-5918.patch
        0.6 kB
        Varun Thacker

        Activity

        Hide
        Varun Thacker added a comment -

        Simple patch but maybe I am missing something here -

        'ant clean' now removes the zoo_data directory, but when I build again using 'ant example' and start up SolrCloud I get the following errors

        java -DzkRun -DnumShards=1 -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -jar start.jar
        
        758 [coreLoadExecutor-4-thread-1] ERROR org.apache.solr.core.CoreContainer  – Unable to create core: collection1
        org.apache.solr.common.SolrException: Error initializing QueryElevationComponent.
        	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:857)
        	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:640)
        	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556)
        	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261)
        	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        	at java.lang.Thread.run(Thread.java:744)
        Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent.
        	at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:242)
        	at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:649)
        	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:848)
        	... 10 more
        Caused by: org.apache.solr.common.SolrException: Error loading config name for collection collection1
        	at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:167)
        	at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:212)
        	... 12 more
        Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1
        	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
        	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:276)
        	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273)
        	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
        	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:273)
        	at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:147)
        	... 13 more
        6760 [coreLoadExecutor-4-thread-1] ERROR org.apache.solr.core.CoreContainer  – null:org.apache.solr.common.SolrException: Unable to create core: collection1
        	at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:911)
        	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:568)
        	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261)
        	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        	at java.lang.Thread.run(Thread.java:744)
        Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent.
        	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:857)
        	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:640)
        	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556)
        	... 8 more
        Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent.
        	at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:242)
        	at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:649)
        	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:848)
        	... 10 more
        Caused by: org.apache.solr.common.SolrException: Error loading config name for collection collection1
        	at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:167)
        	at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:212)
        	... 12 more
        Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1
        	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
        	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:276)
        	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273)
        	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
        	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:273)
        	at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:147)
        	... 13 more
        
        
        
        
        Show
        Varun Thacker added a comment - Simple patch but maybe I am missing something here - 'ant clean' now removes the zoo_data directory, but when I build again using 'ant example' and start up SolrCloud I get the following errors java -DzkRun -DnumShards=1 -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -jar start.jar 758 [coreLoadExecutor-4-thread-1] ERROR org.apache.solr.core.CoreContainer – Unable to create core: collection1 org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.core.SolrCore.<init>(SolrCore.java:857) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:640) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:242) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:649) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:848) ... 10 more Caused by: org.apache.solr.common.SolrException: Error loading config name for collection collection1 at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:167) at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:212) ... 12 more Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1 at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:276) at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:273) at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:147) ... 13 more 6760 [coreLoadExecutor-4-thread-1] ERROR org.apache.solr.core.CoreContainer – null:org.apache.solr.common.SolrException: Unable to create core: collection1 at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:911) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:568) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.core.SolrCore.<init>(SolrCore.java:857) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:640) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) ... 8 more Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent. at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:242) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:649) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:848) ... 10 more Caused by: org.apache.solr.common.SolrException: Error loading config name for collection collection1 at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:167) at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:212) ... 12 more Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1 at org.apache.zookeeper.KeeperException.create(KeeperException.java:111) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:276) at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:273) at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:147) ... 13 more
        Hide
        Erick Erickson added a comment -

        I guess my question is whether this is a good idea or not. Imagine I'm working on some cloud code and I want to make sure that my compiled code is clean. I issue "ant clean". Now my zookeeper config has mysteriously disappeared.

        So I'm -1 on adding this.

        I haven't a clue what's causing the QEV problem...

        Show
        Erick Erickson added a comment - I guess my question is whether this is a good idea or not. Imagine I'm working on some cloud code and I want to make sure that my compiled code is clean. I issue "ant clean". Now my zookeeper config has mysteriously disappeared. So I'm -1 on adding this. I haven't a clue what's causing the QEV problem...
        Hide
        Mark Miller added a comment -

        Now my zookeeper config has mysteriously disappeared.

        Seems like your data directories going away is just as mysterious though, and that has worked this way a long time.

        cloud-dev scripts have moved to not using example at all, but copying it. That's probably the way to go in general.

        Show
        Mark Miller added a comment - Now my zookeeper config has mysteriously disappeared. Seems like your data directories going away is just as mysterious though, and that has worked this way a long time. cloud-dev scripts have moved to not using example at all, but copying it. That's probably the way to go in general.
        Hide
        Varun Thacker added a comment -

        Hi Erick Erickson,

        I created this Jira only after noticing that the data/ directory is nuked when 'ant clean' is run. So I thought we show clean up the ZK data as well.

        Show
        Varun Thacker added a comment - Hi Erick Erickson , I created this Jira only after noticing that the data/ directory is nuked when 'ant clean' is run. So I thought we show clean up the ZK data as well.
        Hide
        Erick Erickson added a comment -

        Varun Thacker Mostly I was making sure the issue was raised. I'll defer to whatever Mark says, he's one of the people deep in this stuff...

        Show
        Erick Erickson added a comment - Varun Thacker Mostly I was making sure the issue was raised. I'll defer to whatever Mark says, he's one of the people deep in this stuff...
        Hide
        Varun Thacker added a comment -

        Well I checked out a previous commit - 6ebb4351d05e32452406d0c1e9f94107590f12fe. This is just before SOLR-5881 got committed.

        Now when I ran the following I don't get any exceptions.

        {nocommit}
        java -DzkRun -DnumShards=1 -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -jar start.jar{nocommit}
        Show
        Varun Thacker added a comment - Well I checked out a previous commit - 6ebb4351d05e32452406d0c1e9f94107590f12fe. This is just before SOLR-5881 got committed. Now when I ran the following I don't get any exceptions. {nocommit} java -DzkRun -DnumShards=1 -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -jar start.jar{nocommit}
        Hide
        Varun Thacker added a comment -

        With the latest checkout of trunk I don't get any errors related to initializing QueryElevationComponent. Maybe I was doing something wrong that time. Sorry for the noise there.

        Show
        Varun Thacker added a comment - With the latest checkout of trunk I don't get any errors related to initializing QueryElevationComponent. Maybe I was doing something wrong that time. Sorry for the noise there.
        Hide
        Steve Rowe added a comment -

        The clean target is set up now to delete server/solr/zoo_data, and the solr homes created by bin/solr -e <example> under example/ are removed.

        server/solr/zoo_data/ isn't getting deleted though. At this point the only problem appears to be that there's a missing trailing slash after the directory in the <include> - when I add the slash the directory is deleted:

        Index: solr/build.xml
        ===================================================================
        --- solr/build.xml	(revision 1654902)
        +++ solr/build.xml	(working copy)
        @@ -319,7 +319,7 @@
               </fileset>
               <fileset dir="server">
                 <include name="**/data/**/*" />
        -        <include name="solr/zoo_data" />
        +        <include name="solr/zoo_data/" />
                 <include name="start.jar" />
                 <include name="logs/*" />
                 <include name="webapps/**/*" />
        

        Committing shortly.

        Show
        Steve Rowe added a comment - The clean target is set up now to delete server/solr/zoo_data , and the solr homes created by bin/solr -e <example> under example/ are removed. server/solr/zoo_data/ isn't getting deleted though. At this point the only problem appears to be that there's a missing trailing slash after the directory in the <include> - when I add the slash the directory is deleted: Index: solr/build.xml =================================================================== --- solr/build.xml (revision 1654902) +++ solr/build.xml (working copy) @@ -319,7 +319,7 @@ </fileset> <fileset dir= "server" > <include name= "**/data/**/*" /> - <include name= "solr/zoo_data" /> + <include name= "solr/zoo_data/" /> <include name= "start.jar" /> <include name= "logs/*" /> <include name= "webapps/**/*" /> Committing shortly.
        Hide
        ASF subversion and git services added a comment -

        Commit 1654913 from Steve Rowe in branch 'dev/trunk'
        [ https://svn.apache.org/r1654913 ]

        SOLR-5918: ant clean does not remove ZooKeeper data

        Show
        ASF subversion and git services added a comment - Commit 1654913 from Steve Rowe in branch 'dev/trunk' [ https://svn.apache.org/r1654913 ] SOLR-5918 : ant clean does not remove ZooKeeper data
        Hide
        ASF subversion and git services added a comment -

        Commit 1654914 from Steve Rowe in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1654914 ]

        SOLR-5918: ant clean does not remove ZooKeeper data (merged trunk r1654913)

        Show
        ASF subversion and git services added a comment - Commit 1654914 from Steve Rowe in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1654914 ] SOLR-5918 : ant clean does not remove ZooKeeper data (merged trunk r1654913)
        Hide
        ASF subversion and git services added a comment -

        Commit 1654915 from Steve Rowe in branch 'dev/branches/lucene_solr_5_0'
        [ https://svn.apache.org/r1654915 ]

        SOLR-5918: ant clean does not remove ZooKeeper data (merged trunk r1654913)

        Show
        ASF subversion and git services added a comment - Commit 1654915 from Steve Rowe in branch 'dev/branches/lucene_solr_5_0' [ https://svn.apache.org/r1654915 ] SOLR-5918 : ant clean does not remove ZooKeeper data (merged trunk r1654913)
        Hide
        Steve Rowe added a comment -

        Committed to trunk, branch_5x, and lucene_solr_5_0.

        Thanks Varun!

        Show
        Steve Rowe added a comment - Committed to trunk, branch_5x, and lucene_solr_5_0. Thanks Varun!
        Hide
        Anshum Gupta added a comment -

        Bulk close after 5.0 release.

        Show
        Anshum Gupta added a comment - Bulk close after 5.0 release.

          People

          • Assignee:
            Steve Rowe
            Reporter:
            Varun Thacker
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development