Solr
  1. Solr
  2. SOLR-3432

deleteByQuery silently ignored if updateLog is enabled, but {{_version_}} field does not exist in schema

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 4.0-ALPHA, 4.0-BETA
    • Fix Version/s: 4.0, 5.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Tomcat 7

      Description

      deleteByQuery is silently ignored if there the updateLog is configurd in solrconfig.xml, but there is no _version_ field in the schema.xml (ie: if someone copies the example configs, and then prunes down the schema to remove fields they don't think they need/want)

      To reproduce:

      • comment out _version_ in example schema
      • java -jar start.jar
      • java -Ddata=args -jar post.jar '<add><doc><field name="id">HOSS</field></doc></add>'
      • java -Ddata=args -jar post.jar '<delete><query>id:HOSS</query></delete>'
        • or: java -Ddata=args -jar post.jar '<delete><query>*:*</query></delete>'

      Note in the logs that SolrCore logs the deleteByQuery, but there is no log of it executing...

      May 3, 2012 4:36:24 PM org.apache.solr.update.processor.LogUpdateProcessor finish
      INFO: [collection1] webapp=/solr path=/update params={} {deleteByQuery=*:*} 0 41
      

      Workarround: add this ield to your schema.xml...

         <field name="_version_" type="long" indexed="true" stored="true"/>
      
      1. SOLR-3432.test.patch
        1 kB
        Hoss Man
      2. web.xml
        6 kB
        David Boychuck
      3. solr.xml
        2 kB
        David Boychuck
      4. solrconfig.xml
        65 kB
        David Boychuck
      5. schema.xml
        58 kB
        David Boychuck

        Issue Links

          Activity

          Hide
          Erick Erickson added a comment -

          First, it's usually a good idea to raise this kind of issue on the user's list before raising a JIRA, just to determine whether it's really a bug or something else. It gets a wider audience and often faster answers.

          It works fine for me with a build just now (May 3). Actually, all I did was copy your delete statement to my url in the standard example....

          Is there any chance you're hitting your browser cache?

          Show
          Erick Erickson added a comment - First, it's usually a good idea to raise this kind of issue on the user's list before raising a JIRA, just to determine whether it's really a bug or something else. It gets a wider audience and often faster answers. It works fine for me with a build just now (May 3). Actually, all I did was copy your delete statement to my url in the standard example.... Is there any chance you're hitting your browser cache?
          Hide
          David Boychuck added a comment -

          No, no chance. Was working with hoss in IRC and he instructed me to make
          this ticket. He thinks it has something to do with my configuration. Did
          you try setup with my configuration?


          David Boychuck
          Software Engineer
          Build.com, Inc. <http://corp.build.com/>
          Smarter Home Improvement™
          P.O. Box 7990 Chico, CA 95927
          P: 800.375.3403
          F: 530.566.1893
          dboychuck@build.com | Network of
          Stores<http://www.build.com/index.cfm?page=help:networkstores&source=emailSignature>

          Show
          David Boychuck added a comment - No, no chance. Was working with hoss in IRC and he instructed me to make this ticket. He thinks it has something to do with my configuration. Did you try setup with my configuration? – David Boychuck Software Engineer Build.com, Inc. < http://corp.build.com/ > Smarter Home Improvement™ P.O. Box 7990 Chico, CA 95927 P : 800.375.3403 F : 530.566.1893 dboychuck@build.com | Network of Stores< http://www.build.com/index.cfm?page=help:networkstores&source=emailSignature >
          Hide
          Erick Erickson added a comment -

          Ahh, didn't know you were working with Hoss, that puts a bit of a different spin on it. Nope, I didn't try your config, plus I don't have Tomcat. I don't have time to delve into it much now though....

          Show
          Erick Erickson added a comment - Ahh, didn't know you were working with Hoss, that puts a bit of a different spin on it. Nope, I didn't try your config, plus I don't have Tomcat. I don't have time to delve into it much now though....
          Hide
          Hoss Man added a comment -

          Using the provided configs, i was able to reproduce this, and by process of elimination, narrowed it down to the fact that there was no version field declared in the schema.xml.

          Seems like a side effect of some of the SOlrCloud related deleteByQuery work, definitely needs to be fixed.

          I'll work on a simple test

          Show
          Hoss Man added a comment - Using the provided configs, i was able to reproduce this, and by process of elimination, narrowed it down to the fact that there was no version field declared in the schema.xml. Seems like a side effect of some of the SOlrCloud related deleteByQuery work, definitely needs to be fixed. I'll work on a simple test
          Hide
          Hoss Man added a comment -

          Moved initial bug report to comment to keep description simple now that root issue is understood...

          original description

          When I run a delete the logs show the command is logged and the commit happens but the delete is entirely ignored.

          I Am attaching my solrconfig.xml and schema.xml

          Here is a document you can try to add

          <add overwrite="true"> <doc boost=".25"> <field name="id">1862441</field> <field name="groupid">140tdeloaq2thousanddegrees</field> <field name="productID">140TDELOAQ</field> <field name="manufacturer">2 thousand degrees</field> <field name="productTitle">Replacement Shade for Large Echo Cylinder Pendant 700TDECPQ</field> <field name="theme"></field> <field name="series"></field> <field name="onSale">false</field> <field name="isNew">false</field> <field name="isTopSeller">false</field> <field name="popularity">0</field> <field name="description">%3Ch3%3EReplacement%20Shade%20for%202%20Thousand%20Degrees%20Large%20Echo%20Cylinder%20Pendant%20700TDECPQ%3C%2Fh3%3E</field> <field name="productAddDate">2011-12-20T00:00:00Z</field> <field name="discontinued">false</field> <field name="siteid">82</field> <field name="siteid">15</field> <field name="sku">140TDELOAQ</field> <field name="upc"></field> <field name="price">212.00</field> <field name="price_pricebook_1_numeric">212.00</field> <field name="price_pricebook_2_numeric">212.00</field> <field name="price_pricebook_3_numeric">212.00</field> <field name="price_pricebook_5_numeric">212.00</field> <field name="price_pricebook_6_numeric">212.00</field> <field name="price_pricebook_7_numeric">212.00</field> <field name="price_pricebook_8_numeric">212.00</field> <field name="price_pricebook_10_numeric">212.00</field> <field name="price_pricebook_11_numeric">212.00</field> <field name="price_pricebook_12_numeric">212.00</field> <field name="price_pricebook_14_numeric">212.00</field> <field name="price_pricebook_15_numeric">212.00</field> <field name="price_pricebook_16_numeric">212.00</field> <field name="price_pricebook_17_numeric">212.00</field> <field name="price_pricebook_18_numeric">212.00</field> <field name="price_pricebook_19_numeric">212.00</field> <field name="price_pricebook_20_numeric">212.00</field> <field name="price_pricebook_21_numeric">212.00</field> <field name="price_pricebook_22_numeric">212.00</field> <field name="price_pricebook_23_numeric">212.00</field> <field name="price_pricebook_24_numeric">212.00</field> <field name="price_pricebook_25_numeric">201.40</field> <field name="finish">Aquamarine</field> <field name="finish_string">Aquamarine</field> <field name="masterFinish">blues</field> <field name="masterFinish_string">Blues</field> <field name="inStock">false</field> <field name="searchKeyword"></field>	 <field name="type_string">Accessory</field> <field name="manufacturer_string">2 Thousand Degrees</field> <field name="handletype_string">Replacement Glass</field> <field name="application_string">Shades</field> <field name="price_low_pricebook_1_d">212.00</field> <field name="price_high_pricebook_1_d">212.00</field> <field name="price_low_pricebook_2_d">212.00</field> <field name="price_high_pricebook_2_d">212.00</field> <field name="price_low_pricebook_3_d">212.00</field> <field name="price_high_pricebook_3_d">212.00</field> <field name="price_low_pricebook_5_d">212.00</field> <field name="price_high_pricebook_5_d">212.00</field> <field name="price_low_pricebook_6_d">212.00</field> <field name="price_high_pricebook_6_d">212.00</field> <field name="price_low_pricebook_7_d">212.00</field> <field name="price_high_pricebook_7_d">212.00</field> <field name="price_low_pricebook_8_d">212.00</field> <field name="price_high_pricebook_8_d">212.00</field> <field name="price_low_pricebook_10_d">212.00</field> <field name="price_high_pricebook_10_d">212.00</field> <field name="price_low_pricebook_11_d">212.00</field> <field name="price_high_pricebook_11_d">212.00</field> <field name="price_low_pricebook_12_d">212.00</field> <field name="price_high_pricebook_12_d">212.00</field> <field name="price_low_pricebook_14_d">212.00</field> <field name="price_high_pricebook_14_d">212.00</field> <field name="price_low_pricebook_15_d">212.00</field> <field name="price_high_pricebook_15_d">212.00</field> <field name="price_low_pricebook_16_d">212.00</field> <field name="price_high_pricebook_16_d">212.00</field> <field name="price_low_pricebook_17_d">212.00</field> <field name="price_high_pricebook_17_d">212.00</field> <field name="price_low_pricebook_18_d">212.00</field> <field name="price_high_pricebook_18_d">212.00</field> <field name="price_low_pricebook_19_d">212.00</field> <field name="price_high_pricebook_19_d">212.00</field> <field name="price_low_pricebook_20_d">212.00</field> <field name="price_high_pricebook_20_d">212.00</field> <field name="price_low_pricebook_21_d">212.00</field> <field name="price_high_pricebook_21_d">212.00</field> <field name="price_low_pricebook_22_d">201.40</field> <field name="price_high_pricebook_22_d">201.40</field> <field name="price_low_pricebook_23_d">212.00</field> <field name="price_high_pricebook_23_d">212.00</field> <field name="price_low_pricebook_24_d">212.00</field> <field name="price_high_pricebook_24_d">212.00</field> <field name="price_low_pricebook_25_d">212.00</field> <field name="price_high_pricebook_25_d">212.00</field> </doc> </add>
          

          You can then try to delete all documents however you wish. I tried with a GET, POST, and using the json update handler.

          http://solrserver:8080/apache-solr-4.0/productIndex/update?stream.body=%3Cdelete%3E%3Cquery%3E*:*%3C/query%3E%3C/delete%3E&commit=true
          

          Then verify that none of the documents were deleted.

          Here are my catalina logs

          May 2, 2012 5:36:16 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861078]} 0 5
          May 2, 2012 5:36:17 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861079]} 0 9
          May 2, 2012 5:36:17 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861080]} 0 7
          May 2, 2012 5:36:17 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861081]} 0 7
          May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861082]} 0 7
          May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861083]} 0 6
          May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861084]} 0 6
          May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861085]} 0 5
          May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861086]} 0 6
          May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861087]} 0 6
          May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861088]} 0 6
          May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861089]} 0 6
          May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861090]} 0 7
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861091, 1861092]} 0 10
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861093]} 0 5
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861094]} 0 5
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861095]} 0 7
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861096]} 0 6
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861097]} 0 5
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861098]} 0 5
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861099]} 0 7
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861100]} 0 7
          May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861101]} 0 6
          May 2, 2012 5:36:20 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861102]} 0 6
          May 2, 2012 5:36:20 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861103]} 0 7
          May 2, 2012 5:36:21 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861104]} 0 6
          May 2, 2012 5:36:21 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861105]} 0 7
          May 2, 2012 5:36:21 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861106]} 0 6
          May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861109, 1861108, 1861107, 1861110]} 0 22
          May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861111]} 0 5
          May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861112]} 0 7
          May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861113]} 0 7
          May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861114]} 0 6
          May 2, 2012 5:36:23 PM org.apache.solr.update.DirectUpdateHandler2 commit
          INFO: start commit{flags=0,version=0,optimize=false,openSearcher=true,waitSearcher=false,expungeDeletes=false,softCommit=false}
          May 2, 2012 5:36:23 PM org.apache.solr.core.SolrDeletionPolicy onCommit
          INFO: SolrDeletionPolicy.onCommit: commits:num=2
          	commit{dir=/opt/apache-solr-4.0/data/index,segFN=segments_4,generation=4,filenames=[_3_nrm.cfe, _3_0.tip, _2_0.frq, _2_0.tip, _2_0.prx, _2.per, _2.fdt, _3.fdt, _3.fnm, _2.fdx, _3_0.frq, _2.fnm, _3.per, _3.fdx, _3_nrm.cfs, _2_nrm.cfe, _3_0.prx, _2_0.tim, _2_nrm.cfs, _2_1.del, _3_0.tim, segments_4]
          	commit{dir=/opt/apache-solr-4.0/data/index,segFN=segments_5,generation=5,filenames=[_3_nrm.cfe, _5_nrm.cfe, _5_0.tip, _5.fdt, _5_0.tim, _4_0.frq, _4_nrm.cfe, _4.fnm, _3.fdt, _3.fnm, _4.fdx, _3.per, _3_0.prx, _4_0.prx, _5_0.frq, _3_0.tip, _5.fnm, _4_0.tim, _4_0.tip, _4.per, _4.fdt, _3_0.frq, _3.fdx, _3_nrm.cfs, _5_nrm.cfs, _4_nrm.cfs, _5.fdx, segments_5, _5.per, _3_0.tim, _5_0.prx]
          May 2, 2012 5:36:23 PM org.apache.solr.core.SolrDeletionPolicy updateCommits
          INFO: newest commit = 5
          May 2, 2012 5:36:23 PM org.apache.solr.search.SolrIndexSearcher <init>
          INFO: Opening Searcher@79e4620e main
          May 2, 2012 5:36:23 PM org.apache.solr.core.QuerySenderListener newSearcher
          INFO: QuerySenderListener sending requests to Searcher@79e4620e main{StandardDirectoryReader(segments_5:18:nrt _3(4.0):C150 _4(4.0):C1 _5(4.0):C44)}
          May 2, 2012 5:36:23 PM org.apache.solr.core.QuerySenderListener newSearcher
          INFO: QuerySenderListener done.
          May 2, 2012 5:36:23 PM org.apache.solr.core.SolrCore registerSearcher
          INFO: [productIndex] Registered new searcher Searcher@79e4620e main{StandardDirectoryReader(segments_5:18:nrt _3(4.0):C150 _4(4.0):C1 _5(4.0):C44)}
          May 2, 2012 5:36:23 PM org.apache.solr.update.DirectUpdateHandler2 commit
          INFO: end_commit_flush
          May 2, 2012 5:36:23 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {commit=} 0 317
          May 2, 2012 5:37:09 PM org.apache.solr.update.DirectUpdateHandler2 commit
          INFO: start commit{flags=0,version=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
          May 2, 2012 5:37:09 PM org.apache.solr.search.SolrIndexSearcher <init>
          INFO: Opening Searcher@1e071f47 main
          May 2, 2012 5:37:09 PM org.apache.solr.update.DirectUpdateHandler2 commit
          INFO: end_commit_flush
          May 2, 2012 5:37:09 PM org.apache.solr.core.QuerySenderListener newSearcher
          INFO: QuerySenderListener sending requests to Searcher@1e071f47 main{StandardDirectoryReader(segments_5:18:nrt _3(4.0):C150 _4(4.0):C1 _5(4.0):C44)}
          May 2, 2012 5:37:09 PM org.apache.solr.core.QuerySenderListener newSearcher
          INFO: QuerySenderListener done.
          May 2, 2012 5:37:09 PM org.apache.solr.core.SolrCore registerSearcher
          INFO: [productIndex] Registered new searcher Searcher@1e071f47 main{StandardDirectoryReader(segments_5:18:nrt _3(4.0):C150 _4(4.0):C1 _5(4.0):C44)}
          May 2, 2012 5:37:09 PM org.apache.solr.update.processor.LogUpdateProcessor finish
          INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={stream.body=<delete><query>*:*</query></delete>&commit=true} {deleteByQuery=*:*,commit=} 0 11
          faucet
          
          Show
          Hoss Man added a comment - Moved initial bug report to comment to keep description simple now that root issue is understood... original description When I run a delete the logs show the command is logged and the commit happens but the delete is entirely ignored. I Am attaching my solrconfig.xml and schema.xml Here is a document you can try to add <add overwrite= " true " > <doc boost= ".25" > <field name= "id" >1862441</field> <field name= "groupid" >140tdeloaq2thousanddegrees</field> <field name= "productID" >140TDELOAQ</field> <field name= "manufacturer" >2 thousand degrees</field> <field name= "productTitle" >Replacement Shade for Large Echo Cylinder Pendant 700TDECPQ</field> <field name= "theme" ></field> <field name= "series" ></field> <field name= "onSale" > false </field> <field name= "isNew" > false </field> <field name= "isTopSeller" > false </field> <field name= "popularity" >0</field> <field name= "description" >%3Ch3%3EReplacement%20Shade%20for%202%20Thousand%20Degrees%20Large%20Echo%20Cylinder%20Pendant%20700TDECPQ%3C%2Fh3%3E</field> <field name= "productAddDate" >2011-12-20T00:00:00Z</field> <field name= "discontinued" > false </field> <field name= "siteid" >82</field> <field name= "siteid" >15</field> <field name= "sku" >140TDELOAQ</field> <field name= "upc" ></field> <field name= "price" >212.00</field> <field name= "price_pricebook_1_numeric" >212.00</field> <field name= "price_pricebook_2_numeric" >212.00</field> <field name= "price_pricebook_3_numeric" >212.00</field> <field name= "price_pricebook_5_numeric" >212.00</field> <field name= "price_pricebook_6_numeric" >212.00</field> <field name= "price_pricebook_7_numeric" >212.00</field> <field name= "price_pricebook_8_numeric" >212.00</field> <field name= "price_pricebook_10_numeric" >212.00</field> <field name= "price_pricebook_11_numeric" >212.00</field> <field name= "price_pricebook_12_numeric" >212.00</field> <field name= "price_pricebook_14_numeric" >212.00</field> <field name= "price_pricebook_15_numeric" >212.00</field> <field name= "price_pricebook_16_numeric" >212.00</field> <field name= "price_pricebook_17_numeric" >212.00</field> <field name= "price_pricebook_18_numeric" >212.00</field> <field name= "price_pricebook_19_numeric" >212.00</field> <field name= "price_pricebook_20_numeric" >212.00</field> <field name= "price_pricebook_21_numeric" >212.00</field> <field name= "price_pricebook_22_numeric" >212.00</field> <field name= "price_pricebook_23_numeric" >212.00</field> <field name= "price_pricebook_24_numeric" >212.00</field> <field name= "price_pricebook_25_numeric" >201.40</field> <field name= "finish" >Aquamarine</field> <field name= "finish_string" >Aquamarine</field> <field name= "masterFinish" >blues</field> <field name= "masterFinish_string" >Blues</field> <field name= "inStock" > false </field> <field name= "searchKeyword" ></field> <field name= "type_string" >Accessory</field> <field name= "manufacturer_string" >2 Thousand Degrees</field> <field name= "handletype_string" >Replacement Glass</field> <field name= "application_string" >Shades</field> <field name= "price_low_pricebook_1_d" >212.00</field> <field name= "price_high_pricebook_1_d" >212.00</field> <field name= "price_low_pricebook_2_d" >212.00</field> <field name= "price_high_pricebook_2_d" >212.00</field> <field name= "price_low_pricebook_3_d" >212.00</field> <field name= "price_high_pricebook_3_d" >212.00</field> <field name= "price_low_pricebook_5_d" >212.00</field> <field name= "price_high_pricebook_5_d" >212.00</field> <field name= "price_low_pricebook_6_d" >212.00</field> <field name= "price_high_pricebook_6_d" >212.00</field> <field name= "price_low_pricebook_7_d" >212.00</field> <field name= "price_high_pricebook_7_d" >212.00</field> <field name= "price_low_pricebook_8_d" >212.00</field> <field name= "price_high_pricebook_8_d" >212.00</field> <field name= "price_low_pricebook_10_d" >212.00</field> <field name= "price_high_pricebook_10_d" >212.00</field> <field name= "price_low_pricebook_11_d" >212.00</field> <field name= "price_high_pricebook_11_d" >212.00</field> <field name= "price_low_pricebook_12_d" >212.00</field> <field name= "price_high_pricebook_12_d" >212.00</field> <field name= "price_low_pricebook_14_d" >212.00</field> <field name= "price_high_pricebook_14_d" >212.00</field> <field name= "price_low_pricebook_15_d" >212.00</field> <field name= "price_high_pricebook_15_d" >212.00</field> <field name= "price_low_pricebook_16_d" >212.00</field> <field name= "price_high_pricebook_16_d" >212.00</field> <field name= "price_low_pricebook_17_d" >212.00</field> <field name= "price_high_pricebook_17_d" >212.00</field> <field name= "price_low_pricebook_18_d" >212.00</field> <field name= "price_high_pricebook_18_d" >212.00</field> <field name= "price_low_pricebook_19_d" >212.00</field> <field name= "price_high_pricebook_19_d" >212.00</field> <field name= "price_low_pricebook_20_d" >212.00</field> <field name= "price_high_pricebook_20_d" >212.00</field> <field name= "price_low_pricebook_21_d" >212.00</field> <field name= "price_high_pricebook_21_d" >212.00</field> <field name= "price_low_pricebook_22_d" >201.40</field> <field name= "price_high_pricebook_22_d" >201.40</field> <field name= "price_low_pricebook_23_d" >212.00</field> <field name= "price_high_pricebook_23_d" >212.00</field> <field name= "price_low_pricebook_24_d" >212.00</field> <field name= "price_high_pricebook_24_d" >212.00</field> <field name= "price_low_pricebook_25_d" >212.00</field> <field name= "price_high_pricebook_25_d" >212.00</field> </doc> </add> You can then try to delete all documents however you wish. I tried with a GET, POST, and using the json update handler. http: //solrserver:8080/apache-solr-4.0/productIndex/update?stream.body=%3Cdelete%3E%3Cquery%3E*:*%3C/query%3E%3C/delete%3E&commit= true Then verify that none of the documents were deleted. Here are my catalina logs May 2, 2012 5:36:16 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861078]} 0 5 May 2, 2012 5:36:17 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861079]} 0 9 May 2, 2012 5:36:17 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861080]} 0 7 May 2, 2012 5:36:17 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861081]} 0 7 May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861082]} 0 7 May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861083]} 0 6 May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861084]} 0 6 May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861085]} 0 5 May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861086]} 0 6 May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861087]} 0 6 May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861088]} 0 6 May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861089]} 0 6 May 2, 2012 5:36:18 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861090]} 0 7 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861091, 1861092]} 0 10 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861093]} 0 5 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861094]} 0 5 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861095]} 0 7 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861096]} 0 6 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861097]} 0 5 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861098]} 0 5 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861099]} 0 7 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861100]} 0 7 May 2, 2012 5:36:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861101]} 0 6 May 2, 2012 5:36:20 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861102]} 0 6 May 2, 2012 5:36:20 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861103]} 0 7 May 2, 2012 5:36:21 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861104]} 0 6 May 2, 2012 5:36:21 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861105]} 0 7 May 2, 2012 5:36:21 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861106]} 0 6 May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861109, 1861108, 1861107, 1861110]} 0 22 May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861111]} 0 5 May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861112]} 0 7 May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861113]} 0 7 May 2, 2012 5:36:22 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {add=[1861114]} 0 6 May 2, 2012 5:36:23 PM org.apache.solr.update.DirectUpdateHandler2 commit INFO: start commit{flags=0,version=0,optimize= false ,openSearcher= true ,waitSearcher= false ,expungeDeletes= false ,softCommit= false } May 2, 2012 5:36:23 PM org.apache.solr.core.SolrDeletionPolicy onCommit INFO: SolrDeletionPolicy.onCommit: commits:num=2 commit{dir=/opt/apache-solr-4.0/data/index,segFN=segments_4,generation=4,filenames=[_3_nrm.cfe, _3_0.tip, _2_0.frq, _2_0.tip, _2_0.prx, _2.per, _2.fdt, _3.fdt, _3.fnm, _2.fdx, _3_0.frq, _2.fnm, _3.per, _3.fdx, _3_nrm.cfs, _2_nrm.cfe, _3_0.prx, _2_0.tim, _2_nrm.cfs, _2_1.del, _3_0.tim, segments_4] commit{dir=/opt/apache-solr-4.0/data/index,segFN=segments_5,generation=5,filenames=[_3_nrm.cfe, _5_nrm.cfe, _5_0.tip, _5.fdt, _5_0.tim, _4_0.frq, _4_nrm.cfe, _4.fnm, _3.fdt, _3.fnm, _4.fdx, _3.per, _3_0.prx, _4_0.prx, _5_0.frq, _3_0.tip, _5.fnm, _4_0.tim, _4_0.tip, _4.per, _4.fdt, _3_0.frq, _3.fdx, _3_nrm.cfs, _5_nrm.cfs, _4_nrm.cfs, _5.fdx, segments_5, _5.per, _3_0.tim, _5_0.prx] May 2, 2012 5:36:23 PM org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: newest commit = 5 May 2, 2012 5:36:23 PM org.apache.solr.search.SolrIndexSearcher <init> INFO: Opening Searcher@79e4620e main May 2, 2012 5:36:23 PM org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener sending requests to Searcher@79e4620e main{StandardDirectoryReader(segments_5:18:nrt _3(4.0):C150 _4(4.0):C1 _5(4.0):C44)} May 2, 2012 5:36:23 PM org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener done. May 2, 2012 5:36:23 PM org.apache.solr.core.SolrCore registerSearcher INFO: [productIndex] Registered new searcher Searcher@79e4620e main{StandardDirectoryReader(segments_5:18:nrt _3(4.0):C150 _4(4.0):C1 _5(4.0):C44)} May 2, 2012 5:36:23 PM org.apache.solr.update.DirectUpdateHandler2 commit INFO: end_commit_flush May 2, 2012 5:36:23 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={} {commit=} 0 317 May 2, 2012 5:37:09 PM org.apache.solr.update.DirectUpdateHandler2 commit INFO: start commit{flags=0,version=0,optimize= false ,openSearcher= true ,waitSearcher= true ,expungeDeletes= false ,softCommit= false } May 2, 2012 5:37:09 PM org.apache.solr.search.SolrIndexSearcher <init> INFO: Opening Searcher@1e071f47 main May 2, 2012 5:37:09 PM org.apache.solr.update.DirectUpdateHandler2 commit INFO: end_commit_flush May 2, 2012 5:37:09 PM org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener sending requests to Searcher@1e071f47 main{StandardDirectoryReader(segments_5:18:nrt _3(4.0):C150 _4(4.0):C1 _5(4.0):C44)} May 2, 2012 5:37:09 PM org.apache.solr.core.QuerySenderListener newSearcher INFO: QuerySenderListener done. May 2, 2012 5:37:09 PM org.apache.solr.core.SolrCore registerSearcher INFO: [productIndex] Registered new searcher Searcher@1e071f47 main{StandardDirectoryReader(segments_5:18:nrt _3(4.0):C150 _4(4.0):C1 _5(4.0):C44)} May 2, 2012 5:37:09 PM org.apache.solr.update.processor.LogUpdateProcessor finish INFO: [productIndex] webapp=/apache-solr-4.0 path=/update params={stream.body=<delete><query>*:*</query></delete>&commit= true } {deleteByQuery=*:*,commit=} 0 11 faucet
          Hide
          Hoss Man added a comment -

          In r1333704 i refactored a few of the newer tests that expect the version field to use schema15.xml instead of schema12.xml, so i could remove version from schema12.xml.

          I thought that would make it trivial to attach a patch for DirectupdateHandlerTest demonstrating the bug, but to my surprise even when i updated that test to do some deleteByQuery commands, the test still passes (see r1333738).

          I'm at a loss to understand why the newly updated test passes, but the steps above fail on the example.

          Show
          Hoss Man added a comment - In r1333704 i refactored a few of the newer tests that expect the version field to use schema15.xml instead of schema12.xml, so i could remove version from schema12.xml. I thought that would make it trivial to attach a patch for DirectupdateHandlerTest demonstrating the bug, but to my surprise even when i updated that test to do some deleteByQuery commands, the test still passes (see r1333738). I'm at a loss to understand why the newly updated test passes, but the steps above fail on the example.
          Hide
          Hoss Man added a comment -

          figured out the problem has to do with the updateLog: if you have the updateLog enabled, but you do not have a version field, you get no warnings or errors – just silently ignored deleteByQuery.

          attached patch causes the udpateLog to randomly be enabled in DirectoUpdateHandlerTest, so with this patch, you should see testBasics() fail 50% of the time.

          Show
          Hoss Man added a comment - figured out the problem has to do with the updateLog: if you have the updateLog enabled, but you do not have a version field, you get no warnings or errors – just silently ignored deleteByQuery. attached patch causes the udpateLog to randomly be enabled in DirectoUpdateHandlerTest, so with this patch, you should see testBasics() fail 50% of the time.
          Hide
          Hoss Man added a comment -

          If the underlying problem is that updateLog is useless w/o a _version_ field, then it should probably be a hard startup failure if updateLog is enabled w/o _version_ existing.

          Alternatively: given some of the problems we've seen (and will likley continue to see) with users not realizing when _version_ is mandatory (ie: upgrading from simple solr to SolrCloud) we might consider adding to the IndexSchema automagically on startup if it isn't explicitly declared in schema.xml

          Show
          Hoss Man added a comment - If the underlying problem is that updateLog is useless w/o a _version_ field, then it should probably be a hard startup failure if updateLog is enabled w/o _version_ existing. Alternatively: given some of the problems we've seen (and will likley continue to see) with users not realizing when _version_ is mandatory (ie: upgrading from simple solr to SolrCloud) we might consider adding to the IndexSchema automagically on startup if it isn't explicitly declared in schema.xml
          Hide
          Lance Norskog added a comment -

          we might consider adding to the IndexSchema automagically on startup if it isn't explicitly declared in schema.xml

          Please don't do automagic things. They can cover up real errors, which then become hell to find.

          Just state clearly in the exception what is wrong. Fail Loud.

          Show
          Lance Norskog added a comment - we might consider adding to the IndexSchema automagically on startup if it isn't explicitly declared in schema.xml Please don't do automagic things. They can cover up real errors, which then become hell to find. Just state clearly in the exception what is wrong. Fail Loud.
          Hide
          Hoss Man added a comment -

          bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment

          Show
          Hoss Man added a comment - bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment
          Hide
          Robert Muir added a comment -

          rmuir20120906-bulk-40-change

          Show
          Robert Muir added a comment - rmuir20120906-bulk-40-change
          Hide
          Hoss Man added a comment -

          SOLR-3746 fixed solr to fail to let you have updateLog configured if you don't have the neccessary _version_ field, so this can't be an issue moving forward

          Show
          Hoss Man added a comment - SOLR-3746 fixed solr to fail to let you have updateLog configured if you don't have the neccessary _version_ field, so this can't be an issue moving forward
          Hide
          Uwe Schindler added a comment -

          Closed after release.

          Show
          Uwe Schindler added a comment - Closed after release.

            People

            • Assignee:
              Hoss Man
              Reporter:
              David Boychuck
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development