Solr
  1. Solr
  2. SOLR-4127

Atomic updates used w/o updateLog should throw an error

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.0
    • Fix Version/s: 4.1, 6.0
    • Component/s: update
    • Labels:
      None

      Description

      The atomic update feature described in SOLR-139 seems to depend on having an <updateLog /> configured in solrconfig.xml.

      When used without an update log, the update commands like set or add don't result in an error and the transaction being aborted, but produce garbled documents instead. This is the case for both the XML and JSON formats for the update message.

      Example:

      I initially created some content like this:

      $ curl 'localhost:8983/solr/update?commit=true' -H 'Content-type:application/json' -d '
      [{"id":"7cb8a43c","Title":"My original Title", "Creator": "John Doe"}]'
      

      Which resulted in this document:

      <doc>
          <str name="id">7cb8a43c</str>
          <str name="Title">My original Title</str>
          <str name="Creator">John Doe</str>
      </doc>
      

      Then I attempted to update that document with this statement:

      $ curl 'localhost:8983/solr/update?commit=true' -H 'Content-type:application/json' -d '
      [{"id":"7cb8a43c","Title":{"set":"My new title"}}]'
      

      Which resulted in this garbled document, with the fields that weren't updated missing:

      <doc>
          <str name="id">7cb8a43c</str>
          <str name="Title">{set=My new title}</str>
      </doc>
      

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          Patch that adds tests for two scenerios:

          • updateLog is not in use, but distrib processor is in the chain
          • distrib processor is not in the chainat all

          In both cases we assert that a useful error is thrown if the document contains atomic update instructions, via three modifications:

          1) refactored "isAtomicUpdate(AddUpdateCommand)" (ie: a field value is a map) into existences
          2) distrib processor throws an error if updateLog is not configured but the command isAtomicUpdate
          3) run processor throws an error if it is given a command that isAtomicUpdate

          (this way even if someone really wants to use their own distrib processor, they can still use atomic updates as long as they implement it and process the maps before the docs make it to the run processor)

          Show
          Hoss Man added a comment - Patch that adds tests for two scenerios: updateLog is not in use, but distrib processor is in the chain distrib processor is not in the chainat all In both cases we assert that a useful error is thrown if the document contains atomic update instructions, via three modifications: 1) refactored "isAtomicUpdate(AddUpdateCommand)" (ie: a field value is a map) into existences 2) distrib processor throws an error if updateLog is not configured but the command isAtomicUpdate 3) run processor throws an error if it is given a command that isAtomicUpdate (this way even if someone really wants to use their own distrib processor, they can still use atomic updates as long as they implement it and process the maps before the docs make it to the run processor)
          Hide
          Markus Jelsma added a comment -

          We have seen this too on a very recent trunk cluster. It has updateLog enabled and the distributed processor is in the chain. I could not successfully reproduce it when i disabled the langid processor in the chain, nor could i reliably reproduce this problem with the langid enabled.

          Show
          Markus Jelsma added a comment - We have seen this too on a very recent trunk cluster. It has updateLog enabled and the distributed processor is in the chain. I could not successfully reproduce it when i disabled the langid processor in the chain, nor could i reliably reproduce this problem with the langid enabled.
          Hide
          Hoss Man added a comment -

          marjus...

          We have seen this too on a very recent trunk cluster. It has updateLog enabled and the distributed processor is in the chain...

          that sounds like it must be a differnet bug with similar symptoms – the next time you reproduce, can you please file a bug with your configs and the specifics of hte document you tried to updated (ie: it's current state, and what atomic update you attempted)

          Show
          Hoss Man added a comment - marjus... We have seen this too on a very recent trunk cluster. It has updateLog enabled and the distributed processor is in the chain... that sounds like it must be a differnet bug with similar symptoms – the next time you reproduce, can you please file a bug with your configs and the specifics of hte document you tried to updated (ie: it's current state, and what atomic update you attempted)
          Hide
          Lukas Graf added a comment -

          In my experience, the same symptoms could possibly be caused by either

          • using the wrong syntax for the atomic update instructions
          • or using the atomic "add" update instruction in an XML update message, which apparently isn't implemented (only in JSON).

          If nobody beats me to it, I will add two separate issues for those.

          Show
          Lukas Graf added a comment - In my experience, the same symptoms could possibly be caused by either using the wrong syntax for the atomic update instructions or using the atomic "add" update instruction in an XML update message, which apparently isn't implemented (only in JSON). If nobody beats me to it, I will add two separate issues for those.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] Chris M. Hostetter
          http://svn.apache.org/viewvc?view=revision&revision=1420297

          SOLR-4127: Added explicit error message if users attempt Atomic document updates with either updateLog or DistribUpdateProcessor

          Show
          Commit Tag Bot added a comment - [trunk commit] Chris M. Hostetter http://svn.apache.org/viewvc?view=revision&revision=1420297 SOLR-4127 : Added explicit error message if users attempt Atomic document updates with either updateLog or DistribUpdateProcessor
          Hide
          Hoss Man added a comment -

          If nobody beats me to it, I will add two separate issues for those.

          Yes please, with as many details as you can provide

          As for this particular fix...

          Committed revision 1420297. - trunk
          Committed revision 1420305. - 5x

          Show
          Hoss Man added a comment - If nobody beats me to it, I will add two separate issues for those. Yes please, with as many details as you can provide As for this particular fix... Committed revision 1420297. - trunk Committed revision 1420305. - 5x
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] Chris M. Hostetter
          http://svn.apache.org/viewvc?view=revision&revision=1420305

          SOLR-4127: Added explicit error message if users attempt Atomic document updates with either updateLog or DistribUpdateProcessor (merge r1420297)

          Show
          Commit Tag Bot added a comment - [branch_4x commit] Chris M. Hostetter http://svn.apache.org/viewvc?view=revision&revision=1420305 SOLR-4127 : Added explicit error message if users attempt Atomic document updates with either updateLog or DistribUpdateProcessor (merge r1420297)
          Hide
          Lukas Graf added a comment - - edited

          Gave this a test with current trunk (r1421381) - works like a charm.
          Thanks for fixing this so quickly Chris!

          Show
          Lukas Graf added a comment - - edited Gave this a test with current trunk (r1421381) - works like a charm. Thanks for fixing this so quickly Chris!
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] Erik Hatcher
          http://svn.apache.org/viewvc?view=revision&revision=1432794

          SOLR-4127: post.jar ignores -Dparams when -Durl is used

          Show
          Commit Tag Bot added a comment - [trunk commit] Erik Hatcher http://svn.apache.org/viewvc?view=revision&revision=1432794 SOLR-4127 : post.jar ignores -Dparams when -Durl is used
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] Erik Hatcher
          http://svn.apache.org/viewvc?view=revision&revision=1432795

          SOLR-4127: post.jar ignores -Dparams when -Durl is used (merged from trunk)

          Show
          Commit Tag Bot added a comment - [branch_4x commit] Erik Hatcher http://svn.apache.org/viewvc?view=revision&revision=1432795 SOLR-4127 : post.jar ignores -Dparams when -Durl is used (merged from trunk)

            People

            • Assignee:
              Hoss Man
              Reporter:
              Lukas Graf
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development