Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.6, 7.0
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      I'd like to explore the idea of adding a new update processor to help ingest partial updates.

      Example use-case - There are two datasets with a common id field. How can I merge both of them at index time?

      So the first JSON dump could be ingested against http://localhost:8983/solr/gettingstarted/update/json

      And then the second JSON could be ingested against
      http://localhost:8983/solr/gettingstarted/update/json?processor=atomic

      The Atomic Update Processor could support all the atomic update operations currently supported.

      1. assertU(...)-works.png
        579 kB
        Amrit Sarkar
      2. commit()-doesn't-work.png
        569 kB
        Amrit Sarkar
      3. SOLR-9530.patch
        0.7 kB
        Amrit Sarkar
      4. SOLR-9530.patch
        0.7 kB
        Amrit Sarkar
      5. SOLR-9530.patch
        20 kB
        Amrit Sarkar
      6. SOLR-9530.patch
        20 kB
        Amrit Sarkar
      7. SOLR-9530.patch
        20 kB
        Amrit Sarkar
      8. SOLR-9530.patch
        17 kB
        Noble Paul
      9. SOLR-9530.patch
        19 kB
        Amrit Sarkar
      10. SOLR-9530.patch
        20 kB
        Amrit Sarkar
      11. SOLR-9530.patch
        23 kB
        Ishan Chattopadhyaya
      12. SOLR-9530.patch
        23 kB
        Amrit Sarkar
      13. SOLR-9530.patch
        26 kB
        Amrit Sarkar
      14. SOLR-9530.patch
        22 kB
        Amrit Sarkar
      15. SOLR-9530.patch
        19 kB
        Amrit Sarkar
      16. SOLR-9530.patch
        18 kB
        Amrit Sarkar

        Issue Links

          Activity

          Hide
          arafalov Alexandre Rafalovitch added a comment -

          I am not sure i understand the description. What will the second operation actually do? Convert a submitted document into an Update document against the first document?

          Show
          arafalov Alexandre Rafalovitch added a comment - I am not sure i understand the description. What will the second operation actually do? Convert a submitted document into an Update document against the first document?
          Hide
          varunthacker Varun Thacker added a comment -

          Hi Alexandre,

          The AtomicUpdateProcessorFactory update job would be to take the document and all the fields , but convert the request into an atomic update.

          So lets say the first JSON is like this:

           {"id":"1" , "brand" : "apple" } 

          This will be indexed . Now say the second json file has documents in this format

           {"id":"1" , "price" : 1} 

          This can be indexed against the atomic update processor chain.

          So the final document in the index should be

           {"id": "1" , "price" : 1 , "brand" : "apple" } 
          Show
          varunthacker Varun Thacker added a comment - Hi Alexandre, The AtomicUpdateProcessorFactory update job would be to take the document and all the fields , but convert the request into an atomic update. So lets say the first JSON is like this: { "id" : "1" , "brand" : "apple" } This will be indexed . Now say the second json file has documents in this format { "id" : "1" , "price" : 1} This can be indexed against the atomic update processor chain. So the final document in the index should be { "id" : "1" , "price" : 1 , "brand" : "apple" }
          Hide
          arafalov Alexandre Rafalovitch added a comment -

          Ok, that makes sense. And if this URP is defined as a standalone item, it could be triggered as a pre-processor in a request parameter for the second call. This would simplify output formats for 3rd parties.

          +1

          Show
          arafalov Alexandre Rafalovitch added a comment - Ok, that makes sense. And if this URP is defined as a standalone item, it could be triggered as a pre-processor in a request parameter for the second call. This would simplify output formats for 3rd parties. +1
          Hide
          noble.paul Noble Paul added a comment -

          Adding to Alexandre Rafalovitch I would say all URPs should be able to optionally accept request params and they must be available all the time. This will free up users from unnecessarily mucking up with solrconfig.xml

          example:

           /update?preprocessor=atomic&field=my_new_field 
          
          Show
          noble.paul Noble Paul added a comment - Adding to Alexandre Rafalovitch I would say all URPs should be able to optionally accept request params and they must be available all the time. This will free up users from unnecessarily mucking up with solrconfig.xml example: /update?preprocessor=atomic&field=my_new_field
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited

          Hi Varun, Alexandre, Noble,

          SOLR-9530.patch uploaded for a new update processor - AtomicUpdateProcessor which which will accept conventional key-value update document and convert them into atomic update document for the fields specified in the processor definition. Fields which are not specified in the processor parameters will be updated in conventional manner.

          Files specified in the patch:
          1. AtomicUpdateProcessorFactory.java
          2. AtomicUpdateProcessorFactoryTest.java (test class for AtomicUpdateProcessorFactory)
          3. solrconfig-atomic-update-processor.xml (sample solrconfig for AtomicUpdateProcessorFactoryTest test cases)

          As Alexandre mentioned, it will work as a standalone processor doing the conversion and updated document will passed onto next processor defined.
          Noble, this patch right now doesn't support accepting request params as it is difficult to assign atomic operation to the respective field.

          I will request you to review the patch and your feedback will be deeply appreciated.

          Thanks
          Amrit Sarkar

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited Hi Varun, Alexandre, Noble, SOLR-9530 .patch uploaded for a new update processor - AtomicUpdateProcessor which which will accept conventional key-value update document and convert them into atomic update document for the fields specified in the processor definition. Fields which are not specified in the processor parameters will be updated in conventional manner. Files specified in the patch: 1. AtomicUpdateProcessorFactory.java 2. AtomicUpdateProcessorFactoryTest.java (test class for AtomicUpdateProcessorFactory) 3. solrconfig-atomic-update-processor.xml (sample solrconfig for AtomicUpdateProcessorFactoryTest test cases) As Alexandre mentioned, it will work as a standalone processor doing the conversion and updated document will passed onto next processor defined. Noble, this patch right now doesn't support accepting request params as it is difficult to assign atomic operation to the respective field. I will request you to review the patch and your feedback will be deeply appreciated. Thanks Amrit Sarkar
          Hide
          noble.paul Noble Paul added a comment -

          Don't we already support partial updates? Ishan Chattopadhyaya ? What is new in this?

          The configuration is not serving any real purpose. It is better to not accept any initial configuration and just support request params. The URP is one of the most complex configuration Solr has and most of the times they could be just avoided

          Show
          noble.paul Noble Paul added a comment - Don't we already support partial updates? Ishan Chattopadhyaya ? What is new in this? The configuration is not serving any real purpose. It is better to not accept any initial configuration and just support request params. The URP is one of the most complex configuration Solr has and most of the times they could be just avoided
          Hide
          ichattopadhyaya Ishan Chattopadhyaya added a comment - - edited

          As far as I understand, this update processor is only for updates (if document pre-exists). I see this as useful in cases where we're ingesting CSV files with disjoint information about a document in different files. As an example:

          id country
          1 Japan
          2 Russia
          id capital
          1 Tokyo
          2 Moscow

          Needs to be both ingested, and hence if both these are ingested through this Update Processor, we would end up with 2 documents with 3 fields each (id, country, capital).

          Did I understand the motivation correctly?

          Show
          ichattopadhyaya Ishan Chattopadhyaya added a comment - - edited As far as I understand, this update processor is only for updates (if document pre-exists). I see this as useful in cases where we're ingesting CSV files with disjoint information about a document in different files. As an example: id country 1 Japan 2 Russia id capital 1 Tokyo 2 Moscow Needs to be both ingested, and hence if both these are ingested through this Update Processor, we would end up with 2 documents with 3 fields each (id, country, capital). Did I understand the motivation correctly?
          Hide
          varunthacker Varun Thacker added a comment -

          Ishan - Exactly.. That was my motivation behind creating this Jira.

          Show
          varunthacker Varun Thacker added a comment - Ishan - Exactly.. That was my motivation behind creating this Jira.
          Hide
          noble.paul Noble Paul added a comment - - edited

          IIUC , you can use this URP and keep calling ADD instead of SET and achieve the same ?

          Show
          noble.paul Noble Paul added a comment - - edited IIUC , you can use this URP and keep calling ADD instead of SET and achieve the same ?
          Hide
          noble.paul Noble Paul added a comment -

          The description is not clear. This is relevant when the user does not have the choice of using a SET command . for example /update/csv or /update/json/docs .

          The patch is not thread-safe. What if multiple threads try to update the same document in parallel?

          Show
          noble.paul Noble Paul added a comment - The description is not clear. This is relevant when the user does not have the choice of using a SET command . for example /update/csv or /update/json/docs . The patch is not thread-safe. What if multiple threads try to update the same document in parallel?
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited

          Noble Paul Ishan Chattopadhyaya Varun,
          thank you for looking into the patch.

          Noble Paul, the patch is not thread safe as the processor is thread-specific and is mutually independent of other threads. This processor is somehow doing in-place conversion of documents (more like a plugin) and will be passed ultimately to DistributedUpdateProcessor (which is thread-safe), where the actual "atomic" update to the documents in index takes place.

          If multiple threads try to update the same document in parallel via AtomicUpdateProcessor, that case will be similar to multiple threads carrying "atomic-style" update of same document, which is already happening in our latest Solr. DistributedUpdateProcessor, being thread-safe, handles the resources well and don't let the versions conflict each other.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited Noble Paul Ishan Chattopadhyaya Varun, thank you for looking into the patch. Noble Paul , the patch is not thread safe as the processor is thread-specific and is mutually independent of other threads. This processor is somehow doing in-place conversion of documents (more like a plugin) and will be passed ultimately to DistributedUpdateProcessor (which is thread-safe), where the actual "atomic" update to the documents in index takes place. If multiple threads try to update the same document in parallel via AtomicUpdateProcessor, that case will be similar to multiple threads carrying "atomic-style" update of same document, which is already happening in our latest Solr. DistributedUpdateProcessor, being thread-safe, handles the resources well and don't let the versions conflict each other.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Meanwhile as Noble Paul mentioned, if two threads try to update the same SolrInputDocument via AtomicUpdateProcessor, it may fail or give abrupt results. Allow me some time to review this, test it out and make thread-safe if deem necessary. Thank you.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Meanwhile as Noble Paul mentioned, if two threads try to update the same SolrInputDocument via AtomicUpdateProcessor, it may fail or give abrupt results. Allow me some time to review this, test it out and make thread-safe if deem necessary. Thank you.
          Hide
          ichattopadhyaya Ishan Chattopadhyaya added a comment -

          To solve the problem of multiple clients concurrently updating the same document, I think the URP should add the current version to the updated document. If two or more such documents hit DUP with the same version on update, all except the first one will fail with a version conflict and the client could retry that document. Only doing so, i.e. leveraging the optimistic concurrency for atomic updates, would make this URP a truly "atomic" update processor.

          Show
          ichattopadhyaya Ishan Chattopadhyaya added a comment - To solve the problem of multiple clients concurrently updating the same document, I think the URP should add the current version to the updated document. If two or more such documents hit DUP with the same version on update, all except the first one will fail with a version conflict and the client could retry that document. Only doing so, i.e. leveraging the optimistic concurrency for atomic updates, would make this URP a truly "atomic" update processor.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Updated Patch:

          403 Forbidden error will be thrown if the URP receives atomic-type document with fields specified in processor definition. Test cases are added for the same.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Updated Patch: 403 Forbidden error will be thrown if the URP receives atomic-type document with fields specified in processor definition. Test cases are added for the same.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Updated Patch: SOLR-9530.patch

          To comply multiple threads executing Atomic URP don't make incoming doc inconsistent which will eventually end up in DUP, necessary changes are done as follows:

          1. Earlier only conventional updates are allowed through this URP, now atomic updates can be passed.
          2. If the incoming doc contains fields with atomic update and those particular fields are not part of Atomic URP definition, they will be sent as it as to next processor
          3. If the incoming doc contains fields with atomic update and those particular fields are part of Atomic URP definition:
          a. if the atomic-op on field in incoming doc matches with atomic-op in processor definition, pass as it is to the next processor.
          b. if the atomic-op on field in incoming doc doesn't matches with atomic-op in processor definition, throw SolrException (403 Forbidden).

          In case of multiple threads try to update the incoming doc to atomic-type update doc, all the threads will end up forming same atomic-type update doc (as same set of operations will be performed by 'SET' field).

          Noble Paul kindly verify if the above changes satisfies the parallel execution of this URP by multiple threads.

          Ishan Chattopadhyaya regarding including the updated current version of the doc in Atomic URP, DUP itself is performing the same on its end for every atomic update doc, including current version in the doc. If multiple threads executes the update chain (includes the Atomic URP), once it reaches the DUP, it performs exactly the same you have specified i.e all except the first one will fail with a version conflict and the client could retry that document. Kindly review and let me know if my understanding is correct and adding version info in Atomic URP will make any significant difference in this regard.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Updated Patch: SOLR-9530 .patch To comply multiple threads executing Atomic URP don't make incoming doc inconsistent which will eventually end up in DUP, necessary changes are done as follows: 1. Earlier only conventional updates are allowed through this URP, now atomic updates can be passed. 2. If the incoming doc contains fields with atomic update and those particular fields are not part of Atomic URP definition, they will be sent as it as to next processor 3. If the incoming doc contains fields with atomic update and those particular fields are part of Atomic URP definition: a. if the atomic-op on field in incoming doc matches with atomic-op in processor definition, pass as it is to the next processor. b. if the atomic-op on field in incoming doc doesn't matches with atomic-op in processor definition, throw SolrException (403 Forbidden) . In case of multiple threads try to update the incoming doc to atomic-type update doc, all the threads will end up forming same atomic-type update doc (as same set of operations will be performed by 'SET' field). Noble Paul kindly verify if the above changes satisfies the parallel execution of this URP by multiple threads. Ishan Chattopadhyaya regarding including the updated current version of the doc in Atomic URP, DUP itself is performing the same on its end for every atomic update doc, including current version in the doc. If multiple threads executes the update chain (includes the Atomic URP), once it reaches the DUP, it performs exactly the same you have specified i.e all except the first one will fail with a version conflict and the client could retry that document. Kindly review and let me know if my understanding is correct and adding version info in Atomic URP will make any significant difference in this regard.
          Hide
          ichattopadhyaya Ishan Chattopadhyaya added a comment - - edited

          In case of multiple threads try to update the incoming doc to atomic-type update doc, all the threads will end up forming same atomic-type update doc (as same set of operations will be performed by 'SET' field).

          The problem is with "inc" operations. When two clients see the value to be, say 100, and want to increase by 50, they can supply the document version along with "inc":50. One of them would be executed first, and the second one would be rejected since the document version is no longer the same as what this client saw. Without optimistic concurrency, the value will end up being 200, but intended value was 150.

          Also, do consider the cases when one client is indexing without this URP, but another client is using this URP, both in parallel.

          Show
          ichattopadhyaya Ishan Chattopadhyaya added a comment - - edited In case of multiple threads try to update the incoming doc to atomic-type update doc, all the threads will end up forming same atomic-type update doc (as same set of operations will be performed by 'SET' field). The problem is with "inc" operations. When two clients see the value to be, say 100, and want to increase by 50, they can supply the document version along with "inc":50. One of them would be executed first, and the second one would be rejected since the document version is no longer the same as what this client saw. Without optimistic concurrency, the value will end up being 200, but intended value was 150. Also, do consider the cases when one client is indexing without this URP, but another client is using this URP, both in parallel.
          Hide
          noble.paul Noble Paul added a comment - - edited

          Let's get rid of any URP configuration from solrconfig.xml. Let's move everything to parameters and define what is required. The problem is , the config API does not support URP chain and it does not plan to do so. So, let's keep it as simple parameters

          accept params as follows and nuke all the configuration required

          processor=Atomic&Atomic.my_newfield=add&Atomic.subject=set&Atomic.count_i=inc
          
          Show
          noble.paul Noble Paul added a comment - - edited Let's get rid of any URP configuration from solrconfig.xml . Let's move everything to parameters and define what is required. The problem is , the config API does not support URP chain and it does not plan to do so. So, let's keep it as simple parameters accept params as follows and nuke all the configuration required processor=Atomic&Atomic.my_newfield=add&Atomic.subject=set&Atomic.count_i=inc
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited

          Ishan and Noble, the comments are spot-on and thank you for correcting me out.

          I will update the description on not touching solrconfig for this URP and use in-line parameters like Noble mentioned, this makes much more sense. Also suitable examples on what we are trying to achieve here.

          Regarding the optimistic concurrency, I will add the _version_ field in the incoming document which will make sure the second request fails if two updates for a same doc is received via different threads. My doubt is how to deal with the failure, Noble mentioned to retry so that it gets ingested(with latest version info) and the updates get reflected. Ishan mentioned just throw 409 version conflict for the doc and let the user decide whether to ingest again or not for those particular doc-ids.

          Again, I am thankful to both of you for the pointers.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited Ishan and Noble, the comments are spot-on and thank you for correcting me out. I will update the description on not touching solrconfig for this URP and use in-line parameters like Noble mentioned, this makes much more sense. Also suitable examples on what we are trying to achieve here. Regarding the optimistic concurrency, I will add the _version_ field in the incoming document which will make sure the second request fails if two updates for a same doc is received via different threads. My doubt is how to deal with the failure, Noble mentioned to retry so that it gets ingested(with latest version info) and the updates get reflected. Ishan mentioned just throw 409 version conflict for the doc and let the user decide whether to ingest again or not for those particular doc-ids. Again, I am thankful to both of you for the pointers.
          Hide
          noble.paul Noble Paul added a comment -

          I'm not sure if the second request must fail

          imagine a doc already exists with

          {id: 1}
          

          subsequently the user sends 2 parallel requests

          {id:1, firstName: Tom}
          {id:1, lastName: Cruise}
          

          after these two operations are performed the final doc should be

          {id:1, firstName:Tom, lastName:Cruise}
          

          The system should handle race conditions gracefully

          The URP can fetch the _version_ before sending the appropriate atomic operation using optimistic concurrency. if the request fails , it can reload the _version_ and retry

          Show
          noble.paul Noble Paul added a comment - I'm not sure if the second request must fail imagine a doc already exists with {id: 1} subsequently the user sends 2 parallel requests {id:1, firstName: Tom} {id:1, lastName: Cruise} after these two operations are performed the final doc should be {id:1, firstName:Tom, lastName:Cruise} The system should handle race conditions gracefully The URP can fetch the _version_ before sending the appropriate atomic operation using optimistic concurrency. if the request fails , it can reload the _version_ and retry
          Hide
          Hamso Hamso added a comment -

          I have a question.
          What is if we have:

          doc1

          {id: 1, Street: xyz}
          

          and doc2

          {id: 1, firstName: Tom, lastName: Cruise}
          {id: 1, firstName: Max, lastName: Mueller}
          

          How will the final doc look like?

          Show
          Hamso Hamso added a comment - I have a question. What is if we have: doc1 {id: 1, Street: xyz} and doc2 {id: 1, firstName: Tom, lastName: Cruise} {id: 1, firstName: Max, lastName: Mueller} How will the final doc look like?
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Hamso

          Depending on what atomic operation you are applying on doc2:

          if "add" for both fields, considering firstName and lastName are multiValued:

          {id: 1, Street: xyz, firstName: [Tom,Max], lastName: [Cruise,Mueller]}
          

          if not, it will throw error, multiple values not supported for single-valued fields

          if "set" for both fields or any one of them,

          {id: 1, Street: xyz, firstName: Max, lastName: Mueller}
          

          It will be considered as atomic-update document at DUP.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Hamso Depending on what atomic operation you are applying on doc2: if "add" for both fields, considering firstName and lastName are multiValued: {id: 1, Street: xyz, firstName: [Tom,Max], lastName: [Cruise,Mueller]} if not, it will throw error, multiple values not supported for single-valued fields if "set" for both fields or any one of them, {id: 1, Street: xyz, firstName: Max, lastName: Mueller} It will be considered as atomic-update document at DUP.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited

          As discussed, the system will handle race conditions gracefully. The URP will fetch the version before sending the appropriate atomic operation using optimistic concurrency. if the request fails , it with retry with updated version.

          Working on the latest developments.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited As discussed, the system will handle race conditions gracefully. The URP will fetch the version before sending the appropriate atomic operation using optimistic concurrency. if the request fails , it with retry with updated version . Working on the latest developments.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited

          Considering Noble's and Ishan's suggestions, cooked up a new patch with the following:

          1. No solrconfig parameter(s) required for this URP now.

          2. The URP will take inline parameters exactly as Noble mentioned:

          processor=Atomic&Atomic.my_newfield=add&Atomic.subject=set&Atomic.count_i=inc

          3. Both atomic and conventional updates as incoming documents to the URP are allowed.
          a. for atomic updates, the atomic operation in incoming doc should match with the parameters specified in processor call.
          e.g. {"id":"1","title":{"set":"A"}} || processor=Atomic&Atomic.title=set

          4. After the conversion to atomic-style, latest version will be added in the updated doc. If version, not present, send as it is.

          5. if the update faces version conflict, retry by fetching latest version from index, updating the SolrInputDoc. Maximum retries set to 5, hardcoded.

          6. If the parameters are not sufficient to convert incoming document to atomic-style, abort the update.
          e.g

          {"id":"1","title":"A"}

          || processor=Atomic&Atomic.subject=set
          there is no point sending this document for update via URP

          new file:   solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateProcessorFactory.java
          new file:   solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java
          

          Tried to write a test case for multiple threads executing URP simultaneously, but was not able to replicate the scenario exactly. The test-method is commented out in the patch.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited Considering Noble's and Ishan's suggestions, cooked up a new patch with the following: 1. No solrconfig parameter(s) required for this URP now. 2. The URP will take inline parameters exactly as Noble mentioned: processor=Atomic&Atomic.my_newfield=add&Atomic.subject=set&Atomic.count_i=inc 3. Both atomic and conventional updates as incoming documents to the URP are allowed. a. for atomic updates, the atomic operation in incoming doc should match with the parameters specified in processor call. e.g. {"id":"1","title":{"set":"A"}} || processor=Atomic&Atomic.title=set 4. After the conversion to atomic-style, latest version will be added in the updated doc. If version , not present, send as it is. 5. if the update faces version conflict, retry by fetching latest version from index, updating the SolrInputDoc. Maximum retries set to 5, hardcoded. 6. If the parameters are not sufficient to convert incoming document to atomic-style, abort the update. e.g {"id":"1","title":"A"} || processor=Atomic&Atomic.subject=set there is no point sending this document for update via URP new file: solr/core/src/java/org/apache/solr/update/processor/AtomicUpdateProcessorFactory.java new file: solr/core/src/test/org/apache/solr/update/processor/AtomicUpdateProcessorFactoryTest.java Tried to write a test case for multiple threads executing URP simultaneously, but was not able to replicate the scenario exactly. The test-method is commented out in the patch.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          As per discussion with Noble,

          Refactored the code to optimise and remove unwanted elements.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - As per discussion with Noble, Refactored the code to optimise and remove unwanted elements.
          Hide
          ichattopadhyaya Ishan Chattopadhyaya added a comment -

          Updated the patch with formatting fixes:
          Please always use braces around if/else/for blocks, even if they are one-liners.

          I think a test should be added to validate the retry logic. The test should use parallel threads to update different fields of the same document and assert that both updates are made to the document finally.

          Show
          ichattopadhyaya Ishan Chattopadhyaya added a comment - Updated the patch with formatting fixes: Please always use braces around if/else/for blocks, even if they are one-liners. I think a test should be added to validate the retry logic. The test should use parallel threads to update different fields of the same document and assert that both updates are made to the document finally.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Considering Noble's and Ishan's feedback again,

          removed all unwanted checks, reused elements and introduced multithreaded thread (I have used the most basic example for this, can be better). I have included two checks, for fieldName and atomicOperation passed in parameters, both will be verified later in DUP, but to me it made a lot of sense sending along a directed error message from this URP itself before processing any document altogether.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Considering Noble's and Ishan's feedback again, removed all unwanted checks, reused elements and introduced multithreaded thread (I have used the most basic example for this, can be better). I have included two checks, for fieldName and atomicOperation passed in parameters, both will be verified later in DUP, but to me it made a lot of sense sending along a directed error message from this URP itself before processing any document altogether.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Updated patch as per Noble's suggestions.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Updated patch as per Noble's suggestions.
          Hide
          noble.paul Noble Paul added a comment -

          added some optimizations to the patch

          Show
          noble.paul Noble Paul added a comment - added some optimizations to the patch
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Been a while posted a patch with the updates.

          Following up with Noble's patch, corrected test methodsl

          Small concern:
          I have a "nocommit" in one of the test methods where we need to issue explicit commit. I don't know how to do that via SolrTestCaseJ4. the "commit()" method doesn't work.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Been a while posted a patch with the updates. Following up with Noble's patch, corrected test methodsl Small concern: I have a "nocommit" in one of the test methods where we need to issue explicit commit. I don't know how to do that via SolrTestCaseJ4. the "commit()" method doesn't work.
          Hide
          erickerickson Erick Erickson added a comment -

          Amrit:

          Possibly SOLR-10562? The commit should work, but apparently there's a weird interaction with reload. Why does the test use reload() rather than just a commit()? I didn't really look the patch over, but that shouldn't be necessary when just updating documents.

          1> Simplest (and not what I recommend) is just to put a sleep in FOR TESTING ONLY to see if this the same or similar problem.

          2> Put in a retry loop something like this until we get to the bottom of 10562:

          commit()
          for (int idx = 0; idx < 100; idx++) {
             if (test succeeds ) { // do not assert
                break; 
             }
             sleep(100);
          }
          // This assert will fail the test if we make it through 10 seconds of sleep/test cycles above.
          assertQ("Check the total number of docs",
                  req("q", "id:10"), "//result[@numFound=1]");
          
          
          Show
          erickerickson Erick Erickson added a comment - Amrit: Possibly SOLR-10562 ? The commit should work, but apparently there's a weird interaction with reload. Why does the test use reload() rather than just a commit()? I didn't really look the patch over, but that shouldn't be necessary when just updating documents. 1> Simplest (and not what I recommend) is just to put a sleep in FOR TESTING ONLY to see if this the same or similar problem. 2> Put in a retry loop something like this until we get to the bottom of 10562: commit() for ( int idx = 0; idx < 100; idx++) { if (test succeeds ) { // do not assert break ; } sleep(100); } // This assert will fail the test if we make it through 10 seconds of sleep/test cycles above. assertQ( "Check the total number of docs" , req( "q" , "id:10" ), " //result[@numFound=1]" );
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Thank you Erick for the above suggestions,

          I digged more and used commitWithin set it to 1 ms and put a sleep of 500 ms and I am passing through the tests without reload. This is obviously not a good manner to write a test as it will be machine subjective. Seeking suggestions if there is a better way or should I replace with above suggestion.

          New patch uploaded.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Thank you Erick for the above suggestions, I digged more and used commitWithin set it to 1 ms and put a sleep of 500 ms and I am passing through the tests without reload. This is obviously not a good manner to write a test as it will be machine subjective. Seeking suggestions if there is a better way or should I replace with above suggestion. New patch uploaded.
          Hide
          erickerickson Erick Erickson added a comment -

          1 ms is much too short. I'd just issue the commit() when done sending docs then the wait.

          The question is did your change work? As in prevent errors? If so let's try the loop above as a stop-gap.

          Show
          erickerickson Erick Erickson added a comment - 1 ms is much too short. I'd just issue the commit() when done sending docs then the wait. The question is did your change work? As in prevent errors? If so let's try the loop above as a stop-gap.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Yeah it worked. If I put commitwithin anything > 0 say X, and put sleep for Y >> X, I don't need to reload or issue commit() and it is working as expected.

          Well I also put ModifiableSolrParam, "commit", "true" like we do in a curl update. That too is not getting picked for explicit commit by the test.

           AddUpdateCommand cmd = new AddUpdateCommand(new LocalSolrQueryRequest(h.getCore(),
                  new ModifiableSolrParams()
                      .add("processor", "Atomic")
                      .add("Atomic.cat", "delete")
                      .add("commit","true")
              ));
          
          factory.getInstance(cmd.getReq(), new SolrQueryResponse(),
                          new DistributedUpdateProcessor(cmd.getReq(), new SolrQueryResponse(),
                              new RunUpdateProcessor(cmd.getReq(), null))).processAdd(cmd);
          

          I like the stop-gap loop, retries are better, will incorporate this. Thank you.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Yeah it worked. If I put commitwithin anything > 0 say X, and put sleep for Y >> X, I don't need to reload or issue commit() and it is working as expected. Well I also put ModifiableSolrParam, "commit", "true" like we do in a curl update. That too is not getting picked for explicit commit by the test. AddUpdateCommand cmd = new AddUpdateCommand( new LocalSolrQueryRequest(h.getCore(), new ModifiableSolrParams() .add( "processor" , "Atomic" ) .add( "Atomic.cat" , "delete" ) .add( "commit" , " true " ) )); factory.getInstance(cmd.getReq(), new SolrQueryResponse(), new DistributedUpdateProcessor(cmd.getReq(), new SolrQueryResponse(), new RunUpdateProcessor(cmd.getReq(), null ))).processAdd(cmd); I like the stop-gap loop, retries are better, will incorporate this. Thank you.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          Figured out. commit() itself doesn't work but assertU(commit()).

          I have attached screenshots for the same, looked into assertU(..) implementation, don't know what special is there. Thank you Erick for looking into it.

          Updated patch, Noble I think we got everything correct this time. Kindly review whenever you get time.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - Figured out. commit() itself doesn't work but assertU(commit()) . I have attached screenshots for the same, looked into assertU(..) implementation, don't know what special is there. Thank you Erick for looking into it. Updated patch, Noble I think we got everything correct this time. Kindly review whenever you get time.
          Hide
          noble.paul Noble Paul added a comment -

          Amrit Sarkar going through the patch. I like the clever multithreaded test

          Show
          noble.paul Noble Paul added a comment - Amrit Sarkar going through the patch. I like the clever multithreaded test
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit dfe41f1e1e838d974ddb64297ca3d65d6c097ead in lucene-solr's branch refs/heads/branch_6x from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dfe41f1 ]

          SOLR-9530: An Update Processor to convert normal update operation to an atomic operations such as add, set,inc, remove ,set, removeregex

          Show
          jira-bot ASF subversion and git services added a comment - Commit dfe41f1e1e838d974ddb64297ca3d65d6c097ead in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dfe41f1 ] SOLR-9530 : An Update Processor to convert normal update operation to an atomic operations such as add, set,inc, remove ,set, removeregex
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit faa74ec7dcddb9fccf240e81a36b21f90336726c in lucene-solr's branch refs/heads/master from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=faa74ec ]

          SOLR-9530: An Update Processor to convert normal update operation to an atomic operations such as add, set,inc, remove ,set, removeregex

          Show
          jira-bot ASF subversion and git services added a comment - Commit faa74ec7dcddb9fccf240e81a36b21f90336726c in lucene-solr's branch refs/heads/master from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=faa74ec ] SOLR-9530 : An Update Processor to convert normal update operation to an atomic operations such as add, set,inc, remove ,set, removeregex
          Hide
          steve_rowe Steve Rowe added a comment -

          UpdateRequestProcessorFactoryTest.testRequestTimeUrp() is failing regularly in Jenkins after this commit, e.g. from https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/850/:

             [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UpdateRequestProcessorFactoryTest -Dtests.method=testRequestTimeUrp -Dtests.seed=34D1B16033B852CD -Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=PRT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
             [junit4] ERROR   0.01s J0 | UpdateRequestProcessorFactoryTest.testRequestTimeUrp <<<
             [junit4]    > Throwable #1: org.apache.solr.common.SolrException: No such processor Template
             [junit4]    > 	at __randomizedtesting.SeedInfo.seed([34D1B16033B852CD:9163F1A895F69B39]:0)
             [junit4]    > 	at org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286)
             [junit4]    > 	at org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235)
             [junit4]    > 	at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1413)
             [junit4]    > 	at org.apache.solr.update.processor.UpdateRequestProcessorFactoryTest.testRequestTimeUrp(UpdateRequestProcessorFactoryTest.java:53)
          

          This test fails for me without a seed.

          Show
          steve_rowe Steve Rowe added a comment - UpdateRequestProcessorFactoryTest.testRequestTimeUrp() is failing regularly in Jenkins after this commit, e.g. from https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/850/ : [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=UpdateRequestProcessorFactoryTest -Dtests.method=testRequestTimeUrp -Dtests.seed=34D1B16033B852CD -Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=PRT -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [junit4] ERROR 0.01s J0 | UpdateRequestProcessorFactoryTest.testRequestTimeUrp <<< [junit4] > Throwable #1: org.apache.solr.common.SolrException: No such processor Template [junit4] > at __randomizedtesting.SeedInfo.seed([34D1B16033B852CD:9163F1A895F69B39]:0) [junit4] > at org.apache.solr.update.processor.UpdateRequestProcessorChain.getReqProcessors(UpdateRequestProcessorChain.java:286) [junit4] > at org.apache.solr.update.processor.UpdateRequestProcessorChain.constructChain(UpdateRequestProcessorChain.java:235) [junit4] > at org.apache.solr.core.SolrCore.getUpdateProcessorChain(SolrCore.java:1413) [junit4] > at org.apache.solr.update.processor.UpdateRequestProcessorFactoryTest.testRequestTimeUrp(UpdateRequestProcessorFactoryTest.java:53) This test fails for me without a seed.
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited

          Ah! That's fault at my part. We made some changes in URPFactory to make all URPs SolrCoreAware which essentially needs Solr to start with system property "enable.runtime.lib=true".

          I have set it explicitly in beforeClass in URPFactoryTest and all the test cases are passing successfully now. Sorry for the hiccup.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - - edited Ah! That's fault at my part. We made some changes in URPFactory to make all URPs SolrCoreAware which essentially needs Solr to start with system property "enable.runtime.lib=true". I have set it explicitly in beforeClass in URPFactoryTest and all the test cases are passing successfully now. Sorry for the hiccup.
          Hide
          steve_rowe Steve Rowe added a comment -

          My Jenkins found a reproducing seed for several AtomicUpdateProcessorFactoryTest failures:

          Checking out Revision 52b252cc6eddd7bcb71bdf2f9c533332fe228b0e (refs/remotes/origin/master)
          [...]
             [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testBasics -Dtests.seed=9F9128B8E3E8FAA7 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=US/Samoa -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
             [junit4] ERROR   0.01s J11 | AtomicUpdateProcessorFactoryTest.testBasics <<<
             [junit4]    > Throwable #1: org.apache.solr.common.SolrException: Atomic document updates are not supported unless <updateLog/> is configured
             [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F9128B8E3E8FAA7:A2498694DB06A4D7]:0)
             [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactory.getInstance(AtomicUpdateProcessorFactory.java:88)
             [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testBasics(AtomicUpdateProcessorFactoryTest.java:106)
          [...]
             [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testWrongAtomicOpPassed -Dtests.seed=9F9128B8E3E8FAA7 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=US/Samoa -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
             [junit4] FAILURE 0.00s J11 | AtomicUpdateProcessorFactoryTest.testWrongAtomicOpPassed <<<
             [junit4]    > Throwable #1: org.junit.ComparisonFailure: expected:<[Unexpected param(s) for AtomicUpdateProcessor, invalid atomic op passed: 'delete']> but was:<[Atomic document updates are not supported unless <updateLog/> is configured]>
             [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F9128B8E3E8FAA7:4E887D44E42A13F4]:0)
             [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testWrongAtomicOpPassed(AtomicUpdateProcessorFactoryTest.java:57)
          [...]
             [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testMultipleThreads -Dtests.seed=9F9128B8E3E8FAA7 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=US/Samoa -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
             [junit4] ERROR   0.01s J11 | AtomicUpdateProcessorFactoryTest.testMultipleThreads <<<
             [junit4]    > Throwable #1: org.apache.solr.common.SolrException: Atomic document updates are not supported unless <updateLog/> is configured
             [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F9128B8E3E8FAA7:B3A35F356F2C8C23]:0)
             [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactory.getInstance(AtomicUpdateProcessorFactory.java:88)
             [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest$1.run(AtomicUpdateProcessorFactoryTest.java:233)
             [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:240)
          [...]
             [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testNoUniqueIdPassed -Dtests.seed=9F9128B8E3E8FAA7 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=US/Samoa -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
             [junit4] FAILURE 0.00s J11 | AtomicUpdateProcessorFactoryTest.testNoUniqueIdPassed <<<
             [junit4]    > Throwable #1: org.junit.ComparisonFailure: expected:<[Document passed with no unique field: 'id']> but was:<[Atomic document updates are not supported unless <updateLog/> is configured]>
             [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F9128B8E3E8FAA7:62C4243917AB64F1]:0)
             [junit4]    > 	at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testNoUniqueIdPassed(AtomicUpdateProcessorFactoryTest.java:79)
          [...]
             [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=10, maxDocsPerChunk=8, blockSize=5), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=10, blockSize=5)), sim=RandomSimilarity(queryNorm=true): {}, locale=th-TH, timezone=US/Samoa
             [junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 (64-bit)/cpus=16,threads=1,free=238618688,total=523239424
          
          Show
          steve_rowe Steve Rowe added a comment - My Jenkins found a reproducing seed for several AtomicUpdateProcessorFactoryTest failures: Checking out Revision 52b252cc6eddd7bcb71bdf2f9c533332fe228b0e (refs/remotes/origin/master) [...] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testBasics -Dtests.seed=9F9128B8E3E8FAA7 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=US/Samoa -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 0.01s J11 | AtomicUpdateProcessorFactoryTest.testBasics <<< [junit4] > Throwable #1: org.apache.solr.common.SolrException: Atomic document updates are not supported unless <updateLog/> is configured [junit4] > at __randomizedtesting.SeedInfo.seed([9F9128B8E3E8FAA7:A2498694DB06A4D7]:0) [junit4] > at org.apache.solr.update.processor.AtomicUpdateProcessorFactory.getInstance(AtomicUpdateProcessorFactory.java:88) [junit4] > at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testBasics(AtomicUpdateProcessorFactoryTest.java:106) [...] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testWrongAtomicOpPassed -Dtests.seed=9F9128B8E3E8FAA7 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=US/Samoa -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] FAILURE 0.00s J11 | AtomicUpdateProcessorFactoryTest.testWrongAtomicOpPassed <<< [junit4] > Throwable #1: org.junit.ComparisonFailure: expected:<[Unexpected param(s) for AtomicUpdateProcessor, invalid atomic op passed: 'delete']> but was:<[Atomic document updates are not supported unless <updateLog/> is configured]> [junit4] > at __randomizedtesting.SeedInfo.seed([9F9128B8E3E8FAA7:4E887D44E42A13F4]:0) [junit4] > at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testWrongAtomicOpPassed(AtomicUpdateProcessorFactoryTest.java:57) [...] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testMultipleThreads -Dtests.seed=9F9128B8E3E8FAA7 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=US/Samoa -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 0.01s J11 | AtomicUpdateProcessorFactoryTest.testMultipleThreads <<< [junit4] > Throwable #1: org.apache.solr.common.SolrException: Atomic document updates are not supported unless <updateLog/> is configured [junit4] > at __randomizedtesting.SeedInfo.seed([9F9128B8E3E8FAA7:B3A35F356F2C8C23]:0) [junit4] > at org.apache.solr.update.processor.AtomicUpdateProcessorFactory.getInstance(AtomicUpdateProcessorFactory.java:88) [junit4] > at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest$1.run(AtomicUpdateProcessorFactoryTest.java:233) [junit4] > at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testMultipleThreads(AtomicUpdateProcessorFactoryTest.java:240) [...] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=AtomicUpdateProcessorFactoryTest -Dtests.method=testNoUniqueIdPassed -Dtests.seed=9F9128B8E3E8FAA7 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=US/Samoa -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] FAILURE 0.00s J11 | AtomicUpdateProcessorFactoryTest.testNoUniqueIdPassed <<< [junit4] > Throwable #1: org.junit.ComparisonFailure: expected:<[Document passed with no unique field: 'id']> but was:<[Atomic document updates are not supported unless <updateLog/> is configured]> [junit4] > at __randomizedtesting.SeedInfo.seed([9F9128B8E3E8FAA7:62C4243917AB64F1]:0) [junit4] > at org.apache.solr.update.processor.AtomicUpdateProcessorFactoryTest.testNoUniqueIdPassed(AtomicUpdateProcessorFactoryTest.java:79) [...] [junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=10, maxDocsPerChunk=8, blockSize=5), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=10, blockSize=5)), sim=RandomSimilarity(queryNorm=true): {}, locale=th-TH, timezone=US/Samoa [junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 (64-bit)/cpus=16,threads=1,free=238618688,total=523239424
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 468d479b5f90d6bf3d1ae5f62019956b76fa2a25 in lucene-solr's branch refs/heads/master from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=468d479 ]

          SOLR-9530: fixing test error

          Show
          jira-bot ASF subversion and git services added a comment - Commit 468d479b5f90d6bf3d1ae5f62019956b76fa2a25 in lucene-solr's branch refs/heads/master from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=468d479 ] SOLR-9530 : fixing test error
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1f39b5175122735e3012f6400244e8b3a9481a3f in lucene-solr's branch refs/heads/branch_6x from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1f39b51 ]

          SOLR-9530: fixing test error

          Show
          jira-bot ASF subversion and git services added a comment - Commit 1f39b5175122735e3012f6400244e8b3a9481a3f in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1f39b51 ] SOLR-9530 : fixing test error
          Hide
          sarkaramrit2@gmail.com Amrit Sarkar added a comment -

          The solrconfig used in the test has updateLog setting as:

              <updateLog enable="${enable.update.log:true}">
                <str name="dir">${solr.ulog.dir:}</str>
              </updateLog> 
          

          So the seed 9F9128B8E3E8FAA7 setting the "enable.update.log=false" making all the test methods break. Rightfully!

          Again explicitly set "enable.update.log=true" in beforeClass of AUPTest in current patch, as it is must.

          Show
          sarkaramrit2@gmail.com Amrit Sarkar added a comment - The solrconfig used in the test has updateLog setting as: <updateLog enable= "${enable.update.log: true }" > <str name= "dir" >${solr.ulog.dir:}</str> </updateLog> So the seed 9F9128B8E3E8FAA7 setting the "enable.update.log=false" making all the test methods break. Rightfully! Again explicitly set "enable.update.log=true" in beforeClass of AUPTest in current patch, as it is must.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 58837104abf9efda84a2969557046dc02e7fffb1 in lucene-solr's branch refs/heads/master from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5883710 ]

          SOLR-9530: addressing test failures with seed 9F9128B8E3E8FAA7

          Show
          jira-bot ASF subversion and git services added a comment - Commit 58837104abf9efda84a2969557046dc02e7fffb1 in lucene-solr's branch refs/heads/master from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5883710 ] SOLR-9530 : addressing test failures with seed 9F9128B8E3E8FAA7
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit f4d204650b467aabe37f49b94813dc1576747fc1 in lucene-solr's branch refs/heads/branch_6x from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f4d2046 ]

          SOLR-9530: addressing test failures with seed 9F9128B8E3E8FAA7

          Show
          jira-bot ASF subversion and git services added a comment - Commit f4d204650b467aabe37f49b94813dc1576747fc1 in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f4d2046 ] SOLR-9530 : addressing test failures with seed 9F9128B8E3E8FAA7
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit fa76171a63cca0b42f81213f4cf1284bdaab6b63 in lucene-solr's branch refs/heads/master from Cassandra Targett
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fa76171 ]

          Ref Guide: add AtomicUpdateRequestProcessorFactory from SOLR-9530

          Show
          jira-bot ASF subversion and git services added a comment - Commit fa76171a63cca0b42f81213f4cf1284bdaab6b63 in lucene-solr's branch refs/heads/master from Cassandra Targett [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fa76171 ] Ref Guide: add AtomicUpdateRequestProcessorFactory from SOLR-9530
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 31e02e93a5712d8fa2cafff3b13ce329563c6579 in lucene-solr's branch refs/heads/master from Cassandra Targett
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=31e02e9 ]

          Ref Guide: fix note for atomic updates after SOLR-9530

          Show
          jira-bot ASF subversion and git services added a comment - Commit 31e02e93a5712d8fa2cafff3b13ce329563c6579 in lucene-solr's branch refs/heads/master from Cassandra Targett [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=31e02e9 ] Ref Guide: fix note for atomic updates after SOLR-9530
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit ce5b184b17fb000f9d3469807872df30e1550397 in lucene-solr's branch refs/heads/master from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ce5b184 ]

          SOLR-9530: documentation added

          Show
          jira-bot ASF subversion and git services added a comment - Commit ce5b184b17fb000f9d3469807872df30e1550397 in lucene-solr's branch refs/heads/master from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ce5b184 ] SOLR-9530 : documentation added

            People

            • Assignee:
              noble.paul Noble Paul
              Reporter:
              varunthacker Varun Thacker
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development