Solr
  1. Solr
  2. SOLR-1529

NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4
    • Fix Version/s: 1.4
    • Component/s: None
    • Labels:
      None

      Description

      The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

      1. logger.patch
        0.8 kB
        Asmodean

        Activity

        Hide
        Asmodean added a comment -

        Simple patch that will avoid the NullPointerException, but doesn't fix the issue.

        Show
        Asmodean added a comment - Simple patch that will avoid the NullPointerException, but doesn't fix the issue.
        Hide
        Hoss Man added a comment - - edited

        I'm not at a machine where i can try to reproduce this, but i want to try and understand the severity ... looking at LogUPdateProcessorFactory via viewvc, I believe what this bug is reporting is that if a single request includes more then 8 (maxNumToLog) deleteByQuery directives, w/o including any deleteById directives, then the logger will generate an NPE. (regardless of how many docs were deleted)

        Which means something like this would probably be a way to recreate it in the example...

        java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query>
                   <query>X</query><query>X</query><query>X</query><query>X</query>
                   <query>X</query><query>X</query><query>X</query></delete>"
        

        ... and adding a single <id> entry (even if bogus, or possibly blank) would work arround the problem....

        java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query>
                     <query>X</query><query>X</query><query>X</query><query>X</query>
                     <query>X</query><query>X</query><query>X</query><id /></delete>"
        
        java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query>
                     <query>X</query><query>X</query><query>X</query><query>X</query>
                     <query>X</query><query>X</query><query>X</query><id>NOT_A_REAL_ID</id></delete>"
        

        ...is that all correct?

        Questions should influence whether we consider this a show stopper for 1.4...

        • Does the update succeed in spite of the NPE, or does the entire update fail?
        • Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)
        Show
        Hoss Man added a comment - - edited I'm not at a machine where i can try to reproduce this, but i want to try and understand the severity ... looking at LogUPdateProcessorFactory via viewvc, I believe what this bug is reporting is that if a single request includes more then 8 (maxNumToLog) deleteByQuery directives, w/o including any deleteById directives, then the logger will generate an NPE. (regardless of how many docs were deleted) Which means something like this would probably be a way to recreate it in the example... java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query></delete>" ... and adding a single <id> entry (even if bogus, or possibly blank) would work arround the problem.... java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><id /></delete>" java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><id>NOT_A_REAL_ID</id></delete>" ...is that all correct? Questions should influence whether we consider this a show stopper for 1.4... Does the update succeed in spite of the NPE, or does the entire update fail? Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)
        Hide
        Yonik Seeley added a comment -

        Does the update succeed in spite of the NPE, or does the entire update fail?

        The deletes should still succeed - the logger chains before it does stuff.

        Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)

        It does indeed look like it's been around since the beginning (7/07)

        Show
        Yonik Seeley added a comment - Does the update succeed in spite of the NPE, or does the entire update fail? The deletes should still succeed - the logger chains before it does stuff. Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while) It does indeed look like it's been around since the beginning (7/07)
        Hide
        Yonik Seeley added a comment -

        In case it wasn't clear - I don't consider this a 1.4 showstopper.

        Show
        Yonik Seeley added a comment - In case it wasn't clear - I don't consider this a 1.4 showstopper.
        Hide
        Asmodean added a comment - - edited

        Just to clarify from our point of view:

        What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors, we simply get an exception and have to rollback). It might for example look like the following sample code:

        PSEUDO-CODE

        
        try {
          while (hasMoreDeletesToProcessFromOurApplication) {
            if (requestBatchMaxSizeReached) {
              break;
            }
        
            request.deleteByQuery(...)
          }
        
          request.process(solrServer)
          solServer.commit()
        } catch (Exception e) {
          // SolrServerException or IOException
        
          solrServer.rollback();
          throw e;
        }
        
        
        Show
        Asmodean added a comment - - edited Just to clarify from our point of view: What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors, we simply get an exception and have to rollback). It might for example look like the following sample code: PSEUDO-CODE try { while (hasMoreDeletesToProcessFromOurApplication) { if (requestBatchMaxSizeReached) { break ; } request.deleteByQuery(...) } request.process(solrServer) solServer.commit() } catch (Exception e) { // SolrServerException or IOException solrServer.rollback(); throw e; }
        Hide
        Yonik Seeley added a comment -

        Thanks, I just committed your patch (though I had to apply it by hand).

        Show
        Yonik Seeley added a comment - Thanks, I just committed your patch (though I had to apply it by hand).
        Hide
        Grant Ingersoll added a comment -

        Bulk close for Solr 1.4

        Show
        Grant Ingersoll added a comment - Bulk close for Solr 1.4

          People

          • Assignee:
            Yonik Seeley
            Reporter:
            Asmodean
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development