Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-1529

NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4
    • Fix Version/s: 1.4
    • Component/s: None
    • Labels:
      None

      Description

      The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

      1. logger.patch
        0.8 kB
        Asmodean

        Activity

        Hide
        gsingers Grant Ingersoll added a comment -

        Bulk close for Solr 1.4

        Show
        gsingers Grant Ingersoll added a comment - Bulk close for Solr 1.4
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Thanks, I just committed your patch (though I had to apply it by hand).

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Thanks, I just committed your patch (though I had to apply it by hand).
        Hide
        asmodean Asmodean added a comment - - edited

        Just to clarify from our point of view:

        What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors, we simply get an exception and have to rollback). It might for example look like the following sample code:

        PSEUDO-CODE

        try {
          while (hasMoreDeletesToProcessFromOurApplication) {
            if (requestBatchMaxSizeReached) {
              break;
            }
        
            request.deleteByQuery(...)
          }
        
          request.process(solrServer)
          solServer.commit()
        } catch (Exception e) {
          // SolrServerException or IOException
        
          solrServer.rollback();
          throw e;
        }
        
        
        Show
        asmodean Asmodean added a comment - - edited Just to clarify from our point of view: What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors, we simply get an exception and have to rollback). It might for example look like the following sample code: PSEUDO-CODE try { while (hasMoreDeletesToProcessFromOurApplication) { if (requestBatchMaxSizeReached) { break ; } request.deleteByQuery(...) } request.process(solrServer) solServer.commit() } catch (Exception e) { // SolrServerException or IOException solrServer.rollback(); throw e; }
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        In case it wasn't clear - I don't consider this a 1.4 showstopper.

        Show
        yseeley@gmail.com Yonik Seeley added a comment - In case it wasn't clear - I don't consider this a 1.4 showstopper.
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Does the update succeed in spite of the NPE, or does the entire update fail?

        The deletes should still succeed - the logger chains before it does stuff.

        Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)

        It does indeed look like it's been around since the beginning (7/07)

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Does the update succeed in spite of the NPE, or does the entire update fail? The deletes should still succeed - the logger chains before it does stuff. Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while) It does indeed look like it's been around since the beginning (7/07)
        Hide
        hossman Hoss Man added a comment - - edited

        I'm not at a machine where i can try to reproduce this, but i want to try and understand the severity ... looking at LogUPdateProcessorFactory via viewvc, I believe what this bug is reporting is that if a single request includes more then 8 (maxNumToLog) deleteByQuery directives, w/o including any deleteById directives, then the logger will generate an NPE. (regardless of how many docs were deleted)

        Which means something like this would probably be a way to recreate it in the example...

        java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query>
                   <query>X</query><query>X</query><query>X</query><query>X</query>
                   <query>X</query><query>X</query><query>X</query></delete>"
        

        ... and adding a single <id> entry (even if bogus, or possibly blank) would work arround the problem....

        java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query>
                     <query>X</query><query>X</query><query>X</query><query>X</query>
                     <query>X</query><query>X</query><query>X</query><id /></delete>"
        
        java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query>
                     <query>X</query><query>X</query><query>X</query><query>X</query>
                     <query>X</query><query>X</query><query>X</query><id>NOT_A_REAL_ID</id></delete>"
        

        ...is that all correct?

        Questions should influence whether we consider this a show stopper for 1.4...

        • Does the update succeed in spite of the NPE, or does the entire update fail?
        • Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)
        Show
        hossman Hoss Man added a comment - - edited I'm not at a machine where i can try to reproduce this, but i want to try and understand the severity ... looking at LogUPdateProcessorFactory via viewvc, I believe what this bug is reporting is that if a single request includes more then 8 (maxNumToLog) deleteByQuery directives, w/o including any deleteById directives, then the logger will generate an NPE. (regardless of how many docs were deleted) Which means something like this would probably be a way to recreate it in the example... java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query></delete>" ... and adding a single <id> entry (even if bogus, or possibly blank) would work arround the problem.... java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><id /></delete>" java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><query>X</query> <query>X</query><query>X</query><query>X</query><id>NOT_A_REAL_ID</id></delete>" ...is that all correct? Questions should influence whether we consider this a show stopper for 1.4... Does the update succeed in spite of the NPE, or does the entire update fail? Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)
        Hide
        asmodean Asmodean added a comment -

        Simple patch that will avoid the NullPointerException, but doesn't fix the issue.

        Show
        asmodean Asmodean added a comment - Simple patch that will avoid the NullPointerException, but doesn't fix the issue.

          People

          • Assignee:
            yseeley@gmail.com Yonik Seeley
            Reporter:
            asmodean Asmodean
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development