Solr
  1. Solr
  2. SOLR-5535

The "partialResults" header should be set for shards that error out using shards.tolerant

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.5
    • Fix Version/s: 4.7, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently the only way to know if shards error out while using the shards.tolerant flag is to set the shards.info flag and iterate through each shard's info to see if an error is reported.

      The "partialResults" response header value should be set when errors are detected from distributed searches. This header value is currently being used by the timeAllowed request parameter if shards don't respond in time. This change will provide a more consistent mechanism to detect distributed search errors.

      1. SOLR-5535.patch
        4 kB
        Steve Davids
      2. SOLR-5535-testfix.patch
        0.9 kB
        Shalin Shekhar Mangar

        Activity

        Hide
        Shalin Shekhar Mangar added a comment -

        You're right Steve. I agree that we should set this header. Another reason is that in the absence of this header, http cache headers are set on the response. We shouldn't set http cache headers if some shards had failed to respond.

        Do you have a patch?

        Show
        Shalin Shekhar Mangar added a comment - You're right Steve. I agree that we should set this header. Another reason is that in the absence of this header, http cache headers are set on the response. We shouldn't set http cache headers if some shards had failed to respond. Do you have a patch?
        Hide
        Steve Davids added a comment -

        No, I don't have a patch although I can hopefully create one sometime next week.

        Show
        Steve Davids added a comment - No, I don't have a patch although I can hopefully create one sometime next week.
        Hide
        Steve Davids added a comment -

        Attached a patch which reports partial results for both standard and grouped search requests.

        Show
        Steve Davids added a comment - Attached a patch which reports partial results for both standard and grouped search requests.
        Hide
        Steve Davids added a comment -

        [~shalinda] Does the patch seem reasonable?

        Show
        Steve Davids added a comment - [~shalinda] Does the patch seem reasonable?
        Hide
        ASF subversion and git services added a comment -

        Commit 1563576 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1563576 ]

        SOLR-5535: Set partialResults header for shards that error out if shards.tolerant is specified

        Show
        ASF subversion and git services added a comment - Commit 1563576 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1563576 ] SOLR-5535 : Set partialResults header for shards that error out if shards.tolerant is specified
        Hide
        ASF subversion and git services added a comment -

        Commit 1563577 from shalin@apache.org in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1563577 ]

        SOLR-5535: Set partialResults header for shards that error out if shards.tolerant is specified

        Show
        ASF subversion and git services added a comment - Commit 1563577 from shalin@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1563577 ] SOLR-5535 : Set partialResults header for shards that error out if shards.tolerant is specified
        Hide
        Shalin Shekhar Mangar added a comment -

        This is fixed. Thanks Steve!

        Show
        Shalin Shekhar Mangar added a comment - This is fixed. Thanks Steve!
        Hide
        Shalin Shekhar Mangar added a comment -

        Re-opening due to jenkins failures related to this fix.

        Show
        Shalin Shekhar Mangar added a comment - Re-opening due to jenkins failures related to this fix.
        Hide
        Shalin Shekhar Mangar added a comment -

        Here's the stacktrace:

        java.lang.AssertionError: Expected the partialResults header to be null
                at __randomizedtesting.SeedInfo.seed([75E757505E2B6367:F401D9482974035B]:0)
                at org.junit.Assert.fail(Assert.java:93)
                at org.junit.Assert.assertTrue(Assert.java:43)
                at org.junit.Assert.assertNull(Assert.java:551)
                at org.apache.solr.TestDistributedSearch.validateControlData(TestDistributedSearch.java:525)
                at org.apache.solr.TestDistributedSearch.queryPartialResults(TestDistributedSearch.java:445)
                at org.apache.solr.TestDistributedSearch.doTest(TestDistributedSearch.java:379)
        

        The problem is that the test asserts that control responses should never have the partialResults header. Normally that will never happen because control is a single shard so a partialResults is not possible and instead the whole request will error out. But, if timeAllowed param is specified then the control shard can have a time exceeded exception causing the partialResults flag to be set.

        This patch checks if timeAllowed is specified and if so, disables the control data validation.

        Show
        Shalin Shekhar Mangar added a comment - Here's the stacktrace: java.lang.AssertionError: Expected the partialResults header to be null at __randomizedtesting.SeedInfo.seed([75E757505E2B6367:F401D9482974035B]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertNull(Assert.java:551) at org.apache.solr.TestDistributedSearch.validateControlData(TestDistributedSearch.java:525) at org.apache.solr.TestDistributedSearch.queryPartialResults(TestDistributedSearch.java:445) at org.apache.solr.TestDistributedSearch.doTest(TestDistributedSearch.java:379) The problem is that the test asserts that control responses should never have the partialResults header. Normally that will never happen because control is a single shard so a partialResults is not possible and instead the whole request will error out. But, if timeAllowed param is specified then the control shard can have a time exceeded exception causing the partialResults flag to be set. This patch checks if timeAllowed is specified and if so, disables the control data validation.
        Hide
        ASF subversion and git services added a comment -

        Commit 1569188 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1569188 ]

        SOLR-5535: partialResults on control responses is allowed if timeAllowed param is present

        Show
        ASF subversion and git services added a comment - Commit 1569188 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1569188 ] SOLR-5535 : partialResults on control responses is allowed if timeAllowed param is present
        Hide
        ASF subversion and git services added a comment -

        Commit 1569189 from shalin@apache.org in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1569189 ]

        SOLR-5535: partialResults on control responses is allowed if timeAllowed param is present

        Show
        ASF subversion and git services added a comment - Commit 1569189 from shalin@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1569189 ] SOLR-5535 : partialResults on control responses is allowed if timeAllowed param is present

          People

          • Assignee:
            Shalin Shekhar Mangar
            Reporter:
            Steve Davids
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development