Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-11880

Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search and update operations

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 7.4, 8.0
    • None
    • None

    Description

      MDCAwareThreadPoolExecutor has this line in it's{execute method

       

      final Exception submitterStackTrace = new Exception("Submitter stack trace");

      This means that every call via the a thread pool will create this exception, and only when it sees an error will it be used.

      Attachments

        1. SOLR-11880.patch
          7 kB
          Shalin Shekhar Mangar

        Activity

          If we remove that exception then we're running blind. There would be no way to get the stack trace of the submitter when an actual exception happens inside the executor thread.

          shalin Shalin Shekhar Mangar added a comment - If we remove that exception then we're running blind. There would be no way to get the stack trace of the submitter when an actual exception happens inside the executor thread.
          noble.paul Noble Paul added a comment -

          We should create a custom Exception which doesn't fillInStackTrace() in constructor. we can do the fillInStackTrace lazily

          noble.paul Noble Paul added a comment - We should create a custom Exception which doesn't fillInStackTrace() in constructor. we can do the fillInStackTrace lazily

          I don't think that is possible. If we want the caller's stack trace then we either call fillInStackTrace at the point of the call or we use a cached copy from the right place. If this is indeed a bottleneck (and I am not sure that it is) then we can either eliminate the submitter stack trace with a flag or do something ugly/clever like having the caller of execute method pass down a static exception object

          shalin Shalin Shekhar Mangar added a comment - I don't think that is possible. If we want the caller's stack trace then we either call fillInStackTrace at the point of the call or we use a cached copy from the right place. If this is indeed a bottleneck (and I am not sure that it is) then we can either eliminate the submitter stack trace with a flag or do something ugly/clever like having the caller of execute method pass down a static exception object

          noble.paul - I suspect the commits so far here were actually meant for SOLR-11067?

          ctargett Cassandra Targett added a comment - noble.paul - I suspect the commits so far here were actually meant for SOLR-11067 ?
          noble.paul Noble Paul added a comment -

          ctargett right. this is what happens when you work on multiple tickets at the same time

          noble.paul Noble Paul added a comment - ctargett right. this is what happens when you work on multiple tickets at the same time

          Taking another look at this one. The distributed update and search requests are the most frequent users of this executor. Tracing each, I see that the Runnable added by HttpShardHandler.submit, SolrCmdDistributor.submit and StreamingSolrClients.ErrorReportingConcurrentUpdateSolrClient never throw any exceptions but they are tracked and logged as part of the shard response or similar classes. Therefore there is no point in creating an Exception object for these two usages. So, this patches adds another constructor for MDCAwareThreadPoolExecutor which disables the submitter stack trace for search and update use-cases.

          shalin Shalin Shekhar Mangar added a comment - Taking another look at this one. The distributed update and search requests are the most frequent users of this executor. Tracing each, I see that the Runnable added by HttpShardHandler.submit, SolrCmdDistributor.submit and StreamingSolrClients.ErrorReportingConcurrentUpdateSolrClient never throw any exceptions but they are tracked and logged as part of the shard response or similar classes. Therefore there is no point in creating an Exception object for these two usages. So, this patches adds another constructor for MDCAwareThreadPoolExecutor which disables the submitter stack trace for search and update use-cases.

          Commit 5a47ed4209620f6ea80af00774f270a02604c966 in lucene-solr's branch refs/heads/master from shalin
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5a47ed4 ]

          SOLR-11880: Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search and update operations

          jira-bot ASF subversion and git services added a comment - - edited Commit 5a47ed4209620f6ea80af00774f270a02604c966 in lucene-solr's branch refs/heads/master from shalin [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5a47ed4 ] SOLR-11880 : Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search and update operations

          Commit c4a41c4c6a368f4461f6d31cfa6647348a3f7666 in lucene-solr's branch refs/heads/branch_7x from shalin
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c4a41c4 ]

          SOLR-11880: Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search and update operations

          (cherry picked from commit 5a47ed4)

          jira-bot ASF subversion and git services added a comment - - edited Commit c4a41c4c6a368f4461f6d31cfa6647348a3f7666 in lucene-solr's branch refs/heads/branch_7x from shalin [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c4a41c4 ] SOLR-11880 : Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search and update operations (cherry picked from commit 5a47ed4)

          Commit d52c1021e543b8cc2dd7b8d1d181e3dba160a760 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar
          [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d52c102 ]

          SOLR-14402: Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search.

          This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which fixed distributed updates but not distributed search.

          jira-bot ASF subversion and git services added a comment - Commit d52c1021e543b8cc2dd7b8d1d181e3dba160a760 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d52c102 ] SOLR-14402 : Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search. This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which fixed distributed updates but not distributed search.

          Commit f68c498e4fb891efbd598ea2007985334c731baf in lucene-solr's branch refs/heads/branch_8x from Shalin Shekhar Mangar
          [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f68c498 ]

          SOLR-14402: Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search.

          This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which fixed distributed updates but not distributed search.

          (cherry picked from commit d52c1021e543b8cc2dd7b8d1d181e3dba160a760)

          jira-bot ASF subversion and git services added a comment - Commit f68c498e4fb891efbd598ea2007985334c731baf in lucene-solr's branch refs/heads/branch_8x from Shalin Shekhar Mangar [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f68c498 ] SOLR-14402 : Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search. This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which fixed distributed updates but not distributed search. (cherry picked from commit d52c1021e543b8cc2dd7b8d1d181e3dba160a760)

          People

            shalin Shalin Shekhar Mangar
            varun Varun Thacker
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: