Uploaded image for project: 'CouchDB'
  1. CouchDB
  2. COUCHDB-3251

Remove hot loop usage of filename:rootname/1

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Database Core
    • Labels:
      None

      Description

      We added a call to filename:rootname/1 that removes the ".couch" extension when it exists. We've been doing some profiling of CouchDB 2.0 recently and found this to be a fairly expensive call. It and related calls are in the top few most expensive functions according to eprof (this is VM wide, so not just cherry picking couch_server where its actually even worse).

      lists:zip/2                                                             157491702  1.35   77463688  [      0.49]
      erlang:setelement/3                                                     139509262  1.48   85212600  [      0.61]
      erlang:term_to_binary/2                                                  14724676  1.52   87419458  [      5.94]
      erlang:phash/2                                                           30943420  1.54   88195214  [      2.85]
      erlang:send/3                                                            13487486  2.06  118261137  [      8.77]
      filename:rootname/4                                                     514574672  2.59  148907072  [      0.29]
      ets:lookup/2                                                             32852756  2.66  152952875  [      4.66]
      erts_internal:port_command/3                                             10448091  2.95  169649699  [     16.24]
      ioq_server:matching_request/4                                           906453003  3.19  183041235  [      0.20]
      ioq_server:split/4                                                      535820540  3.31  189913578  [      0.35]
      snappy:compress/1                                                         7950803  3.42  196220575  [     24.68]
      filename:do_flatten/2                                                   516517594  4.21  241562020  [      0.47]
      gen_server:try_handle_call/4                                              9529789  5.66  324927694  [     34.10]
      gen_server:loop/6                                                        16844687  7.41  425628355  [     25.27]
      

      There's an obvious easy way to optimize this by using binary matching so simple PR is incoming.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user davisp opened a pull request:

          https://github.com/apache/couchdb-couch/pull/217

          Remove use of filename:rootname/1

          It turns out that filename:rootname/1 is fairly expensive. Given that we
          call it millions of times when doing database name validation it adds up
          significantly. This is a fairly easy optimization to just use binary
          matching instead.

          COUCHDB-3251

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/cloudant/couchdb-couch 3251-remove-filename-rootname

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/couchdb-couch/pull/217.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #217


          commit e8a97f52b0cf0d0408d5a4ea625644684edaa6f5
          Author: Paul J. Davis <paul.joseph.davis@gmail.com>
          Date: 2016-12-06T19:37:45Z

          Remove use of filename:rootname/1

          It turns out that filename:rootname/1 is fairly expensive. Given that we
          call it millions of times when doing database name validation it adds up
          significantly. This is a fairly easy optimization to just use binary
          matching instead.

          COUCHDB-3251


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user davisp opened a pull request: https://github.com/apache/couchdb-couch/pull/217 Remove use of filename:rootname/1 It turns out that filename:rootname/1 is fairly expensive. Given that we call it millions of times when doing database name validation it adds up significantly. This is a fairly easy optimization to just use binary matching instead. COUCHDB-3251 You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloudant/couchdb-couch 3251-remove-filename-rootname Alternatively you can review and apply these changes as the patch at: https://github.com/apache/couchdb-couch/pull/217.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #217 commit e8a97f52b0cf0d0408d5a4ea625644684edaa6f5 Author: Paul J. Davis <paul.joseph.davis@gmail.com> Date: 2016-12-06T19:37:45Z Remove use of filename:rootname/1 It turns out that filename:rootname/1 is fairly expensive. Given that we call it millions of times when doing database name validation it adds up significantly. This is a fairly easy optimization to just use binary matching instead. COUCHDB-3251
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit e8a97f52b0cf0d0408d5a4ea625644684edaa6f5 in couchdb-couch's branch refs/heads/master from Paul Joseph Davis
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=e8a97f5 ]

          Remove use of filename:rootname/1

          It turns out that filename:rootname/1 is fairly expensive. Given that we
          call it millions of times when doing database name validation it adds up
          significantly. This is a fairly easy optimization to just use binary
          matching instead.

          COUCHDB-3251

          Show
          jira-bot ASF subversion and git services added a comment - Commit e8a97f52b0cf0d0408d5a4ea625644684edaa6f5 in couchdb-couch's branch refs/heads/master from Paul Joseph Davis [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=e8a97f5 ] Remove use of filename:rootname/1 It turns out that filename:rootname/1 is fairly expensive. Given that we call it millions of times when doing database name validation it adds up significantly. This is a fairly easy optimization to just use binary matching instead. COUCHDB-3251
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/couchdb-couch/pull/217

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/couchdb-couch/pull/217
          Hide
          paul.joseph.davis Paul Joseph Davis added a comment -

          Merged.

          Show
          paul.joseph.davis Paul Joseph Davis added a comment - Merged.

            People

            • Assignee:
              Unassigned
              Reporter:
              paul.joseph.davis Paul Joseph Davis
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development