Uploaded image for project: 'CouchDB'
  1. CouchDB
  2. COUCHDB-3284

8Kb read-ahead in couch_file causes extra IO and binary memory usage

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: None
    • Component/s: Database Core
    • Labels:
      None

      Description

      8Kb read-ahead logic in couch_file seems to cause extra input IO thrashing, binary memory usage but doesn't speed-up

      1. jira_io_increased.png
        174 kB
        Nick Vatamaniuc

        Issue Links

          Activity

          Hide
          vatamane Nick Vatamaniuc added a comment -

          Middle column is current code. Right column is without read-ahead.

          Show
          vatamane Nick Vatamaniuc added a comment - Middle column is current code. Right column is without read-ahead.
          Hide
          vatamane Nick Vatamaniuc added a comment -

          Attached performance graphs showing increased IO usage and increased binary memory usage. Those go away after disabling 8kb read-ahead logic

          Show
          vatamane Nick Vatamaniuc added a comment - Attached performance graphs showing increased IO usage and increased binary memory usage. Those go away after disabling 8kb read-ahead logic
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user nickva opened a pull request:

          https://github.com/apache/couchdb-couch/pull/223

          Remove 8kB read-ahead from couch_file

          In production it showed an increased input Erlang IO and increased binary
          memory usage.

          See attached file in Jira ticket:

          COUCHDB-3284

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/cloudant/couchdb-couch couchdb-3284

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/couchdb-couch/pull/223.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #223


          commit d52a5335d930d11ade4953c8576d22f55872ff6f
          Author: Nick Vatamaniuc <vatamane@apache.org>
          Date: 2017-01-27T17:04:01Z

          Remove 8kB read-ahead from couch_file

          In production it showed an increased input Erlang IO and increased binary
          memory usage.

          See attached file in Jira ticket:

          COUCHDB-3284


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user nickva opened a pull request: https://github.com/apache/couchdb-couch/pull/223 Remove 8kB read-ahead from couch_file In production it showed an increased input Erlang IO and increased binary memory usage. See attached file in Jira ticket: COUCHDB-3284 You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloudant/couchdb-couch couchdb-3284 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/couchdb-couch/pull/223.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #223 commit d52a5335d930d11ade4953c8576d22f55872ff6f Author: Nick Vatamaniuc <vatamane@apache.org> Date: 2017-01-27T17:04:01Z Remove 8kB read-ahead from couch_file In production it showed an increased input Erlang IO and increased binary memory usage. See attached file in Jira ticket: COUCHDB-3284
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit d52a5335d930d11ade4953c8576d22f55872ff6f in couchdb-couch's branch refs/heads/master from Nick Vatamaniuc
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=d52a533 ]

          Remove 8kB read-ahead from couch_file

          In production it showed an increased input Erlang IO and increased binary
          memory usage.

          See attached file in Jira ticket:

          COUCHDB-3284

          Show
          jira-bot ASF subversion and git services added a comment - Commit d52a5335d930d11ade4953c8576d22f55872ff6f in couchdb-couch's branch refs/heads/master from Nick Vatamaniuc [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=d52a533 ] Remove 8kB read-ahead from couch_file In production it showed an increased input Erlang IO and increased binary memory usage. See attached file in Jira ticket: COUCHDB-3284
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/couchdb-couch/pull/223

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/couchdb-couch/pull/223
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 7c7a1d66e4e8800775723eaf24fdd81d73adffcd in couchdb's branch refs/heads/master from Nick Vatamaniuc
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=7c7a1d6 ]

          Bump couch dependency for 8kB read-ahead fix

          COUCHDB-3284

          Show
          jira-bot ASF subversion and git services added a comment - Commit 7c7a1d66e4e8800775723eaf24fdd81d73adffcd in couchdb's branch refs/heads/master from Nick Vatamaniuc [ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=7c7a1d6 ] Bump couch dependency for 8kB read-ahead fix COUCHDB-3284
          Hide
          vatamane Nick Vatamaniuc added a comment -

          Merged fix

          Show
          vatamane Nick Vatamaniuc added a comment - Merged fix
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user nickva opened a pull request:

          https://github.com/apache/couchdb-couch/pull/225

          Remove dead code from couch_file

          This code was left over after removing 8kB read-ahead

          https://github.com/apache/couchdb-couch/pull/223/commits/d52a5335d930d11ade4953c8576d22f55872ff6f

          COUCHDB-3284

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/cloudant/couchdb-couch couchdb-3284-remove-dead-code

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/couchdb-couch/pull/225.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #225


          commit bf12a7a3336bd438493e7db1d090160f1c1a09f3
          Author: Nick Vatamaniuc <vatamane@apache.org>
          Date: 2017-02-01T19:42:59Z

          Remove dead code from couch_file

          This code was left over after removing 8kB read-ahead

          https://github.com/apache/couchdb-couch/pull/223/commits/d52a5335d930d11ade4953c8576d22f55872ff6f

          COUCHDB-3284


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user nickva opened a pull request: https://github.com/apache/couchdb-couch/pull/225 Remove dead code from couch_file This code was left over after removing 8kB read-ahead https://github.com/apache/couchdb-couch/pull/223/commits/d52a5335d930d11ade4953c8576d22f55872ff6f COUCHDB-3284 You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloudant/couchdb-couch couchdb-3284-remove-dead-code Alternatively you can review and apply these changes as the patch at: https://github.com/apache/couchdb-couch/pull/225.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #225 commit bf12a7a3336bd438493e7db1d090160f1c1a09f3 Author: Nick Vatamaniuc <vatamane@apache.org> Date: 2017-02-01T19:42:59Z Remove dead code from couch_file This code was left over after removing 8kB read-ahead https://github.com/apache/couchdb-couch/pull/223/commits/d52a5335d930d11ade4953c8576d22f55872ff6f COUCHDB-3284
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit bf12a7a3336bd438493e7db1d090160f1c1a09f3 in couchdb-couch's branch refs/heads/master from Nick Vatamaniuc
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=bf12a7a ]

          Remove dead code from couch_file

          This code was left over after removing 8kB read-ahead

          https://github.com/apache/couchdb-couch/pull/223/commits/d52a5335d930d11ade4953c8576d22f55872ff6f

          COUCHDB-3284

          Show
          jira-bot ASF subversion and git services added a comment - Commit bf12a7a3336bd438493e7db1d090160f1c1a09f3 in couchdb-couch's branch refs/heads/master from Nick Vatamaniuc [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=bf12a7a ] Remove dead code from couch_file This code was left over after removing 8kB read-ahead https://github.com/apache/couchdb-couch/pull/223/commits/d52a5335d930d11ade4953c8576d22f55872ff6f COUCHDB-3284
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/couchdb-couch/pull/225

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/couchdb-couch/pull/225
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit c644280554aba00055886602fba39ad058306d7e in couchdb's branch refs/heads/master from Nick Vatamaniuc
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=c644280 ]

          Bump couch dependency. Remove dead code in couch_file.

          COUCHDB-3284

          Show
          jira-bot ASF subversion and git services added a comment - Commit c644280554aba00055886602fba39ad058306d7e in couchdb's branch refs/heads/master from Nick Vatamaniuc [ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=c644280 ] Bump couch dependency. Remove dead code in couch_file. COUCHDB-3284
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit bf12a7a3336bd438493e7db1d090160f1c1a09f3 in couchdb-couch's branch refs/heads/2971-count-distinct from Nick Vatamaniuc
          [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=bf12a7a ]

          Remove dead code from couch_file

          This code was left over after removing 8kB read-ahead

          https://github.com/apache/couchdb-couch/pull/223/commits/d52a5335d930d11ade4953c8576d22f55872ff6f

          COUCHDB-3284

          Show
          jira-bot ASF subversion and git services added a comment - Commit bf12a7a3336bd438493e7db1d090160f1c1a09f3 in couchdb-couch's branch refs/heads/2971-count-distinct from Nick Vatamaniuc [ https://git-wip-us.apache.org/repos/asf?p=couchdb-couch.git;h=bf12a7a ] Remove dead code from couch_file This code was left over after removing 8kB read-ahead https://github.com/apache/couchdb-couch/pull/223/commits/d52a5335d930d11ade4953c8576d22f55872ff6f COUCHDB-3284

            People

            • Assignee:
              Unassigned
              Reporter:
              vatamane Nick Vatamaniuc
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development