Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      This just as a reminder to consider http://andy.wordpress.com/2012/02/13/erlang-is-a-hoarder/ (incl. comments)

        Activity

        Hide
        Paul Joseph Davis added a comment -

        Was just about to say that we have seen a couple places with similar behavior and similar results in BigCouch. Specifically couch_os_process pids can have this effect. The article is missing the underlying reason but the HN comment is close enough. Simple fix is to call erlang:garbage_collect() when processes are returned. I think there might be a few other places we do similar things but nothing as significant as a couchjs processes.

        Also, its possible that this might not be much of an issue for CouchDB. BigCouch deployments can run in the hundreds to a thousand or so couchjs processes so it was a bit more of an issue with those pids stealing RAM when they went idle. If the total number is smaller and they're reused quite quickly then this issue shouldn't manifest, or at least not as drastically.

        Show
        Paul Joseph Davis added a comment - Was just about to say that we have seen a couple places with similar behavior and similar results in BigCouch. Specifically couch_os_process pids can have this effect. The article is missing the underlying reason but the HN comment is close enough. Simple fix is to call erlang:garbage_collect() when processes are returned. I think there might be a few other places we do similar things but nothing as significant as a couchjs processes. Also, its possible that this might not be much of an issue for CouchDB. BigCouch deployments can run in the hundreds to a thousand or so couchjs processes so it was a bit more of an issue with those pids stealing RAM when they went idle. If the total number is smaller and they're reused quite quickly then this issue shouldn't manifest, or at least not as drastically.
        Hide
        Robert Newson added a comment -

        A further note that Cloudant have some operational experience (and deployed remedies) for this issue. A quick look indicates at least some of these made it into bigcouch already.

        Show
        Robert Newson added a comment - A further note that Cloudant have some operational experience (and deployed remedies) for this issue. A quick look indicates at least some of these made it into bigcouch already.

          People

          • Assignee:
            Unassigned
            Reporter:
            Jan Lehnardt
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development