Uploaded image for project: 'CouchDB'
  1. CouchDB
  2. COUCHDB-2

Socket errors under high load

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Doing some basic load testing with ab, I found that on my machine (amd64,
      ubuntu feisty, couch svn r186), couch starts encountering socket errors at
      concurrency levels above 600. Example:

      ab -n 1000 -c 600 http://localhost:8888/ab_test/hello.txt

      Yields 1000 successful requests, but if I bump -c up to 700, requests start
      to fail with errors like:

      [info] [<0.21516.0>] HTTP Error (code 500): {obj,[{error,{obj,
      [

      {id,"EXIT"}

      , {reason,
      "{{badmatch,socketclosed},[

      {modcouch,sendjson,4}

      ,

      {modcouch,do,1}

      ,

      {httpdresponse,traversemodules,2}

      ,

      {httpdresponse,generateandsendresponse,1}

      ,

      {httpdrequesthandler,handleresponse,1}

      ,

      {genserver,handlemsg,6}

      ,

      {proclib,init_p,5}

      ]}"}]}}]}

      Is there any other testing I can do or information I can provide to help
      track down the problem?

      (Hit enter before entering my email, sorry: jpellerin at gmail d com)

      --Jan:

      You're likely to hit an OS limit there.

      – JP:

      I don't think so, since I haven't seen this with yaws under equivalent
      load. Yaws can handle up to my system's max open files without failing, and
      when it hits that, ab itself fails, not yaws. In this case yaws is running
      a simple appmod that loads a file from a mnesia table:

      jhp@jpdesk:~/work/cloud/nimbus$ ab -n 10000 -c 1000
      http://localhost:8002/content/hello.txt This is ApacheBench, Version
      2.0.40-dev <$Revision: 1.146 $> apache-2.0 Copyright 1996 Adam Twiss, Zeus
      Technology Ltd, http://www.zeustech.net/ Copyright 2006 The Apache Software
      Foundation, http://www.apache.org/

      Benchmarking localhost (be patient) [...] Finished 10000 requests

      Server Software: Yaws/1.68 Server Hostname: localhost Server Port: 8002

      Document Path: /content/hello.txt Document Length: 6 bytes

      Concurrency Level: 1000 Time taken for tests: 7.244038 seconds Complete
      requests: 10000 Failed requests: 0 Write errors: 0

      (etc), with no errors in yaws itself. So I believe this is a bug in couchDB
      or couch_inets, not a configuration issue.

      --Jan

      Thanks for the detailed analysis! We're on it.

      Comment 1 by damienkatz, Sep 26, 2007

      Can someone check and see how the performance of the standard Erlang inets http server holds up under the
      same stress test? That's what CouchDB is using, and it would be very helpful to see if that's the limiting factor.

      In which case we might need to switch to YAWS or another erlang http implementation.

      Comment 2 by damienkatz, Sep 28, 2007

      This bug needs test code before it can be verified.

      Either the original test code, or new test code written in Erlang would be good too.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              nslater Naomi Slater
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: