CouchDB
  1. CouchDB
  2. COUCHDB-1278

Add Apache httpd-style common log access logging.

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Logging
    • Labels:
      None

      Description

      This module introduces a new log file couch_access.log in the spirit of
      Apache httpd's access.log.

      The log format is specified here:

      http://en.wikipedia.org/wiki/Common_Log_Format

      The module defaults to the more commonly used `extended` format,
      but can be configured to use the standard log format as well.

      This is a new module rather than an extension to the couch_log
      module to allow easy addition. Ideally, this will be a plugin in the future.

      A note for chunked responses, the response body length reported is
      "chunked" rather than the number of bytes. The case is unspecified
      in the format description. Maybe the fix is keeping track of bytes
      sent in a chunked response, but I didn't want to make the first stab
      at this more complicated that it has to be. I'm happy to adjust.

      See the full diff here:

      https://github.com/janl/couchdb/commit/614091e49380b3afab3c34ff65e6d87623059b8b

      I'm not proposing this to be added to 1.1.x, but if anybody wants it, there's a patch here:

      https://github.com/janl/couchdb/commit/508af053257c074ec37c9bd658479cc21b8d20d4

      (The only difference is the specification of the couch_access_log module on startup, as that
      moved from couch_server_sup.erl to couch_primary_services.erl)

        Issue Links

          Activity

          Hide
          Alexander Shorin added a comment -

          This issue looks as special case (predefined format) for more common COUCHDB-1401.

          Show
          Alexander Shorin added a comment - This issue looks as special case (predefined format) for more common COUCHDB-1401 .
          Hide
          Adam Kocoloski added a comment -

          Hi Jan, I took a look at https://github.com/janl/couchdb/commit/614091. I'm not sure I understand how the error_logger process is involved. Your clients are writing to the disk_log directly, but you've got a couch_access_log gen_event hanging on the error_logger event manager. That event handler listens for couch_access events and logs them to stdout, but I don't see any couch_access events being generated. You're also making the error_logger process subscribe to disk_log notifications, and removing the couch_access_log handler if you get any (expecting that the event_sup thinger will restart it).

          Looking over Filipe's recent changes to couch_log I can see that most of my comments apply there as well. Perhaps he can shed some light on the design here.

          Show
          Adam Kocoloski added a comment - Hi Jan, I took a look at https://github.com/janl/couchdb/commit/614091 . I'm not sure I understand how the error_logger process is involved. Your clients are writing to the disk_log directly, but you've got a couch_access_log gen_event hanging on the error_logger event manager. That event handler listens for couch_access events and logs them to stdout, but I don't see any couch_access events being generated. You're also making the error_logger process subscribe to disk_log notifications, and removing the couch_access_log handler if you get any (expecting that the event_sup thinger will restart it). Looking over Filipe's recent changes to couch_log I can see that most of my comments apply there as well. Perhaps he can shed some light on the design here.
          Hide
          Paul Joseph Davis added a comment -

          Oh hrm. I guess I didn't realize we couldn't add a nonce directly. I thought there was a reserved field for something like that but it appears that Adam has lied sweet sweet lies to me.

          I haven't reviewed the patch yet so I'll withhold any verdict on waiting or not till after. I was just anxious to see the possibility of splitting access and error logs.

          Show
          Paul Joseph Davis added a comment - Oh hrm. I guess I didn't realize we couldn't add a nonce directly. I thought there was a reserved field for something like that but it appears that Adam has lied sweet sweet lies to me. I haven't reviewed the patch yet so I'll withhold any verdict on waiting or not till after. I was just anxious to see the possibility of splitting access and error logs.
          Hide
          Jan Lehnardt added a comment -

          I'm in no hurry to commit this, but if we like this as is, it could go into 1.2.x as it is rather isolated. Maybe we can add your additions later?

          What would the nonce do? Allow to match errors with requests? I don't think that that'd be compatible with the common/extended formats proposed here. Maybe the thing you are describing should happen as well to take some pressure off of couch.log and make it more useful, but that'd eventually be orthogonal to this proposal. Or maybe I'm missing something

          Show
          Jan Lehnardt added a comment - I'm in no hurry to commit this, but if we like this as is, it could go into 1.2.x as it is rather isolated. Maybe we can add your additions later? What would the nonce do? Allow to match errors with requests? I don't think that that'd be compatible with the common/extended formats proposed here. Maybe the thing you are describing should happen as well to take some pressure off of couch.log and make it more useful, but that'd eventually be orthogonal to this proposal. Or maybe I'm missing something
          Hide
          Paul Joseph Davis added a comment -

          I would be quite in favor of adding this with a bit of an extension after we branch 1.2. Specifically I'd like to see access logs in separate files from error logs with each having independent levels (well, access with an on/off, and verbosity for error logs). The only gotchya is that it means we need to look at adding nonce values to match up between the two files. The nonce patch isn't overly difficult if people are interested.

          Show
          Paul Joseph Davis added a comment - I would be quite in favor of adding this with a bit of an extension after we branch 1.2. Specifically I'd like to see access logs in separate files from error logs with each having independent levels (well, access with an on/off, and verbosity for error logs). The only gotchya is that it means we need to look at adding nonce values to match up between the two files. The nonce patch isn't overly difficult if people are interested.

            People

            • Assignee:
              Unassigned
              Reporter:
              Jan Lehnardt
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:

                Development