CouchDB
  1. CouchDB
  2. COUCHDB-412

CouchDB fails to start when log file is a pipe

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9
    • Fix Version/s: None
    • Component/s: Logging
    • Labels:
      None
    • Environment:

      CentOS x86_64

    • Skill Level:
      New Contributors Level (Easy)

      Description

      I have an 0.9 CouchDB. When the local.ini file's [log] file is actually a pipe (with a known to be running consumer) rather than a file, CouchDB fails to start.

      1. pipe_wrapper.sh
        0.1 kB
        Matthew Hooker
      2. COUCHDB_412_01.patch
        4 kB
        Matthew Hooker

        Activity

        Enda Farrell created issue -
        Hide
        Paul Joseph Davis added a comment -

        Are we sure this isn't a permissions error?

        Show
        Paul Joseph Davis added a comment - Are we sure this isn't a permissions error?
        Hide
        Enda Farrell added a comment -

        Yes - the pipes were created by root using:
        "mkfifo -m 777 5984.log"
        and result in:
        "prwxrwxrwx 1 root root 0 Jul 13 13:57 5984.log"

        A thought had occurred later - the ini file uses "[log] file = ..." and I tried "[log] pipe = ..." and CouchDB starts up and appears to run. I am starting tests now to see if this is actually the case.

        Show
        Enda Farrell added a comment - Yes - the pipes were created by root using: "mkfifo -m 777 5984.log" and result in: "prwxrwxrwx 1 root root 0 Jul 13 13:57 5984.log" A thought had occurred later - the ini file uses " [log] file = ..." and I tried " [log] pipe = ..." and CouchDB starts up and appears to run. I am starting tests now to see if this is actually the case.
        Hide
        Paul Joseph Davis added a comment -

        setting 'pipe = something' in the ini file will just result in an ignored value.

        Show
        Paul Joseph Davis added a comment - setting 'pipe = something' in the ini file will just result in an ignored value.
        Hide
        Enda Farrell added a comment - - edited

        Unfortunatly, the "[log] pipe = ..." directive does not seem to work.

        In my "defaults.ini" I have "/usr/local/couchdb/var/log/couch.log" as the default log. In my per-instance ini (I run many instances on a single machine) I have overriding definitions tghat puts CouchDB logs into specific directories. When I change the per-instance ini, the override does not work, CouchDB is NOT logging into the pipe, it is logging into the default log. It's not failing on startup - it fails silently instead of throwing an error.

        Show
        Enda Farrell added a comment - - edited Unfortunatly, the " [log] pipe = ..." directive does not seem to work. In my "defaults.ini" I have "/usr/local/couchdb/var/log/couch.log" as the default log. In my per-instance ini (I run many instances on a single machine) I have overriding definitions tghat puts CouchDB logs into specific directories. When I change the per-instance ini, the override does not work, CouchDB is NOT logging into the pipe, it is logging into the default log. It's not failing on startup - it fails silently instead of throwing an error.
        Hide
        Enda Farrell added a comment -

        ... as Paul Joseph Davis points out ...

        Show
        Enda Farrell added a comment - ... as Paul Joseph Davis points out ...
        Hide
        Adam Kocoloski added a comment -

        CouchDB executes the following code:

        Filename = couch_config:get("log", "file", "couchdb.log"),

        {ok, Fd}

        = file:open(Filename, [append]),

        I tried using a pipe in a plain erl shell:

        $ mkfifo -m 777 blah.log
        $ ls -l blah.log
        prwxrwxrwx 1 kocolosk staff 0 Aug 7 09:45 blah.log

        $ erl
        Erlang R13B (erts-5.7.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]

        Eshell V5.7.1 (abort with ^G)
        1> file:open("blah.log", [append]).

        {error,eisdir}

        Sure enough, in the Erlang manual for file:open there's a list of common error codes that includes

        eisdir:
        The named file is not a regular file. It may be a directory, a
        fifo, or a device.

        I'll do a little more digging to see how one is supposed to handle FIFOs in Erlang.

        Show
        Adam Kocoloski added a comment - CouchDB executes the following code: Filename = couch_config:get("log", "file", "couchdb.log"), {ok, Fd} = file:open(Filename , [append] ), I tried using a pipe in a plain erl shell: $ mkfifo -m 777 blah.log $ ls -l blah.log prwxrwxrwx 1 kocolosk staff 0 Aug 7 09:45 blah.log $ erl Erlang R13B (erts-5.7.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.1 (abort with ^G) 1> file:open( "blah.log", [append] ). {error,eisdir} Sure enough, in the Erlang manual for file:open there's a list of common error codes that includes eisdir: The named file is not a regular file. It may be a directory, a fifo, or a device. I'll do a little more digging to see how one is supposed to handle FIFOs in Erlang.
        Hide
        Adam Kocoloski added a comment -

        Hmm, weird stuff. It looks like open_port() used to be able to work with named pipes (~2001), but I think that's no longer the case. It's possible to redirect stdin and stdout to named pipes using run_erl

        run_erl -daemon /tmp/ /tmp "/usr/local/bin/couchdb"

        but I don't really care for that solution.

        Show
        Adam Kocoloski added a comment - Hmm, weird stuff. It looks like open_port() used to be able to work with named pipes (~2001), but I think that's no longer the case. It's possible to redirect stdin and stdout to named pipes using run_erl run_erl -daemon /tmp/ /tmp "/usr/local/bin/couchdb" but I don't really care for that solution.
        Hide
        Adam Kocoloski added a comment -

        Found a thread from 2008 that recommends writing wrapper scripts around named pipes for use with open_port():

        http://erlang.org/pipermail/erlang-questions/2008-August/037234.html

        Show
        Adam Kocoloski added a comment - Found a thread from 2008 that recommends writing wrapper scripts around named pipes for use with open_port(): http://erlang.org/pipermail/erlang-questions/2008-August/037234.html
        Hide
        Matthew Hooker added a comment -

        This is a "rough draft" of a potential fix for this issue.

        This patch makes it so all writes to the log file happen through a port to the attached wrapper script.

        If people think this is a good approach the problem, a few more things should be done:

        1. only use the pipe wrapper if the target file is a named pipe, otherwise use standard file descriptors.
        2. find win32 solution (if applicable)

        I think a risk with this approach is if the wrapper script is aborted for some reason. There's no supervisor process to restart it, so logs will stop working.

        Show
        Matthew Hooker added a comment - This is a "rough draft" of a potential fix for this issue. This patch makes it so all writes to the log file happen through a port to the attached wrapper script. If people think this is a good approach the problem, a few more things should be done: 1. only use the pipe wrapper if the target file is a named pipe, otherwise use standard file descriptors. 2. find win32 solution (if applicable) I think a risk with this approach is if the wrapper script is aborted for some reason. There's no supervisor process to restart it, so logs will stop working.
        Matthew Hooker made changes -
        Field Original Value New Value
        Attachment COUCHDB_412_01.patch [ 12430858 ]
        Attachment pipe_wrapper.sh [ 12430859 ]
        Hide
        Brian Candler added a comment -

        I think this should be made a general facility to log via an external program: i.e. don't hardcoded the path to pipe_wrapper.sh, or the fact that it requires a filename argument for the log destination.

        This would then be comparable to the corresponding feature in Apache httpd:

        CustomLog "|/path/to/program" ...

        It would let you do lots of cool things: e.g. log to syslog or log via HTTP to another couchdb server, without having to launch another process which tails a pipe.

        Show
        Brian Candler added a comment - I think this should be made a general facility to log via an external program: i.e. don't hardcoded the path to pipe_wrapper.sh, or the fact that it requires a filename argument for the log destination. This would then be comparable to the corresponding feature in Apache httpd: CustomLog "|/path/to/program" ... It would let you do lots of cool things: e.g. log to syslog or log via HTTP to another couchdb server, without having to launch another process which tails a pipe.
        Hide
        Matthew Hooker added a comment -

        @Brian I think that makes a lot of sense, but is also a little beyond the scope of this bug. Maybe we should close this out and open a new feature request?

        If closing this bug out is the way to go, what do people think of this proposed solution?

        Show
        Matthew Hooker added a comment - @Brian I think that makes a lot of sense, but is also a little beyond the scope of this bug. Maybe we should close this out and open a new feature request? If closing this bug out is the way to go, what do people think of this proposed solution?
        Hide
        Enda Farrell added a comment -

        I favour the approach that allows the 'CustomLog "|/path/to/program" ' - but as I've not tried to write such a handoff, I don't know if it's difficult or simply a matter of a config ini setting.

        I think it somewhat important that if there's no supervisor process to restart it, we at least have the ability to call (perhaps the _status?) an API to test whether or not it is working/running.

        Show
        Enda Farrell added a comment - I favour the approach that allows the 'CustomLog "|/path/to/program" ' - but as I've not tried to write such a handoff, I don't know if it's difficult or simply a matter of a config ini setting. I think it somewhat important that if there's no supervisor process to restart it, we at least have the ability to call (perhaps the _status?) an API to test whether or not it is working/running.
        Paul Joseph Davis made changes -
        Skill Level New Contributors Level (Easy)
        Alexander Shorin made changes -
        Component/s Logging [ 12320708 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Enda Farrell
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development