ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-1089

zkServer.sh status does not work due to invalid option of nc

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.4, 3.4.0
    • Fix Version/s: 3.4.3, 3.3.5, 3.5.0
    • Component/s: scripts
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The nc command used by zkServer.sh does not have the "-q" option on some linux versions ( I have checked RedHat/Fedora and FreeBSD).

      1. ZOOKEEPER-1089.patch.txt
        8 kB
        Roman Shaposhnik
      2. ZOOKEEPER-1089.3.3.patch.txt
        7 kB
        Roman Shaposhnik

        Activity

        Hide
        Patrick Hunt added a comment -

        Hi Bill, it doesn't have -q on rh/f and freebsd, is that correct? (or those are the ones you checked? particular version(s)?).

        Not having -q will be problematic, that's there to ensure the server has time to respond before nc quits. I've seen issues on ubuntu w/o this (esp the new version of nc that's available on the most recent ubuntu versions)

        Is there another/better way to do this?

        Show
        Patrick Hunt added a comment - Hi Bill, it doesn't have -q on rh/f and freebsd, is that correct? (or those are the ones you checked? particular version(s)?). Not having -q will be problematic, that's there to ensure the server has time to respond before nc quits. I've seen issues on ubuntu w/o this (esp the new version of nc that's available on the most recent ubuntu versions) Is there another/better way to do this?
        Hide
        Bill Au added a comment -

        I have checked RH 5 and FreeBSD 8.2

        FreeBSD 8.2:
        nc: invalid option – q
        usage: nc [-46DdEhklnrStUuvz] [-e policy] [-I length] [-i interval] [-O length]
        [-P proxy_username] [-p source_port] [-s source_ip_address] [-T ToS]
        [-V rtable] [-w timeout] [-X proxy_protocol]
        [-x proxy_address[:port]] [hostname] [port]

        RH 5:
        nc: invalid option – q
        usage: nc [-46DdhklnrStUuvzC] [-i interval] [-p source_port]
        [-s source_ip_address] [-T ToS] [-w timeout] [-X proxy_version]
        [-x proxy_address[:port]] [hostname] [port[s]]

        I am still researching if there is another/better way to do this.

        Show
        Bill Au added a comment - I have checked RH 5 and FreeBSD 8.2 FreeBSD 8.2: nc: invalid option – q usage: nc [-46DdEhklnrStUuvz] [-e policy] [-I length] [-i interval] [-O length] [-P proxy_username] [-p source_port] [-s source_ip_address] [-T ToS] [-V rtable] [-w timeout] [-X proxy_protocol] [-x proxy_address [:port] ] [hostname] [port] RH 5: nc: invalid option – q usage: nc [-46DdhklnrStUuvzC] [-i interval] [-p source_port] [-s source_ip_address] [-T ToS] [-w timeout] [-X proxy_version] [-x proxy_address [:port] ] [hostname] [port [s] ] I am still researching if there is another/better way to do this.
        Hide
        Bill Au added a comment -

        I did some tests and it seems that the "-w" optiion on RH and FreeBSD is the equivalent of "-1" on Ubuntu. For in the worse case we will need to detect the Linux version and use the appropriate option.

        Show
        Bill Au added a comment - I did some tests and it seems that the "-w" optiion on RH and FreeBSD is the equivalent of "-1" on Ubuntu. For in the worse case we will need to detect the Linux version and use the appropriate option.
        Hide
        Bill Au added a comment -

        With bash and ksh, I can use /dev/tcp to make a connection to a port:

        exec 5<>/dev/tcp/127.0.0.1/2181;echo stat >&5;cat <&5

        returns the same output as:

        echo stat | nc 127.0.0.1 2181

        Show
        Bill Au added a comment - With bash and ksh, I can use /dev/tcp to make a connection to a port: exec 5<>/dev/tcp/127.0.0.1/2181;echo stat >&5;cat <&5 returns the same output as: echo stat | nc 127.0.0.1 2181
        Hide
        Patrick Hunt added a comment -

        Is there an idiom for detecting which options a command line supports? Coding linux versions into the script seems like a bad idea longer term, perhaps we can try to run using nc, and if that fails retry using /dev/tcp? (or perhaps first try to run something using "nc -q" that shouldn't fail as long as nc supports -q, and then either use it or /dev/tcp depending on the results?)

        Show
        Patrick Hunt added a comment - Is there an idiom for detecting which options a command line supports? Coding linux versions into the script seems like a bad idea longer term, perhaps we can try to run using nc, and if that fails retry using /dev/tcp? (or perhaps first try to run something using "nc -q" that shouldn't fail as long as nc supports -q, and then either use it or /dev/tcp depending on the results?)
        Hide
        Roman Shaposhnik added a comment -

        First of all I think that the confusion here is based on the fact that there are way too many versions of netcat in the wild (there's GNU netcat, OpenBSD netcat and fork of OpenBSD one just to name a few). In general, I'd try to avoid using nc for that reason – it is difficult to do it in a portable way.

        It would be really nice to come up with a POSIX compliant way of opening TCP socket from inside a script. Unfortunately, my browsing through the Opengroup spec didn't unearth much. As it stands, I'd say that we've got the following options:
        1. As Bill pointed out we can use bash, since the script requires bash already
        2. GNU awk also has support for opening a TCP connection
        3. ZK itself

        Personally, I'd vote for including a very simple wrapper class into a ZK jar that would do what is expected to query a server. That also would take care of the less than ideal hack that is there to get clientPort data from the config.

        Thoughts?

        Show
        Roman Shaposhnik added a comment - First of all I think that the confusion here is based on the fact that there are way too many versions of netcat in the wild (there's GNU netcat, OpenBSD netcat and fork of OpenBSD one just to name a few). In general, I'd try to avoid using nc for that reason – it is difficult to do it in a portable way. It would be really nice to come up with a POSIX compliant way of opening TCP socket from inside a script. Unfortunately, my browsing through the Opengroup spec didn't unearth much. As it stands, I'd say that we've got the following options: 1. As Bill pointed out we can use bash, since the script requires bash already 2. GNU awk also has support for opening a TCP connection 3. ZK itself Personally, I'd vote for including a very simple wrapper class into a ZK jar that would do what is expected to query a server. That also would take care of the less than ideal hack that is there to get clientPort data from the config. Thoughts?
        Hide
        Patrick Hunt added a comment -

        I'd hate to re-implement something so basic.

        Btw, when we do fix this we shouldn't be using "stat", rather we should use "srvr" (saves d/ling all the connections - srvr is a summary, stat contains details on all the connections)

        Show
        Patrick Hunt added a comment - I'd hate to re-implement something so basic. Btw, when we do fix this we shouldn't be using "stat", rather we should use "srvr" (saves d/ling all the connections - srvr is a summary, stat contains details on all the connections)
        Hide
        Roman Shaposhnik added a comment -

        After additional investigation it turns out that '-q' is a Ubuntu specific patch on top of netcat-openbsd: http://archive.ubuntu.com/ubuntu/pool/main/n/netcat-openbsd/netcat-openbsd_1.89-3ubuntu2.diff.gz

        It seems to be dealing with the situation where the channel is half-closed (by the client) and the server still performs
        a write (q corresponds to a qflag):

            else if (pfd[1].revents & POLLHUP) {
            shutdown_wr:
                /* if the user asked to exit on EOF, do it */
                if (qflag == 0) { 
                         shutdown(nfd, SHUT_WR);
                         close(wfd);
                         exit(0);
                }       
                /* if user asked to die after a while, arrange for it */
                if (qflag > 0) {
                         signal(SIGALRM, quit);
                         alarm(qflag);
                }       
                pfd[1].fd = -1;
                pfd[1].events = 0;
            }       
        

        That matches line #235 in http://svn.apache.org/viewvc/zookeeper/trunk/src/java/test/org/apache/zookeeper/test/ClientBase.java?view=markup

        If my analysis is correct, then the only way to really fix this problem is to provide a Java wrapper inside of ZK itself. To the best of my knowledge bash and awk are not capable of dealing with half closed TCP sockets.

        Show
        Roman Shaposhnik added a comment - After additional investigation it turns out that '-q' is a Ubuntu specific patch on top of netcat-openbsd: http://archive.ubuntu.com/ubuntu/pool/main/n/netcat-openbsd/netcat-openbsd_1.89-3ubuntu2.diff.gz It seems to be dealing with the situation where the channel is half-closed (by the client) and the server still performs a write (q corresponds to a qflag): else if (pfd[1].revents & POLLHUP) { shutdown_wr: /* if the user asked to exit on EOF, do it */ if (qflag == 0) { shutdown(nfd, SHUT_WR); close(wfd); exit(0); } /* if user asked to die after a while, arrange for it */ if (qflag > 0) { signal(SIGALRM, quit); alarm(qflag); } pfd[1].fd = -1; pfd[1].events = 0; } That matches line #235 in http://svn.apache.org/viewvc/zookeeper/trunk/src/java/test/org/apache/zookeeper/test/ClientBase.java?view=markup If my analysis is correct, then the only way to really fix this problem is to provide a Java wrapper inside of ZK itself. To the best of my knowledge bash and awk are not capable of dealing with half closed TCP sockets.
        Hide
        Patrick Hunt added a comment -

        That's really depressing, that in this day/age we need our own code to do something so basic.

        Sounds reasonable to me though, if you want to give it a go checkout
        http://svn.apache.org/viewvc/zookeeper/trunk/src/java/test/org/apache/zookeeper/test/ClientBase.java?view=markup
        and look at send4LetterWord, I think you can just refactor this into it's own main.

        You should probably put it into this package: http://svn.apache.org/viewvc/zookeeper/trunk/src/java/main/org/apache/zookeeper/client/
        (not in 3.3 branch but it's there subsequently). FourLetterWordClientMain.java or something like that.

        Show
        Patrick Hunt added a comment - That's really depressing, that in this day/age we need our own code to do something so basic. Sounds reasonable to me though, if you want to give it a go checkout http://svn.apache.org/viewvc/zookeeper/trunk/src/java/test/org/apache/zookeeper/test/ClientBase.java?view=markup and look at send4LetterWord, I think you can just refactor this into it's own main. You should probably put it into this package: http://svn.apache.org/viewvc/zookeeper/trunk/src/java/main/org/apache/zookeeper/client/ (not in 3.3 branch but it's there subsequently). FourLetterWordClientMain.java or something like that.
        Hide
        Roman Shaposhnik added a comment -

        Here's a patch against 3.4/trunk. This is pretty bare bones. We can get fancier by parsing local config files, screening off non-existing 4 letter cmds, etc.

        Let me know if any of that would be desirable.

        Show
        Roman Shaposhnik added a comment - Here's a patch against 3.4/trunk. This is pretty bare bones. We can get fancier by parsing local config files, screening off non-existing 4 letter cmds, etc. Let me know if any of that would be desirable.
        Hide
        Roman Shaposhnik added a comment -

        Here's the same patch against 3.3 branch

        Show
        Roman Shaposhnik added a comment - Here's the same patch against 3.3 branch
        Hide
        Patrick Hunt added a comment -

        Roman, this looks good to me. Thanks! A couple comments:

        1) change "stat" to "srvr" - it's shorter and we don't needs/want all the session information.

        2) have you been able to verify this on any platforms? which?

        I think this is ready to go otw ("submit" plz).

        Show
        Patrick Hunt added a comment - Roman, this looks good to me. Thanks! A couple comments: 1) change "stat" to "srvr" - it's shorter and we don't needs/want all the session information. 2) have you been able to verify this on any platforms? which? I think this is ready to go otw ("submit" plz).
        Hide
        Roman Shaposhnik added a comment -

        @Patrick, thanks for the feedback. I've updated the patches with srvr and also verified them on the following OSes: Debian 5, Debian 6, Ubuntu 10.04, Ubuntu 10.10, CentOS 5, RHEL 6, SLES11

        Show
        Roman Shaposhnik added a comment - @Patrick, thanks for the feedback. I've updated the patches with srvr and also verified them on the following OSes: Debian 5, Debian 6, Ubuntu 10.04, Ubuntu 10.10, CentOS 5, RHEL 6, SLES11
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12508705/ZOOKEEPER-1089.patch.txt
        against trunk revision 1222816.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 9 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/861//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12508705/ZOOKEEPER-1089.patch.txt against trunk revision 1222816. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/861//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12508704/ZOOKEEPER-1089.3.3.patch.txt
        against trunk revision 1222816.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 9 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/862//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12508704/ZOOKEEPER-1089.3.3.patch.txt against trunk revision 1222816. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/862//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12508709/ZOOKEEPER-1089.patch.txt
        against trunk revision 1222816.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 9 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/863//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/863//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/863//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12508709/ZOOKEEPER-1089.patch.txt against trunk revision 1222816. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/863//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/863//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/863//console This message is automatically generated.
        Hide
        Patrick Hunt added a comment -

        Thanks Roman!

        Show
        Patrick Hunt added a comment - Thanks Roman!
        Hide
        Hudson added a comment -

        Integrated in ZooKeeper-trunk #1410 (See https://builds.apache.org/job/ZooKeeper-trunk/1410/)
        ZOOKEEPER-1089. zkServer.sh status does not work due to invalid option of nc (Roman Shaposhnik via phunt)

        phunt : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1225108
        Files :

        • /zookeeper/trunk/CHANGES.txt
        • /zookeeper/trunk/bin/zkServer.sh
        • /zookeeper/trunk/src/java/main/org/apache/zookeeper/client/FourLetterWordMain.java
        • /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/ClientBase.java
        • /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/FourLetterWordsQuorumTest.java
        • /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/FourLetterWordsTest.java
        Show
        Hudson added a comment - Integrated in ZooKeeper-trunk #1410 (See https://builds.apache.org/job/ZooKeeper-trunk/1410/ ) ZOOKEEPER-1089 . zkServer.sh status does not work due to invalid option of nc (Roman Shaposhnik via phunt) phunt : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1225108 Files : /zookeeper/trunk/CHANGES.txt /zookeeper/trunk/bin/zkServer.sh /zookeeper/trunk/src/java/main/org/apache/zookeeper/client/FourLetterWordMain.java /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/ClientBase.java /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/FourLetterWordsQuorumTest.java /zookeeper/trunk/src/java/test/org/apache/zookeeper/test/FourLetterWordsTest.java

          People

          • Assignee:
            Roman Shaposhnik
            Reporter:
            Bill Au
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development