Commons Net
  1. Commons Net
  2. NET-373

NNTP Listgroups not working - broken server implementation

    Details

      Description

      In addition to NET-276 I tried to get all newsgroups from my server with the example script. Unfortunately I got with underneath code the following error:

      As can be imagined this is a pay server so it has more than 31 groups. Unfortunately I have no idea how to get those printed underneath, probably also a bug!

      Output
       
      201 reader.xsnews.nl (frontend-F04-05) Server Ready - support@xsnews.nl
      AUTHINFO USER *******
      381 PASS required
      AUTHINFO PASS *********
      281 Ok
      Authentication succeeded
      LIST ACTIVE alt.fan.*
      215 Newsgroups in form "group high low flags".
       000000000000000 000000000000000 y
      ! 000000000000023 000000000000001 y
      !!! 000000000000001 000000000000001 y
      !vaux23 000000000000001 000000000000001 y
      " 000000000000001 000000000000001 y
      "0.test 000000000000001 000000000000001 y
      ": 000000000000001 000000000000001 y
      "Phyllis 000000000000001 000000000000001 y
      "alt.binaries.boneless 000000000001053 000000000001002 y
      "alt.binaries.multimedia 000000000002422 000000000000004 y
      "alt.binaries.multimedia.erotica 000000000000001 000000000000001 y
      "alt.binaries.multimedia.erotica.lesbians 000000000000002 000000000000001 y
      "alt.binaries.multimedia.scifi; 000000000000001 000000000000001 y
      "alt.binaries.sounds.mp3.lounge 000000000000001 000000000000001 y
      "alt.binaries.sounds.mp3.reggae 000000000000001 000000000000001 y
      "alt.binaries.tv.canadian 000000000000002 000000000000001 y
      "alt.internet 000000000000001 000000000000001 y
      "alt.politics 000000000000002 000000000000001 y
      "alt.religion.christian.east 000000000000001 000000000000001 y
      "webstump+urcm-noack@chiark.greenend.org.uk" 000000000000001 000000000000001 y
      #alt.binaries.department.pron 000000022885781 000000000000001 y
      ( 000000000000007 000000000000005 y
      (analog-island@t-online.de 000000000000001 000000000000001 y
      (cross 000000000000001 000000000000001 y
      * 000000000000006 000000000000006 y
      *.binaries.* 000000000000001 000000000000001 y
      +a.uninarch 000000000000353 000000000000353 y
      - 000000000000002 000000000000002 y
      -- 000000000000005 000000000000001 y
      ------------------ 000000000000004 000000000000002 y
      -adams 000000000000001 000000000000001 y
      31
      LIST ACTIVE alt.fan.*
      00000000000000 000000000000000 y
      java.io.IOException: 00000000000000 000000000000000 y
      	at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroupListing(NNTPClient.java:1043)
      	at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroups(NNTPClient.java:1059)
      	at ListNewsgroups_1.main(ListNewsgroups_1.java:89)
      	at Testing_NNTP.main(Testing_NNTP.java:25){panel}
      

      Code:

      Bar.java
      // Some comments here
      /*
       * Licensed to the Apache Software Foundation (ASF) under one or more
       * contributor license agreements.  See the NOTICE file distributed with
       * this work for additional information regarding copyright ownership.
       * The ASF licenses this file to You under the Apache License, Version 2.0
       * (the "License"); you may not use this file except in compliance with
       * the License.  You may obtain a copy of the License at
       *
       *      http://www.apache.org/licenses/LICENSE-2.0
       *
       * Unless required by applicable law or agreed to in writing, software
       * distributed under the License is distributed on an "AS IS" BASIS,
       * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
       * See the License for the specific language governing permissions and
       * limitations under the License.
       */
      
      //package examples.nntp;
      
      import java.io.IOException;
      import java.io.PrintWriter;
      
      import org.apache.commons.net.PrintCommandListener;
      import org.apache.commons.net.nntp.NNTPClient;
      import org.apache.commons.net.nntp.NewsgroupInfo;
      
      import javax.net.ssl.SSLSocketFactory;
      
      /***
       * This is a trivial example using the NNTP package to approximate the
       * Unix newsgroups command.  It merely connects to the specified news
       * server and issues fetches the list of newsgroups stored by the server.
       * On servers that store a lot of newsgroups, this command can take a very
       * long time (listing upwards of 30,000 groups).
       * <p>
       ***/
      
      public final class ListNewsgroups_1
      {
      
          public final static void main(String[] args)
          {
              if (args.length < 1)
              {
                  System.err.println("Usage: newsgroups newsserver [pattern]");
                  return;
              }
      
              NNTPClient client = new NNTPClient();
              client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
              //String pattern = args.length >= 2 ? args[1] : "";
              //String pattern = "";
              String pattern = "alt.fan.*";
              try
              {
                  
              	if (args[4] == "SSL")
              	{
              		client.setSocketFactory(SSLSocketFactory.getDefault());
                  } 
              	else 
              	{
              		//factory = createSocket(host, port);
                  }
      
      			client.connect(args[0],Integer.parseInt(args[3]));
      
      			// AUTHINFO USER/AUTHINFO PASS
                  boolean success = client.authenticate(args[1], args[2]);
                  if (success) {
                      System.out.println("Authentication succeeded");
                  } else {
                      System.out.println("Authentication failed, error =" + client.getReplyString());
                  }
      
      
                  int j = 0;
                  try {
                      for(String s : client.iterateNewsgroupListing(pattern)) {
                          j++;
                          System.out.println(s);
                      }
                  } catch (IOException e1) {
                      e1.printStackTrace();
                  }
                  System.out.println(j);
      
                  j = 0;
                  for(NewsgroupInfo n : client.iterateNewsgroups(pattern)) {
                      j++;
                      System.out.println(n.getNewsgroup());
                  }
                  System.out.println(j);
              }
              catch (IOException e)
              {
                  e.printStackTrace();
              }
              finally
              {
                  try
                  {
                      if (client.isConnected())
                          client.disconnect();
                  }
                  catch (IOException e)
                  {
                      System.err.println("Error disconnecting from server.");
                      e.printStackTrace();
                      System.exit(1);
                  }
              }
          }
      }
      

      If I missed something let me know and I will try to supply you with necessary data

      1. listing.txt
        4.27 MB
        Bas
      2. Bar.java
        3 kB
        Sebb

        Activity

        Hide
        Sebb added a comment -

        OK, thanks.

        Should there be any further issues, please reopen - or start a new JIRA if the problem is significantly different.

        Show
        Sebb added a comment - OK, thanks. Should there be any further issues, please reopen - or start a new JIRA if the problem is significantly different.
        Hide
        Bas added a comment -

        Sebb, great, just tested it, and all examples are now working.
        Read about the 375. Probably this fixed my issue.

        Going to play with it more after the weekend, but for now I can confirm it is fixed.

        Show
        Bas added a comment - Sebb, great, just tested it, and all examples are now working. Read about the 375. Probably this fixed my issue. Going to play with it more after the weekend, but for now I can confirm it is fixed.
        Hide
        Sebb added a comment -

        I think the fix for NET-375 may well have fixed it so that the code works OK your broken server.

        Please try and let me know.

        Show
        Sebb added a comment - I think the fix for NET-375 may well have fixed it so that the code works OK your broken server. Please try and let me know.
        Hide
        Sebb added a comment -

        At present the code terminates upon reading CRLF<period><not-period> and ignores any data thereafter.

        I'm hoping to update the code so it only terminates when it sees CRLF<period>CRLF as per the RFC, but this may take a while.

        If you wish to continue testing NET code, you will need to use a different server for the time being.

        Meanwhile, you may wish to report the broken behaviour of the server to their support department.

        Show
        Sebb added a comment - At present the code terminates upon reading CRLF<period><not-period> and ignores any data thereafter. I'm hoping to update the code so it only terminates when it sees CRLF<period>CRLF as per the RFC, but this may take a while. If you wish to continue testing NET code, you will need to use a different server for the time being. Meanwhile, you may wish to report the broken behaviour of the server to their support department.
        Hide
        Bas added a comment -

        What would be the best to handle this from this point in time?

        Show
        Bas added a comment - What would be the best to handle this from this point in time?
        Hide
        Sebb added a comment -

        Changed to enhancement as the NNTP code is behaving OK.

        Show
        Sebb added a comment - Changed to enhancement as the NNTP code is behaving OK.
        Hide
        Sebb added a comment -

        The server is broken, as it does not double periods (full stops) at the start of the line
        See RFC977, 2.4.1. Text Responses.

        This is confusing the code, which thinks it has found end of text.
        When the next command is sent, it gets the next line of text, rather than the proper reply (which would presumably be sent after the final period), so reports an error.

        Having said that, it should be possible to allow for this bad behaviour, as the termination line does look uniquely identifiable.

        ==

        The situation is that the server is not RFC-compliant.
        Also that LIST ACTIVE does not seem to work.
        If it is not supported, it should treat it like "LIST invalid" and respond with "501 bad command usage" or something similar.

        The initial part of the listing looks rather odd - why does the server allow such strange names?
        And at the end, there are a couple of odd names, too.

        Show
        Sebb added a comment - The server is broken, as it does not double periods (full stops) at the start of the line See RFC977, 2.4.1. Text Responses. This is confusing the code, which thinks it has found end of text. When the next command is sent, it gets the next line of text, rather than the proper reply (which would presumably be sent after the final period), so reports an error. Having said that, it should be possible to allow for this bad behaviour, as the termination line does look uniquely identifiable. == The situation is that the server is not RFC-compliant. Also that LIST ACTIVE does not seem to work. If it is not supported, it should treat it like "LIST invalid" and respond with "501 bad command usage" or something similar. The initial part of the listing looks rather odd - why does the server allow such strange names? And at the end, there are a couple of odd names, too.
        Hide
        Bas added a comment - - edited

        Output of renewed Bar.java program.
        This gives e all the groups which I also receive when I telnet to the server

        Output:

        201 reader.xsnews.nl (frontend-F04-04) Server Ready - support@xsnews.nl
        AUTHINFO USER yyyyyy
        381 PASS required
        AUTHINFO PASS yyyyyyy
        281 Ok
        Authentication succeeded
        LIST ACTIVE alt.fan.*
        215 Newsgroups in form "group high low flags".
        OK
        

        Addition:

        It does not filter teh groups as given in the pattern, this gives all groups either way!

        Show
        Bas added a comment - - edited Output of renewed Bar.java program. This gives e all the groups which I also receive when I telnet to the server Output: 201 reader.xsnews.nl (frontend-F04-04) Server Ready - support@xsnews.nl AUTHINFO USER yyyyyy 381 PASS required AUTHINFO PASS yyyyyyy 281 Ok Authentication succeeded LIST ACTIVE alt.fan.* 215 Newsgroups in form "group high low flags". OK Addition: It does not filter teh groups as given in the pattern, this gives all groups either way!
        Hide
        Sebb added a comment -

        Amended version of your test program.

        Writes the response from the server to the file "listing.txt".

        It would be helpful if you could run this and attach the output file, which should not contain any private information - but please check first.

        Show
        Sebb added a comment - Amended version of your test program. Writes the response from the server to the file "listing.txt". It would be helpful if you could run this and attach the output file, which should not contain any private information - but please check first.
        Hide
        Bas added a comment -

        Thanks, and btw good work for working so closely on this problem.
        Thumbs up.

        Show
        Bas added a comment - Thanks, and btw good work for working so closely on this problem. Thumbs up.
        Hide
        Sebb added a comment -

        Looks like there is a problem interpreting the response from the server, and NET thinks the listing has finished.

        I'm working on an update to your test program which will dump out the complete response contents to a file.
        This can then be checked to see what's happening.

        Show
        Sebb added a comment - Looks like there is a problem interpreting the response from the server, and NET thinks the listing has finished. I'm working on an update to your test program which will dump out the complete response contents to a file. This can then be checked to see what's happening.
        Hide
        Bas added a comment -

        With the standard code I still get:

         
        LIST ACTIVE 
        00000000000000 000000000000000 y
        java.io.IOException: LIST ACTIVE  command failed: 00000000000000 000000000000000 y
        	at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroupListing(NNTPClient.java:1045)
        	at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroups(NNTPClient.java:1061)
        	at ListNewsgroups_1.main(ListNewsgroups_1.java:91)
        	at Testing_NNTP.main(Testing_NNTP.java:25)
        

        With the change you suggest:

        I needed to change to code to the correct type:

        Bar.java
          for(NewsgroupInfo s : client.listNewsgroups(pattern)) 
                    {
                    	   System.out.println(s);
                    }
        

        With this I get output without an error:

        Authentication succeeded
        LIST ACTIVE 
        215 Newsgroups in form "group high low flags".
        org.apache.commons.net.nntp.NewsgroupInfo@1194a4e
        org.apache.commons.net.nntp.NewsgroupInfo@15d56d5
        org.apache.commons.net.nntp.NewsgroupInfo@efd552
        org.apache.commons.net.nntp.NewsgroupInfo@19dfbff
        org.apache.commons.net.nntp.NewsgroupInfo@10b4b2f
        org.apache.commons.net.nntp.NewsgroupInfo@750159
        org.apache.commons.net.nntp.NewsgroupInfo@1abab88
        org.apache.commons.net.nntp.NewsgroupInfo@18a7efd
        org.apache.commons.net.nntp.NewsgroupInfo@1971afc
        org.apache.commons.net.nntp.NewsgroupInfo@16cd7d5
        org.apache.commons.net.nntp.NewsgroupInfo@cdedfd
        org.apache.commons.net.nntp.NewsgroupInfo@1c39a2d
        org.apache.commons.net.nntp.NewsgroupInfo@bf2d5e
        org.apache.commons.net.nntp.NewsgroupInfo@13bad12
        org.apache.commons.net.nntp.NewsgroupInfo@df8ff1
        org.apache.commons.net.nntp.NewsgroupInfo@1632c2d
        org.apache.commons.net.nntp.NewsgroupInfo@1e97676
        org.apache.commons.net.nntp.NewsgroupInfo@60420f
        org.apache.commons.net.nntp.NewsgroupInfo@19106c7
        org.apache.commons.net.nntp.NewsgroupInfo@540408
        org.apache.commons.net.nntp.NewsgroupInfo@1d4c61c
        org.apache.commons.net.nntp.NewsgroupInfo@1a626f
        org.apache.commons.net.nntp.NewsgroupInfo@34a1fc
        org.apache.commons.net.nntp.NewsgroupInfo@176c74b
        org.apache.commons.net.nntp.NewsgroupInfo@116471f
        org.apache.commons.net.nntp.NewsgroupInfo@1975b59
        org.apache.commons.net.nntp.NewsgroupInfo@1ee3914
        org.apache.commons.net.nntp.NewsgroupInfo@e5855a
        org.apache.commons.net.nntp.NewsgroupInfo@95fd19
        org.apache.commons.net.nntp.NewsgroupInfo@11b9fb1
        org.apache.commons.net.nntp.NewsgroupInfo@913fe2
        

        However this is still only 31 items and not all newsgroups, which might be 3000+.

        Furthermore, when I give an other command after this one, I still get the specific error:

        Authentication succeeded
        LIST ACTIVE 
        215 Newsgroups in form "group high low flags".
        org.apache.commons.net.nntp.NewsgroupInfo@13caecd
        org.apache.commons.net.nntp.NewsgroupInfo@f84386
        org.apache.commons.net.nntp.NewsgroupInfo@1194a4e
        org.apache.commons.net.nntp.NewsgroupInfo@15d56d5
        org.apache.commons.net.nntp.NewsgroupInfo@efd552
        org.apache.commons.net.nntp.NewsgroupInfo@19dfbff
        org.apache.commons.net.nntp.NewsgroupInfo@10b4b2f
        org.apache.commons.net.nntp.NewsgroupInfo@750159
        org.apache.commons.net.nntp.NewsgroupInfo@1abab88
        org.apache.commons.net.nntp.NewsgroupInfo@18a7efd
        org.apache.commons.net.nntp.NewsgroupInfo@1971afc
        org.apache.commons.net.nntp.NewsgroupInfo@16cd7d5
        org.apache.commons.net.nntp.NewsgroupInfo@cdedfd
        org.apache.commons.net.nntp.NewsgroupInfo@1c39a2d
        org.apache.commons.net.nntp.NewsgroupInfo@bf2d5e
        org.apache.commons.net.nntp.NewsgroupInfo@13bad12
        org.apache.commons.net.nntp.NewsgroupInfo@df8ff1
        org.apache.commons.net.nntp.NewsgroupInfo@1632c2d
        org.apache.commons.net.nntp.NewsgroupInfo@1e97676
        org.apache.commons.net.nntp.NewsgroupInfo@60420f
        org.apache.commons.net.nntp.NewsgroupInfo@19106c7
        org.apache.commons.net.nntp.NewsgroupInfo@540408
        org.apache.commons.net.nntp.NewsgroupInfo@1d4c61c
        org.apache.commons.net.nntp.NewsgroupInfo@1a626f
        org.apache.commons.net.nntp.NewsgroupInfo@34a1fc
        org.apache.commons.net.nntp.NewsgroupInfo@176c74b
        org.apache.commons.net.nntp.NewsgroupInfo@116471f
        org.apache.commons.net.nntp.NewsgroupInfo@1975b59
        org.apache.commons.net.nntp.NewsgroupInfo@1ee3914
        org.apache.commons.net.nntp.NewsgroupInfo@e5855a
        org.apache.commons.net.nntp.NewsgroupInfo@95fd19
        LIST ACTIVE 
        00000000000000 000000000000000 y
        0
        java.io.IOException: LIST ACTIVE  command failed: 00000000000000 000000000000000 y
        	at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroupListing(NNTPClient.java:1045)
        	at ListNewsgroups_1.main(ListNewsgroups_1.java:84)
        	at Testing_NNTP.main(Testing_NNTP.java:25)
        

        Full code used for this:

        Bar.java
        /*
         * Licensed to the Apache Software Foundation (ASF) under one or more
         * contributor license agreements.  See the NOTICE file distributed with
         * this work for additional information regarding copyright ownership.
         * The ASF licenses this file to You under the Apache License, Version 2.0
         * (the "License"); you may not use this file except in compliance with
         * the License.  You may obtain a copy of the License at
         *
         *      http://www.apache.org/licenses/LICENSE-2.0
         *
         * Unless required by applicable law or agreed to in writing, software
         * distributed under the License is distributed on an "AS IS" BASIS,
         * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
         * See the License for the specific language governing permissions and
         * limitations under the License.
         */
        
        //package examples.nntp;
        
        import java.io.IOException;
        import java.io.PrintWriter;
        
        import org.apache.commons.net.PrintCommandListener;
        import org.apache.commons.net.nntp.NNTPClient;
        import org.apache.commons.net.nntp.NewsgroupInfo;
        
        import javax.net.ssl.SSLSocketFactory;
        
        /***
         * This is a trivial example using the NNTP package to approximate the
         * Unix newsgroups command.  It merely connects to the specified news
         * server and issues fetches the list of newsgroups stored by the server.
         * On servers that store a lot of newsgroups, this command can take a very
         * long time (listing upwards of 30,000 groups).
         * <p>
         ***/
        
        public final class ListNewsgroups_1
        {
        
            public final static void main(String[] args)
            {
                if (args.length < 1)
                {
                    System.err.println("Usage: newsgroups newsserver [pattern]");
                    return;
                }
        
                NNTPClient client = new NNTPClient();
                client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
                //String pattern = args.length >= 2 ? args[1] : "";
                String pattern = "";
                //String pattern = "alt.fan.*";
                try
                {
                    
                	if (args[4] == "SSL")
                	{
                		client.setSocketFactory(SSLSocketFactory.getDefault());
                    } 
                	else 
                	{
                		//factory = createSocket(host, port);
                    }
        
        			client.connect(args[0],Integer.parseInt(args[3]));
        
        			// AUTHINFO USER/AUTHINFO PASS
                    boolean success = client.authenticate(args[1], args[2]);
                    if (success) {
                        System.out.println("Authentication succeeded");
                    } else {
                        System.out.println("Authentication failed, error =" + client.getReplyString());
                    }
                    //client.iterateNewsgroupListing();
                    
                    for(NewsgroupInfo s : client.listNewsgroups(pattern)) 
                    {
                    	   System.out.println(s);
                    }
                    
                    int j = 0;
                    try {
                        for(String s : client.iterateNewsgroupListing(pattern)) {
                            j++;
                            System.out.println(s);
                        }
                    } catch (IOException e1) {
                        e1.printStackTrace();
                    }
                    System.out.println(j);
        /**
                    j = 0;
                    for(NewsgroupInfo n : client.iterateNewsgroups(pattern)) 
                    {
                        j++;
                        System.out.println(n.getNewsgroup());
                    }
                    System.out.println(j);
                    */
                }
                catch (IOException e)
                {
                    e.printStackTrace();
                }
                finally
                {
                    try
                    {
                        if (client.isConnected())
                            client.disconnect();
                    }
                    catch (IOException e)
                    {
                        System.err.println("Error disconnecting from server.");
                        e.printStackTrace();
                        System.exit(1);
                    }
                }
        
            }
        
        }
        
        Show
        Bas added a comment - With the standard code I still get: LIST ACTIVE 00000000000000 000000000000000 y java.io.IOException: LIST ACTIVE command failed: 00000000000000 000000000000000 y at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroupListing(NNTPClient.java:1045) at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroups(NNTPClient.java:1061) at ListNewsgroups_1.main(ListNewsgroups_1.java:91) at Testing_NNTP.main(Testing_NNTP.java:25) With the change you suggest: I needed to change to code to the correct type: Bar.java for (NewsgroupInfo s : client.listNewsgroups(pattern)) { System .out.println(s); } With this I get output without an error: Authentication succeeded LIST ACTIVE 215 Newsgroups in form "group high low flags". org.apache.commons.net.nntp.NewsgroupInfo@1194a4e org.apache.commons.net.nntp.NewsgroupInfo@15d56d5 org.apache.commons.net.nntp.NewsgroupInfo@efd552 org.apache.commons.net.nntp.NewsgroupInfo@19dfbff org.apache.commons.net.nntp.NewsgroupInfo@10b4b2f org.apache.commons.net.nntp.NewsgroupInfo@750159 org.apache.commons.net.nntp.NewsgroupInfo@1abab88 org.apache.commons.net.nntp.NewsgroupInfo@18a7efd org.apache.commons.net.nntp.NewsgroupInfo@1971afc org.apache.commons.net.nntp.NewsgroupInfo@16cd7d5 org.apache.commons.net.nntp.NewsgroupInfo@cdedfd org.apache.commons.net.nntp.NewsgroupInfo@1c39a2d org.apache.commons.net.nntp.NewsgroupInfo@bf2d5e org.apache.commons.net.nntp.NewsgroupInfo@13bad12 org.apache.commons.net.nntp.NewsgroupInfo@df8ff1 org.apache.commons.net.nntp.NewsgroupInfo@1632c2d org.apache.commons.net.nntp.NewsgroupInfo@1e97676 org.apache.commons.net.nntp.NewsgroupInfo@60420f org.apache.commons.net.nntp.NewsgroupInfo@19106c7 org.apache.commons.net.nntp.NewsgroupInfo@540408 org.apache.commons.net.nntp.NewsgroupInfo@1d4c61c org.apache.commons.net.nntp.NewsgroupInfo@1a626f org.apache.commons.net.nntp.NewsgroupInfo@34a1fc org.apache.commons.net.nntp.NewsgroupInfo@176c74b org.apache.commons.net.nntp.NewsgroupInfo@116471f org.apache.commons.net.nntp.NewsgroupInfo@1975b59 org.apache.commons.net.nntp.NewsgroupInfo@1ee3914 org.apache.commons.net.nntp.NewsgroupInfo@e5855a org.apache.commons.net.nntp.NewsgroupInfo@95fd19 org.apache.commons.net.nntp.NewsgroupInfo@11b9fb1 org.apache.commons.net.nntp.NewsgroupInfo@913fe2 However this is still only 31 items and not all newsgroups, which might be 3000+. Furthermore, when I give an other command after this one, I still get the specific error: Authentication succeeded LIST ACTIVE 215 Newsgroups in form "group high low flags". org.apache.commons.net.nntp.NewsgroupInfo@13caecd org.apache.commons.net.nntp.NewsgroupInfo@f84386 org.apache.commons.net.nntp.NewsgroupInfo@1194a4e org.apache.commons.net.nntp.NewsgroupInfo@15d56d5 org.apache.commons.net.nntp.NewsgroupInfo@efd552 org.apache.commons.net.nntp.NewsgroupInfo@19dfbff org.apache.commons.net.nntp.NewsgroupInfo@10b4b2f org.apache.commons.net.nntp.NewsgroupInfo@750159 org.apache.commons.net.nntp.NewsgroupInfo@1abab88 org.apache.commons.net.nntp.NewsgroupInfo@18a7efd org.apache.commons.net.nntp.NewsgroupInfo@1971afc org.apache.commons.net.nntp.NewsgroupInfo@16cd7d5 org.apache.commons.net.nntp.NewsgroupInfo@cdedfd org.apache.commons.net.nntp.NewsgroupInfo@1c39a2d org.apache.commons.net.nntp.NewsgroupInfo@bf2d5e org.apache.commons.net.nntp.NewsgroupInfo@13bad12 org.apache.commons.net.nntp.NewsgroupInfo@df8ff1 org.apache.commons.net.nntp.NewsgroupInfo@1632c2d org.apache.commons.net.nntp.NewsgroupInfo@1e97676 org.apache.commons.net.nntp.NewsgroupInfo@60420f org.apache.commons.net.nntp.NewsgroupInfo@19106c7 org.apache.commons.net.nntp.NewsgroupInfo@540408 org.apache.commons.net.nntp.NewsgroupInfo@1d4c61c org.apache.commons.net.nntp.NewsgroupInfo@1a626f org.apache.commons.net.nntp.NewsgroupInfo@34a1fc org.apache.commons.net.nntp.NewsgroupInfo@176c74b org.apache.commons.net.nntp.NewsgroupInfo@116471f org.apache.commons.net.nntp.NewsgroupInfo@1975b59 org.apache.commons.net.nntp.NewsgroupInfo@1ee3914 org.apache.commons.net.nntp.NewsgroupInfo@e5855a org.apache.commons.net.nntp.NewsgroupInfo@95fd19 LIST ACTIVE 00000000000000 000000000000000 y 0 java.io.IOException: LIST ACTIVE command failed: 00000000000000 000000000000000 y at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroupListing(NNTPClient.java:1045) at ListNewsgroups_1.main(ListNewsgroups_1.java:84) at Testing_NNTP.main(Testing_NNTP.java:25) Full code used for this: Bar.java /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License" ); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http: //www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ // package examples.nntp; import java.io.IOException; import java.io.PrintWriter; import org.apache.commons.net.PrintCommandListener; import org.apache.commons.net.nntp.NNTPClient; import org.apache.commons.net.nntp.NewsgroupInfo; import javax.net.ssl.SSLSocketFactory; /*** * This is a trivial example using the NNTP package to approximate the * Unix newsgroups command. It merely connects to the specified news * server and issues fetches the list of newsgroups stored by the server. * On servers that store a lot of newsgroups, this command can take a very * long time (listing upwards of 30,000 groups). * <p> ***/ public final class ListNewsgroups_1 { public final static void main( String [] args) { if (args.length < 1) { System .err.println( "Usage: newsgroups newsserver [pattern]" ); return ; } NNTPClient client = new NNTPClient(); client.addProtocolCommandListener( new PrintCommandListener( new PrintWriter( System .out))); // String pattern = args.length >= 2 ? args[1] : ""; String pattern = ""; // String pattern = "alt.fan.*" ; try { if (args[4] == "SSL" ) { client.setSocketFactory(SSLSocketFactory.getDefault()); } else { //factory = createSocket(host, port); } client.connect(args[0], Integer .parseInt(args[3])); // AUTHINFO USER/AUTHINFO PASS boolean success = client.authenticate(args[1], args[2]); if (success) { System .out.println( "Authentication succeeded" ); } else { System .out.println( "Authentication failed, error =" + client.getReplyString()); } //client.iterateNewsgroupListing(); for (NewsgroupInfo s : client.listNewsgroups(pattern)) { System .out.println(s); } int j = 0; try { for ( String s : client.iterateNewsgroupListing(pattern)) { j++; System .out.println(s); } } catch (IOException e1) { e1.printStackTrace(); } System .out.println(j); /** j = 0; for (NewsgroupInfo n : client.iterateNewsgroups(pattern)) { j++; System .out.println(n.getNewsgroup()); } System .out.println(j); */ } catch (IOException e) { e.printStackTrace(); } finally { try { if (client.isConnected()) client.disconnect(); } catch (IOException e) { System .err.println( "Error disconnecting from server." ); e.printStackTrace(); System .exit(1); } } } }
        Hide
        Bas added a comment -

        This morning at around 7 o'clock the update was not in the SVN, the above was created using latest SVN at that time.

        When I am home I will

        svn update
        

        After that:

        mvn package

        And then I will do the test as asked:

        for(String s : client.listNewsgroups(pattern)) {
           System.out.println(s);
        }
        
        Show
        Bas added a comment - This morning at around 7 o'clock the update was not in the SVN, the above was created using latest SVN at that time. When I am home I will svn update After that: mvn package And then I will do the test as asked: for(String s : client.listNewsgroups(pattern)) { System.out.println(s); }
        Hide
        Sebb added a comment -

        Yes, to get the updated exception messages you will need to refresh SVN and rebuild.

        Please also try using the other method, something like:

        for(String s : client.listNewsgroups(pattern)) {
           System.out.println(s);
        }
        

        Also, if you can edit your previous comment, please can you enclose the output in {noformat} tags as it's very difficult to read at present.

        Show
        Sebb added a comment - Yes, to get the updated exception messages you will need to refresh SVN and rebuild. Please also try using the other method, something like: for ( String s : client.listNewsgroups(pattern)) { System .out.println(s); } Also, if you can edit your previous comment, please can you enclose the output in {noformat} tags as it's very difficult to read at present.
        Hide
        Bas added a comment - - edited

        so I do need to update my svn rep?
        I will give you a different output when using no pattern at all!

        As you can see I have disabled the first newsgroup listening. In that case the second is running proberly. However still only 31 groups are listed. My server has more than 1000, that I know for sure...

        So what I see is that when running the 2 independand commands the server gives me the IO

        Code:

        Bar.java
        /*
         * Licensed to the Apache Software Foundation (ASF) under one or more
         * contributor license agreements.  See the NOTICE file distributed with
         * this work for additional information regarding copyright ownership.
         * The ASF licenses this file to You under the Apache License, Version 2.0
         * (the "License"); you may not use this file except in compliance with
         * the License.  You may obtain a copy of the License at
         *
         *      http://www.apache.org/licenses/LICENSE-2.0
         *
         * Unless required by applicable law or agreed to in writing, software
         * distributed under the License is distributed on an "AS IS" BASIS,
         * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
         * See the License for the specific language governing permissions and
         * limitations under the License.
         */
        
        //package examples.nntp;
        
        import java.io.IOException;
        import java.io.PrintWriter;
        
        import org.apache.commons.net.PrintCommandListener;
        import org.apache.commons.net.nntp.NNTPClient;
        import org.apache.commons.net.nntp.NewsgroupInfo;
        
        import javax.net.ssl.SSLSocketFactory;
        
        /***
         * This is a trivial example using the NNTP package to approximate the
         * Unix newsgroups command.  It merely connects to the specified news
         * server and issues fetches the list of newsgroups stored by the server.
         * On servers that store a lot of newsgroups, this command can take a very
         * long time (listing upwards of 30,000 groups).
         * <p>
         ***/
        
        public final class ListNewsgroups_1
        {
        
            public final static void main(String[] args)
            {
                if (args.length < 1)
                {
                    System.err.println("Usage: newsgroups newsserver [pattern]");
                    return;
                }
        
                NNTPClient client = new NNTPClient();
                client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out)));
                //String pattern = args.length >= 2 ? args[1] : "";
                String pattern = "";
                //String pattern = "alt.fan.*";
                try
                {
                    
                	if (args[4] == "SSL")
                	{
                		client.setSocketFactory(SSLSocketFactory.getDefault());
                    } 
                	else 
                	{
                		//factory = createSocket(host, port);
                    }
        
        			client.connect(args[0],Integer.parseInt(args[3]));
        
        			// AUTHINFO USER/AUTHINFO PASS
                    boolean success = client.authenticate(args[1], args[2]);
                    if (success) {
                        System.out.println("Authentication succeeded");
                    } else {
                        System.out.println("Authentication failed, error =" + client.getReplyString());
                    }
                    //client.iterateNewsgroupListing();
                           
                    int j = 0;
                    for(NewsgroupInfo n : client.iterateNewsgroups(pattern)) {
                        j++;
                        System.out.println(n.getNewsgroup());
                    }
                    System.out.println(j);
                }
                catch (IOException e)
                {
                    e.printStackTrace();
                }
                finally
                {
                    try
                    {
                        if (client.isConnected())
                            client.disconnect();
                    }
                    catch (IOException e)
                    {
                        System.err.println("Error disconnecting from server.");
                        e.printStackTrace();
                        System.exit(1);
                    }
                }
        
            }
        
        }
        

        Output (1 function):

        201 reader.xsnews.nl (frontend-F05-04) Server Ready - support@xsnews.nl
        AUTHINFO USER yyyyyyyy
        381 PASS required
        AUTHINFO PASS yyyyyyyy
        281 Ok
        Authentication succeeded
        LIST ACTIVE 
        215 Newsgroups in form "group high low flags".
        
        !
        !!!
        !vaux23
        "
        "0.test
        ":
        "Phyllis
        "alt.binaries.boneless
        "alt.binaries.multimedia
        "alt.binaries.multimedia.erotica
        "alt.binaries.multimedia.erotica.lesbians
        "alt.binaries.multimedia.scifi;
        "alt.binaries.sounds.mp3.lounge
        "alt.binaries.sounds.mp3.reggae
        "alt.binaries.tv.canadian
        "alt.internet
        "alt.politics
        "alt.religion.christian.east
        "webstump+urcm-noack@chiark.greenend.org.uk"
        #alt.binaries.department.pron
        (
        (analog-island@t-online.de
        (cross
        *
        *.binaries.*
        +a.uninarch
        -
        --
        ------------------
        -adams
        31
        

        Output both functions:

        201 reader.xsnews.nl (frontend-E06-02) Server Ready - support@xsnews.nl
        AUTHINFO USER yyyyyyy
        381 PASS required
        AUTHINFO PASS yyyyyyyyy
        281 Ok
        Authentication succeeded
        LIST ACTIVE 
        215 Newsgroups in form "group high low flags".
         000000000000000 000000000000000 y
        ! 000000000000023 000000000000001 y
        !!! 000000000000001 000000000000001 y
        !vaux23 000000000000001 000000000000001 y
        " 000000000000001 000000000000001 y
        "0.test 000000000000001 000000000000001 y
        ": 000000000000001 000000000000001 y
        "Phyllis 000000000000001 000000000000001 y
        "alt.binaries.boneless 000000000001053 000000000001002 y
        "alt.binaries.multimedia 000000000000808 000000000000004 y
        "alt.binaries.multimedia.erotica 000000000000001 000000000000001 y
        "alt.binaries.multimedia.erotica.lesbians 000000000000002 000000000000001 y
        "alt.binaries.multimedia.scifi; 000000000000001 000000000000001 y
        "alt.binaries.sounds.mp3.lounge 000000000000001 000000000000001 y
        "alt.binaries.sounds.mp3.reggae 000000000000001 000000000000001 y
        "alt.binaries.tv.canadian 000000000000002 000000000000001 y
        "alt.internet 000000000000001 000000000000001 y
        "alt.politics 000000000000002 000000000000001 y
        "alt.religion.christian.east 000000000000001 000000000000001 y
        "webstump+urcm-noack@chiark.greenend.org.uk" 000000000000001 000000000000001 y
        #alt.binaries.department.pron 000000022953499 000000000000001 y
        ( 000000000000007 000000000000005 y
        (analog-island@t-online.de 000000000000001 000000000000001 y
        (cross 000000000000001 000000000000001 y
        * 000000000000006 000000000000006 y
        *.binaries.* 000000000000001 000000000000001 y
        +a.uninarch 000000000000353 000000000000353 y
        - 000000000000002 000000000000002 y
        -- 000000000000005 000000000000001 y
        ------------------ 000000000000004 000000000000002 y
        -adams 000000000000001 000000000000001 y
        31
        LIST ACTIVE 
        00000000000000 000000000000000 y
        java.io.IOException: 00000000000000 000000000000000 y
        	at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroupListing(NNTPClient.java:1043)
        	at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroups(NNTPClient.java:1059)
        	at ListNewsgroups_1.main(ListNewsgroups_1.java:91)
        	at Testing_NNTP.main(Testing_NNTP.java:25)
        
        Show
        Bas added a comment - - edited so I do need to update my svn rep? I will give you a different output when using no pattern at all! As you can see I have disabled the first newsgroup listening. In that case the second is running proberly. However still only 31 groups are listed. My server has more than 1000, that I know for sure... So what I see is that when running the 2 independand commands the server gives me the IO Code: Bar.java /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License" ); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http: //www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ // package examples.nntp; import java.io.IOException; import java.io.PrintWriter; import org.apache.commons.net.PrintCommandListener; import org.apache.commons.net.nntp.NNTPClient; import org.apache.commons.net.nntp.NewsgroupInfo; import javax.net.ssl.SSLSocketFactory; /*** * This is a trivial example using the NNTP package to approximate the * Unix newsgroups command. It merely connects to the specified news * server and issues fetches the list of newsgroups stored by the server. * On servers that store a lot of newsgroups, this command can take a very * long time (listing upwards of 30,000 groups). * <p> ***/ public final class ListNewsgroups_1 { public final static void main( String [] args) { if (args.length < 1) { System .err.println( "Usage: newsgroups newsserver [pattern]" ); return ; } NNTPClient client = new NNTPClient(); client.addProtocolCommandListener( new PrintCommandListener( new PrintWriter( System .out))); // String pattern = args.length >= 2 ? args[1] : ""; String pattern = ""; // String pattern = "alt.fan.*" ; try { if (args[4] == "SSL" ) { client.setSocketFactory(SSLSocketFactory.getDefault()); } else { //factory = createSocket(host, port); } client.connect(args[0], Integer .parseInt(args[3])); // AUTHINFO USER/AUTHINFO PASS boolean success = client.authenticate(args[1], args[2]); if (success) { System .out.println( "Authentication succeeded" ); } else { System .out.println( "Authentication failed, error =" + client.getReplyString()); } //client.iterateNewsgroupListing(); int j = 0; for (NewsgroupInfo n : client.iterateNewsgroups(pattern)) { j++; System .out.println(n.getNewsgroup()); } System .out.println(j); } catch (IOException e) { e.printStackTrace(); } finally { try { if (client.isConnected()) client.disconnect(); } catch (IOException e) { System .err.println( "Error disconnecting from server." ); e.printStackTrace(); System .exit(1); } } } } Output (1 function): 201 reader.xsnews.nl (frontend-F05-04) Server Ready - support@xsnews.nl AUTHINFO USER yyyyyyyy 381 PASS required AUTHINFO PASS yyyyyyyy 281 Ok Authentication succeeded LIST ACTIVE 215 Newsgroups in form "group high low flags". ! !!! !vaux23 " "0.test ": "Phyllis "alt.binaries.boneless "alt.binaries.multimedia "alt.binaries.multimedia.erotica "alt.binaries.multimedia.erotica.lesbians "alt.binaries.multimedia.scifi; "alt.binaries.sounds.mp3.lounge "alt.binaries.sounds.mp3.reggae "alt.binaries.tv.canadian "alt.internet "alt.politics "alt.religion.christian.east "webstump+urcm-noack@chiark.greenend.org.uk" #alt.binaries.department.pron ( (analog-island@t-online.de (cross * *.binaries.* +a.uninarch - -- ------------------ -adams 31 Output both functions: 201 reader.xsnews.nl (frontend-E06-02) Server Ready - support@xsnews.nl AUTHINFO USER yyyyyyy 381 PASS required AUTHINFO PASS yyyyyyyyy 281 Ok Authentication succeeded LIST ACTIVE 215 Newsgroups in form "group high low flags". 000000000000000 000000000000000 y ! 000000000000023 000000000000001 y !!! 000000000000001 000000000000001 y !vaux23 000000000000001 000000000000001 y " 000000000000001 000000000000001 y "0.test 000000000000001 000000000000001 y ": 000000000000001 000000000000001 y "Phyllis 000000000000001 000000000000001 y "alt.binaries.boneless 000000000001053 000000000001002 y "alt.binaries.multimedia 000000000000808 000000000000004 y "alt.binaries.multimedia.erotica 000000000000001 000000000000001 y "alt.binaries.multimedia.erotica.lesbians 000000000000002 000000000000001 y "alt.binaries.multimedia.scifi; 000000000000001 000000000000001 y "alt.binaries.sounds.mp3.lounge 000000000000001 000000000000001 y "alt.binaries.sounds.mp3.reggae 000000000000001 000000000000001 y "alt.binaries.tv.canadian 000000000000002 000000000000001 y "alt.internet 000000000000001 000000000000001 y "alt.politics 000000000000002 000000000000001 y "alt.religion.christian.east 000000000000001 000000000000001 y "webstump+urcm-noack@chiark.greenend.org.uk" 000000000000001 000000000000001 y #alt.binaries.department.pron 000000022953499 000000000000001 y ( 000000000000007 000000000000005 y (analog-island@t-online.de 000000000000001 000000000000001 y (cross 000000000000001 000000000000001 y * 000000000000006 000000000000006 y *.binaries.* 000000000000001 000000000000001 y +a.uninarch 000000000000353 000000000000353 y - 000000000000002 000000000000002 y -- 000000000000005 000000000000001 y ------------------ 000000000000004 000000000000002 y -adams 000000000000001 000000000000001 y 31 LIST ACTIVE 00000000000000 000000000000000 y java.io.IOException: 00000000000000 000000000000000 y at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroupListing(NNTPClient.java:1043) at org.apache.commons.net.nntp.NNTPClient.iterateNewsgroups(NNTPClient.java:1059) at ListNewsgroups_1.main(ListNewsgroups_1.java:91) at Testing_NNTP.main(Testing_NNTP.java:25)
        Hide
        Sebb added a comment -

        Also, try the listNewsgroups(pattern) method as well, as that uses a different method of handling the replies.

        The group names returned by the server look rather odd, as they don't start with alt.fan. Maybe the server is misbehaving?

        The program works fine on my news server (which has over 3000 groups under alt.fan).

        [I've updated the IOException messages]

        Show
        Sebb added a comment - Also, try the listNewsgroups(pattern) method as well, as that uses a different method of handling the replies. The group names returned by the server look rather odd, as they don't start with alt.fan. Maybe the server is misbehaving? The program works fine on my news server (which has over 3000 groups under alt.fan). [I've updated the IOException messages]
        Hide
        Sebb added a comment -

        The following sequence:

        LIST ACTIVE alt.fan.*
        00000000000000 000000000000000 y
        java.io.IOException: 00000000000000 000000000000000 y
        

        shows that the code thinks the server has responded with an invalid reply to the LIST ACTIVE command.

        It should return something like:

        LIST ACTIVE alt.fan.*
        215 Newsgroups in form "group high low flags".
         000000000000000 000000000000000 y
        

        For some reason the 215 response seems to be missing.

        The IOException message is rather cryptic, and needs to be improved.

        Does this happen every time?

        Show
        Sebb added a comment - The following sequence: LIST ACTIVE alt.fan.* 00000000000000 000000000000000 y java.io.IOException: 00000000000000 000000000000000 y shows that the code thinks the server has responded with an invalid reply to the LIST ACTIVE command. It should return something like: LIST ACTIVE alt.fan.* 215 Newsgroups in form "group high low flags" . 000000000000000 000000000000000 y For some reason the 215 response seems to be missing. The IOException message is rather cryptic, and needs to be improved. Does this happen every time?
        Hide
        Sebb added a comment -

        Correct formatting of output

        Show
        Sebb added a comment - Correct formatting of output

          People

          • Assignee:
            Unassigned
            Reporter:
            Bas
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development