Details

    • Type: Bug Bug
    • Status: Closed
    • Resolution: Fixed
    • Affects Version/s: 2.1
    • Fix Version/s: 2.2.0
    • Component/s: SMTPServer
    • Labels:
      None
    • Environment:
      Operating System: Other
      Platform: Other

      Description

      When I'm working on James, what I usually do is put the following into the buffer...

      -------------------
      HELO localhost
      MAIL FROM: <sergek@localhost>
      RCPT TO: <sergek@localhost>
      DATA
      Subject: Testing

      This is a test
      .
      QUIT
      -------------------

      Then whenever I want to run tests, I telnet to port 25, and paste what's in the
      buffer.

      Unfortunately the new SMTPHandler code doesn't like this, during the DATA
      command it skips what's in the input stream and waits to process that later
      until DATA is complete. This is somewhat hard to explain, but here's what you
      see when you do this...

      ---------------------
      220 STACCATO SMTP Server (JAMES SMTP Server 2.1) ready Fri, 20 Dec 2002 11:49:22
      -0500 (EST)
      HELO localhost
      MAIL FROM: <sergek@localhost>
      RCPT TO: <sergek@localhost>
      DATA
      Subject: Testing

      This is a test
      .
      QUIT
      250 STACCATO Hello localhost (127.0.0.1 [127.0.0.1])
      250 Sender <sergek@localhost> OK
      250 Recipient <sergek@localhost> OK
      354 Ok Send data ending with <CRLF>.<CRLF>
      ---------------------
      It then hangs waiting for you to type the data for the message. So you type
      something again, and finish with "CRLF.CRLF". Once you do, you then get...
      ---------------------
      .
      250 Message received
      500 STACCATO Syntax error, command unrecognized: SUBJECT:
      500 STACCATO Syntax error, command unrecognized:
      500 STACCATO Syntax error, command unrecognized: THIS
      500 STACCATO Syntax error, command unrecognized: .
      221 STACCATO Service closing transmission channel
      ----------------------

      So basically what it looks like is you're creating that separate stream (or
      however it's handling the DATA command), and it's not reading the buffer of the
      underlying input stream and pushing that data into the forked input stream.

        Issue Links

          Activity

          Hide
          Stefano Bagnara added a comment -

          This has been fixed again in current cvs.

          Show
          Stefano Bagnara added a comment - This has been fixed again in current cvs.
          Hide
          Noel J. Bergman added a comment -

          I tested this with the current code. It was almost certainly fixed last year when the BufferedReader change happened.

          Show
          Noel J. Bergman added a comment - I tested this with the current code. It was almost certainly fixed last year when the BufferedReader change happened.
          Hide
          Noel J. Bergman added a comment -

          Can you still reproduce this problem with the current test build? We replaced
          the BufferedReader.

          Show
          Noel J. Bergman added a comment - Can you still reproduce this problem with the current test build? We replaced the BufferedReader.
          Hide
          Juha Torkkel added a comment -

          > Fixed in v2.1.1 and HEAD (v3).

          I am using latest James release, 2.1.3 and seems like this fix hasnt been included. Here is description of what happends with 2.1.3:
          ===============================================================================================
          [juha@myhost apps]$ uname -mrspv
          Linux 2.4.9-e.3 #1 Fri May 3 17:02:43 EDT 2002 i686 unknown
          [juha@myhost apps]$ java -version
          java version "1.4.2"
          Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2-b28)
          Java HotSpot(TM) Client VM (build 1.4.2-b28, mixed mode)
          [juha@myhost apps]$ cat /etc/redhat-release
          Red Hat Linux Advanced Server release 2.1AS (Pensacola)
          ===============================================================================================
          [juha@myhost apps]$ ../bin/run.sh
          Using PHOENIX_HOME: /home/juha/james
          Using PHOENIX_TMPDIR: /home/juha/james/temp
          Using JAVA_HOME: /home/juha/java
          Running Phoenix:
          Phoenix 4.0.1
          James 2.1.3
          Remote Manager Service started plain:4555
          POP3 Service started plain:6110
          SMTP Service started plain:6025
          NNTP Service started plain:6119
          Fetch POP Disabled
          ===============================================================================================
          [juha@myhost script]$ nc 127.0.0.1 6025 < body.smtp
          220 myhost SMTP Server (JAMES SMTP Server 2.1.3) ready Thu, 24 Jul 2003 13:38:01 +0300 (EEST)
          250 myhost Hello test@localhost (myhost [127.0.0.1])
          250 Sender <test@localhost> OK
          250 Recipient <juha@localhost> OK
          354 Ok Send data ending with <CRLF>.<CRLF>
          250 Message received
          500 myhost Syntax error, command unrecognized: FROM:
          500 myhost Syntax error, command unrecognized: TO:
          500 myhost Syntax error, command unrecognized: SUBJECT:
          500 myhost Syntax error, command unrecognized:
          500 myhost Syntax error, command unrecognized: HI,
          500 myhost Syntax error, command unrecognized:
          500 myhost Syntax error, command unrecognized: THIS
          500 myhost Syntax error, command unrecognized: –
          500 myhost Syntax error, command unrecognized: KALA
          ===============================================================================================
          Attached to this case you can find .tgz file that has the body.smtp file that is used for
          Netcat as input (note: smtp-server is running at port 6025 in the above) and rudimentary
          patch that positions the marker in the BufferedReader to the position where it should
          be after processing DATA. This is not propably the best way to fix this, but merely a suggestion.

          Show
          Juha Torkkel added a comment - > Fixed in v2.1.1 and HEAD (v3). I am using latest James release, 2.1.3 and seems like this fix hasnt been included. Here is description of what happends with 2.1.3: =============================================================================================== [juha@myhost apps] $ uname -mrspv Linux 2.4.9-e.3 #1 Fri May 3 17:02:43 EDT 2002 i686 unknown [juha@myhost apps] $ java -version java version "1.4.2" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2-b28) Java HotSpot(TM) Client VM (build 1.4.2-b28, mixed mode) [juha@myhost apps] $ cat /etc/redhat-release Red Hat Linux Advanced Server release 2.1AS (Pensacola) =============================================================================================== [juha@myhost apps] $ ../bin/run.sh Using PHOENIX_HOME: /home/juha/james Using PHOENIX_TMPDIR: /home/juha/james/temp Using JAVA_HOME: /home/juha/java Running Phoenix: Phoenix 4.0.1 James 2.1.3 Remote Manager Service started plain:4555 POP3 Service started plain:6110 SMTP Service started plain:6025 NNTP Service started plain:6119 Fetch POP Disabled =============================================================================================== [juha@myhost script] $ nc 127.0.0.1 6025 < body.smtp 220 myhost SMTP Server (JAMES SMTP Server 2.1.3) ready Thu, 24 Jul 2003 13:38:01 +0300 (EEST) 250 myhost Hello test@localhost (myhost [127.0.0.1] ) 250 Sender <test@localhost> OK 250 Recipient <juha@localhost> OK 354 Ok Send data ending with <CRLF>.<CRLF> 250 Message received 500 myhost Syntax error, command unrecognized: FROM: 500 myhost Syntax error, command unrecognized: TO: 500 myhost Syntax error, command unrecognized: SUBJECT: 500 myhost Syntax error, command unrecognized: 500 myhost Syntax error, command unrecognized: HI, 500 myhost Syntax error, command unrecognized: 500 myhost Syntax error, command unrecognized: THIS 500 myhost Syntax error, command unrecognized: – 500 myhost Syntax error, command unrecognized: KALA =============================================================================================== Attached to this case you can find .tgz file that has the body.smtp file that is used for Netcat as input (note: smtp-server is running at port 6025 in the above) and rudimentary patch that positions the marker in the BufferedReader to the position where it should be after processing DATA. This is not propably the best way to fix this, but merely a suggestion.
          Hide
          Juha Torkkel added a comment -

          Created an attachment (id=7486)
          Netcat SMTP input and example fix for James bug 15579

          Show
          Juha Torkkel added a comment - Created an attachment (id=7486) Netcat SMTP input and example fix for James bug 15579
          Hide
          Noel J. Bergman added a comment -

          Fixed in v2.1.1 and HEAD (v3).

          Show
          Noel J. Bergman added a comment - Fixed in v2.1.1 and HEAD (v3).
          Hide
          Serge Knystautas added a comment -

          Created an attachment (id=4249)
          Patch to accept messages that are aggressively pipelined

          Show
          Serge Knystautas added a comment - Created an attachment (id=4249) Patch to accept messages that are aggressively pipelined
          Hide
          Peter M. Goldstein added a comment -

          I feel very strongly that this shouldn't be accepted. What follows is my
          reply to Serge's technical points.

          > Spammers will send email this way, and I'd rather not have their
          > already wasteful emails leave me with a bunch of open connections.
          > I'd prefer to close the connection and be done with the transaction.
          > As is, my mail server will receive all the TCP data, it sits in some
          > odd buffer, sit with an open connection until it times out, and then
          > the data gets dumped and I don't even get a chance to log and do
          > whatever else to make sure I can catch more spam in the future.

          I'm not sure I buy the first assertion, especially when I've never seen a
          spammer send mail that way. Spammers have to pay for bandwidth just like
          everybody else, and in general it's in their best interest to confirm that a
          server is an open relay before wasting their time and bandwidth sending
          arbitrary mail to be relayed through the server.

          But let us grant that some spammers do send email this way. IMO the logical
          way to handle this situation is to require more strict compliance from the
          clients, not less. This means that when a command is read off the wire, it
          would be useful to confirm that no additional data is pending. If it is, a
          5xx is immediately returned informing the client that they've sent a malformed
          command. I've got no real objection to that, provided it doesn't break any of
          the standard clients.

          I'd also argue that there are a number of additional, command level parsing
          mechanisms that would be far more effective at dealing with this problem than
          the suggested alternative of accepting the message. For example, imposing
          line length limits. Or imposing limits on the number of malformed commands
          that are allowed before the connection is closed. Both of these would
          be "fail-fast" techniques that would save the James server bandwidth and
          minimize the number of open connections. They are also compatible with the
          RFC and don't encourage standard-violating clients.

          As far as reactive action, I'm not sure what you've got in mind. Let us
          imagine that, in fact, this data is all read in to the appropriate buffers and
          the commands are processed. It's going to be stuck on the spool and
          processed. Now you've seen it - it's in your spam repository or something.
          What're you going to do? I mean I guess you could globally block connections
          from the server that sent it, but if this is really a problem you've got the
          IP address of the server in either case. What does processing and storing the
          message get you?

          > Anyway, this also smells like a great DOS opportunity.

          Honestly, I don't see how this is any kind of DoS opportunity. At least not
          one above and beyond those provided by the protocol itself.

          Fundamentally, SMTP is not a particularly DoS resistant protocol. That's just
          how it is. If I want to kill an SMTP server, all I have to do is open up
          enough connections and send NOOPs, RSETs, or EHLOs over each of them, over and
          over and over again. Since endless loops of these commands are permitted by
          the SMTP protocol grammar, there's basically nothing to be done about that.

          The problem is even worse, since James doesn't have any of the data filtering
          mechanisms described above. So I can attach a base64 encoded version of my
          favorite MP3 and send it along as an argument to RSET. I'll get a 5xx, but it
          will tie up bandwidth and resources while James is processing.

          So James is vulnerable to DoS attacks, but that's basically because it's a
          mail server. We could harden it a bit (using some or all of the techniques I
          describe above) but I don't see how DoS is relevant to the issue under
          discussion.

          Show
          Peter M. Goldstein added a comment - I feel very strongly that this shouldn't be accepted. What follows is my reply to Serge's technical points. > Spammers will send email this way, and I'd rather not have their > already wasteful emails leave me with a bunch of open connections. > I'd prefer to close the connection and be done with the transaction. > As is, my mail server will receive all the TCP data, it sits in some > odd buffer, sit with an open connection until it times out, and then > the data gets dumped and I don't even get a chance to log and do > whatever else to make sure I can catch more spam in the future. I'm not sure I buy the first assertion, especially when I've never seen a spammer send mail that way. Spammers have to pay for bandwidth just like everybody else, and in general it's in their best interest to confirm that a server is an open relay before wasting their time and bandwidth sending arbitrary mail to be relayed through the server. But let us grant that some spammers do send email this way. IMO the logical way to handle this situation is to require more strict compliance from the clients, not less. This means that when a command is read off the wire, it would be useful to confirm that no additional data is pending. If it is, a 5xx is immediately returned informing the client that they've sent a malformed command. I've got no real objection to that, provided it doesn't break any of the standard clients. I'd also argue that there are a number of additional, command level parsing mechanisms that would be far more effective at dealing with this problem than the suggested alternative of accepting the message. For example, imposing line length limits. Or imposing limits on the number of malformed commands that are allowed before the connection is closed. Both of these would be "fail-fast" techniques that would save the James server bandwidth and minimize the number of open connections. They are also compatible with the RFC and don't encourage standard-violating clients. As far as reactive action, I'm not sure what you've got in mind. Let us imagine that, in fact, this data is all read in to the appropriate buffers and the commands are processed. It's going to be stuck on the spool and processed. Now you've seen it - it's in your spam repository or something. What're you going to do? I mean I guess you could globally block connections from the server that sent it, but if this is really a problem you've got the IP address of the server in either case. What does processing and storing the message get you? > Anyway, this also smells like a great DOS opportunity. Honestly, I don't see how this is any kind of DoS opportunity. At least not one above and beyond those provided by the protocol itself. Fundamentally, SMTP is not a particularly DoS resistant protocol. That's just how it is. If I want to kill an SMTP server, all I have to do is open up enough connections and send NOOPs, RSETs, or EHLOs over each of them, over and over and over again. Since endless loops of these commands are permitted by the SMTP protocol grammar, there's basically nothing to be done about that. The problem is even worse, since James doesn't have any of the data filtering mechanisms described above. So I can attach a base64 encoded version of my favorite MP3 and send it along as an argument to RSET. I'll get a 5xx, but it will tie up bandwidth and resources while James is processing. So James is vulnerable to DoS attacks, but that's basically because it's a mail server. We could harden it a bit (using some or all of the techniques I describe above) but I don't see how DoS is relevant to the issue under discussion.
          Hide
          Serge Knystautas added a comment -

          My reasons to address this:

          • This is a valid test case... James should either accept the message or give a
            rejection. It shouldn't just hang until the connection times out.
          • Data accumulates in a buffer in InputStreamReader (as well as
            BufferedInputStream and BufferedReader), and with the hanging until connection
            timeout, this opens the door to a relatively easy DOS.

          My reasons to accept the message:

          • It is easy to implement.
          • Spammers send email without worrying about request/response nature of SMTP.
            By accepting the message, we allow the admin to use mailets to analyze the
            messages and otherwise decide for themselves how they would like to handle these
            messages.
          • With Sendmail, Exchange, Qmail, and Domino all accepting the message with this
            test case, I think the impact the James project would have towards proliferating
            bad client behavior by accepting the message is trivial.
          • This is how James used to work before the recent performance and scaling
            refactoring.

          I'll attach the patch to fix this should be so agree.

          Show
          Serge Knystautas added a comment - My reasons to address this: This is a valid test case... James should either accept the message or give a rejection. It shouldn't just hang until the connection times out. Data accumulates in a buffer in InputStreamReader (as well as BufferedInputStream and BufferedReader), and with the hanging until connection timeout, this opens the door to a relatively easy DOS. My reasons to accept the message: It is easy to implement. Spammers send email without worrying about request/response nature of SMTP. By accepting the message, we allow the admin to use mailets to analyze the messages and otherwise decide for themselves how they would like to handle these messages. With Sendmail, Exchange, Qmail, and Domino all accepting the message with this test case, I think the impact the James project would have towards proliferating bad client behavior by accepting the message is trivial. This is how James used to work before the recent performance and scaling refactoring. I'll attach the patch to fix this should be so agree.
          Hide
          Serge Knystautas added a comment -

          I'd appreciate the courtesy of a discussion so the community can decide whether
          this is invalid.

          Show
          Serge Knystautas added a comment - I'd appreciate the courtesy of a discussion so the community can decide whether this is invalid.
          Hide
          Peter M. Goldstein added a comment -

          This is not a valid bug.

          SMTP is a request/response protocol. Feeding it a set of request messages a
          priori is a violation of the spec. In this specific case, the message data is
          being fed in before the 354 reply is issued. If, for example, the server were
          to issue a 5xx reply rather than a 354 for whatever reason (resources
          unavailable, etc.) this would manifestly put the client in violation (client
          MUST NOT send data if a 354 is not issued).

          Show
          Peter M. Goldstein added a comment - This is not a valid bug. SMTP is a request/response protocol. Feeding it a set of request messages a priori is a violation of the spec. In this specific case, the message data is being fed in before the 354 reply is issued. If, for example, the server were to issue a 5xx reply rather than a 354 for whatever reason (resources unavailable, etc.) this would manifestly put the client in violation (client MUST NOT send data if a 354 is not issued).

            People

            • Assignee:
              Unassigned
              Reporter:
              Serge Knystautas
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development