Bug 5941

> Received: by localhost (fdm 1.5, account "gmx");
>         Mon, 14 Jul 2008 01:04:12 +0200
> Received: (qmail 6881 invoked by alias); 13 Jul 2008 22:29:06 -0000
> Delivered-To: GMX delivery to anyaddress@gmx.net
> Received: (qmail invoked by alias); 13 Jul 2008 22:29:06 -0000
> Received: from 198.pool85-55-41.dynamic.orange.es (EHLO [192.168.0.25]) 
> [85.55.41.198]
>   by mail.gmx.net (mp004) with SMTP; 14 Jul 2008 00:29:06 +0200

The problem here is that there's no noticeable transfer from mail.gmx.net to localhost; there's no Received line containing an IP address etc.  So it results in a single hop, like so:

[26366] dbg: metadata: X-Spam-Relays-External: [ ip=85.55.41.198 rdns=198.pool85-55-41.dynamic.orange.es helo=!192.168.0.25! by=mail.gmx.net ident= envfrom= intl=0 id= auth= msa=0 ]

for the dynamic-IP-to-GMX hop.

I think the best fix for you is to simply trust the GMX mailhost, and the dynamic IP range that you are in; add them to internal_networks.  That way, mail from your machine to your machine via GMX will always wind up trusted, which is what you're after.

Alternatively deliver the mail locally instead of hopping via GMX (simply install an MTA like postfix).  Or alternatively still, fix fdm to add a decent Received line ;)

(In reply to comment #1)
[...]

> Alternatively deliver the mail locally instead of hopping via GMX (simply
> install an MTA like postfix).  Or alternatively still, fix fdm to add a decent
> Received line ;)
> 

From looking at

http://tools.ietf.org/html/rfc2821#section-4.4

I would say that fdm - to be rfc-conform - should add the "FROM" field to the "Received:" line - and then this problem should be solved - is that correct?

If so I would forward this info to the fdm author.


thanks,


Tom

(In reply to comment #2)
> (In reply to comment #1)
> [...]
> 
> > Alternatively deliver the mail locally instead of hopping via GMX (simply
> > install an MTA like postfix).  Or alternatively still, fix fdm to add a decent
> > Received line ;)
> > 
> 
> From looking at
> 
> http://tools.ietf.org/html/rfc2821#section-4.4
> 
> I would say that fdm - to be rfc-conform - should add the "FROM" field to the
> "Received:" line - and then this problem should be solved - is that correct?

If it looked more like sendmail or Postfix' format, it would be best. ;)

Hi,

> The problem here is that there's no noticeable transfer from mail.gmx.net to
> localhost; there's no Received line containing an IP address etc.  So it
> results in a single hop, like so:
> 
> [26366] dbg: metadata: X-Spam-Relays-External: [ ip=85.55.41.198
> rdns=198.pool85-55-41.dynamic.orange.es helo=!192.168.0.25! by=mail.gmx.net
> ident= envfrom= intl=0 id= auth= msa=0 ]
> 
> for the dynamic-IP-to-GMX hop.

I doubt that this is really the problem. I did 2 checks:

1) I checked another mail which has also been send from a dynamic IP through a proper SMTP server (in this case web.de). This time SA doesn't think that the original SMTP server is the dynamic IP address it looks like (when looking at X-Spam-Status).
fdm's "Received:" line is in both mails is exactly the same except for the timestamp.

2) I tested what happens if I remove the following lines (the ones added by fdm) 

---------------------------------------------------------
Received: by localhost (fdm 1.5, account "gmx");
  Sat, 12 Jul 2008 ...
---------------------------------------------------------

from the mail below and from the one in my first post and run them through SA again.
Result ist that in the mail below the original SMTP gets found and in the other one SA still thinks the dynamic IP of the dsl-modem is the original SMTP server.


For testing purpose I switched the original IP 85.55.41.198 (of the senders dsl-modem) with 81.41.72.156 (which is the one from the first post) in the source of the mail just to be sure that the different IP won't make a difference. I also substituted the original surname of the sender with xxx.

Full headers:

-------------------------------------------------------------------------
Return-Path: <assis.xxx@web.de>
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on momo.seclinet.org
X-Spam-Level:
X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00
        autolearn=ham
        bayes=0.0000
        language=de
        report:
        * -3.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
        *      [score: 0.0000]
X-Flags: 1001
Delivered-To: GMX delivery to seclinet@gmx.net
Received: by localhost (fdm 1.5, account "gmx");
  Sat, 12 Jul 2008 18:07:54 +0000
Received: (qmail 19718 invoked by alias); 5 Jul 2008 21:12:56 -0000
Delivered-To: GMX delivery to anyaddress@gmx.net
Received: (qmail invoked by alias); 05 Jul 2008 21:12:56 -0000
Received: from fmmailgate01.web.de (EHLO fmmailgate01.web.de) [217.72.192.221]
  by mx0.gmx.net (mx005) with SMTP; 05 Jul 2008 23:12:56 +0200
Received: from smtp08.web.de (fmsmtp08.dlan.cinetic.de [172.20.5.216])
  by fmmailgate01.web.de (Postfix) with ESMTP id 5C447E69BF81
  for <anyaddress@gmx.net>; Sat,  5 Jul 2008 23:12:56 +0200 (CEST)
Received: from [85.55.41.198] (helo=[85.55.41.198])
  by smtp08.web.de with asmtp (TLSv1:RC4-MD5:128)
  (WEB.DE 4.109 #226)
  id 1KFF3v-0003u5-00
  for anyaddress@gmx.net; Sat, 05 Jul 2008 23:12:56 +0200
Subject: Re: Wegbeschreibung
From: Assis <assis.xxx@web.de>
To: Tom Fernandes <anyaddress@gmx.net>
In-Reply-To: <200807051433.43420.anyaddress@gmx.net>
References: <200807051433.43420.anyaddress@gmx.net>
Content-Type: text/plain;
  charset=utf-8
Date: Sat, 05 Jul 2008 22:26:55 +0200
Message-Id: <1215289615.8598.13.camel@localhost.localdomain>
Mime-Version: 1.0
X-Mailer: Evolution 2.10.3
Content-Transfer-Encoding: quoted-printable
Sender: assis.xxx@web.de
X-Sender: assis.xxx@web.de
X-GMX-Antivirus: 0 (no virus found)
X-GMX-Antispam: -2 (not scanned, spam filter disabled)
X-Resent-By: Forwarder <forwarder@gmx.net>
X-Resent-For: anyaddress@gmx.net
X-Resent-To: seclinet@gmx.net
X-GMX-UID: 4dtZAbI2QEVtHLCYOXRpsxJKNzg2NYJu
X-Length: 2506
X-UID: 1518

--------------------------------------------------------------------------


thanks,


Tom

I don't understand what you're saying.  the fdm handover is invisible to SA -- that's the problem.

Thinking about it more, I think the correct action for you to take is to trust the GMX relays and your own address in the dialup/dynamic address pool you're using, as I suggested.  fundamentally, even if fdm fixes their Received lines, the issue will still be that the mail is originating from a dialup pool -- but the key fact is that it's *your* host in the dialup pool, and that needs to be trusted.

In a case like this, fixing fdm won't help. The Spamhaus hits are the very same when using fetchmail, for example, which is perfectly identified by SA.

The issue is, that you indeed sent your mail via a single hop, aka direct MUA to MX. This is because your outgoing SMTP actually *is* the final recipients MX. You did send a mail from your dialup IP directly to the recipients MX. SA correctly checks that IP against the blacklists, and Spamhaus is punishing you...

FWIW, I'm not convinced that trusting the entire Orange dialup IP space is a proper workaround. ;)

(In reply to comment #6)
> FWIW, I'm not convinced that trusting the entire Orange dialup IP space is a
> proper workaround. ;)

actually, here's what might be better -- if you can login to the outbound relay using SMTP-AUTH SA should be able to detect that, and avoid applying those rules.

Hi,

(In reply to comment #7)
> actually, here's what might be better -- if you can login to the outbound relay
> using SMTP-AUTH SA should be able to detect that, and avoid applying those
> rules.

Test case 1) - headers below
Sent mail with MUA from dynamic IP with SMTP-AUTH through GMX's SMTP server to my own GMX address.
Result: SA thinks the dynamic IP is the originating SMTP server

Test case 2) - headers below
Sent mail with MUA from dynamic IP with SMTP-AUTH through web.de's SMTP server to my own web.de address.
Result: SA doesn't think the dynamic IP is the originating SMTP server


Removing the "Received:" lines added by fdm didn't make any changes.

I also went through some old mails that where send to me by people having an GMX account and using a MUA, and found out that in all of them SA confused the SMTP server to be dynamic IP address.



Full headers:

gmx headers ----------------------------------------------------------------
Return-Path: <seclinet@gmx.net>
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on momo.seclinet.org
X-Spam-Level: *
X-Spam-Status: No, score=1.6 required=5.0 tests=AWL,BAYES_05,RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RDNS_DYNAMIC,SPF_FAIL
        autolearn=no
        bayes=0.0402
        language=
        report:
        *  0.9 RCVD_IN_PBL RBL: Received via a relay in Spamhaus PBL
        *      [85.55.47.166 listed in zen.spamhaus.org]
        *  0.9 RCVD_IN_SORBS_DUL RBL: SORBS: sent directly from dynamic IP address
        *      [85.55.47.166 listed in dnsbl.sorbs.net]
        *  0.7 SPF_FAIL SPF: sender does not match SPF record (fail)
        *      [SPF failed: Please see http://www.openspf.org/Why?s=mfrom&id=seclinet%40gmx.net&ip=85.55.47.166&r=momo.seclinet.org]
        * -1.1 BAYES_05 BODY: Bayesian spam probability is 1 to 5%
        *      [score: 0.0402]
        *  0.1 RDNS_DYNAMIC Delivered to trusted network by host with
        *      dynamic-looking rDNS
        *  0.1 AWL AWL: From: address is in the auto white-list
X-Flags: 0000
Delivered-To: GMX delivery to seclinet@gmx.net
Received: by localhost (fdm 1.5, account "gmx");
  Thu, 17 Jul 2008 15:08:29 +0200
Received: (qmail invoked by alias); 17 Jul 2008 13:08:12 -0000
Received: from 166.pool85-55-47.dynamic.orange.es (EHLO [192.168.0.25]) [85.55.47.166]
  by mail.gmx.net (mp062) with SMTP; 17 Jul 2008 15:08:12 +0200
X-Authenticated: #22064775
X-Provags-ID: V01U2FsdGVkX19nTCR2S3ZnsGjQJEGZie3oRRwaX6FrXXjL3Di4py
  qYRO9qhe6E6KFo
From: Tom Fernandes <seclinet@gmx.net>
To: Tom Fernandes <seclinet@gmx.net>
Subject: test from gmx to gmx
Date: Thu, 17 Jul 2008 15:08:10 +0200
User-Agent: KMail/1.9.9
MIME-Version: 1.0
Content-Type: text/plain;
  charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <200807171508.10972.seclinet@gmx.net>
X-FuHaFi: 0.00
X-GMX-Antivirus: 0 (no virus found)
X-GMX-Antispam: -2 (not scanned, spam filter disabled)
X-GMX-UID: D31McHM3MmApdezCO2BngwIxMjQ1N52M
X-Length: 1949
X-UID: 1529

web.de headers --------------------------------------------------------
X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on momo.seclinet.org
X-Spam-Level:
X-Spam-Status: No, score=-5.4 required=5.0 tests=ALL_TRUSTED,BAYES_00
        autolearn=unavailable
        bayes=0.0000
        language=
        report:
        * -1.8 ALL_TRUSTED Passed through trusted hosts only via SMTP
        * -3.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1%
        *      [score: 0.0000]
Received: by localhost (fdm 1.5, account "web");
  Fri, 18 Jul 2008 12:58:22 +2400
Received: from [85.55.47.166] (helo=[192.168.0.25])
  by smtp05.web.de with asmtp (TLSv1:AES256-SHA:256)
  (WEB.DE 4.109 #226)
  id 1KJSul-0002uZ-00
  for carteras@web.de; Thu, 17 Jul 2008 14:48:55 +0200
From: Tom Fernandes <carteras@web.de>
To: carteras@web.de
Subject: test from web.de to web.de
Date: Thu, 17 Jul 2008 14:48:44 +0200
User-Agent: KMail/1.9.9
MIME-Version: 1.0
Content-Type: text/plain;
  charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <200807171448.44996.carteras@web.de>
Sender: carteras@web.de
X-Sender: carteras@web.de
X-Length: 1051
X-UID: 146

----------------------------------------------------------------------

If this is really an SA problem I will look into SA code on the weekend and see if can come up with a patch.

thanks so far,


Tom

After reading in lib/Mail/SpamAssassin/Message/Metadata/Received.pm and doing some tests I think that the issue is that my SMTP-AUTH with the GMX SMTP server is not recognized by SA.

AFAICT there is nothing in GMXs received headers where SA can tell from that I've been authenticating with GMX before sending my mail. It looks like GMX adds the "X-Authenticated" header for that but SA can't rely on that as it can be easily forged.

When I modify GMXs received header in the mail manually (switch SMTP with ASMTP) and the according part in Received.pm (the line where GMXs mailserver is matched) so that SA thinks that I've been authenticated before relaying through GMX, the connection is trusted and the SPF and other rules are not run on the dynamic IP.

It looks like this is a problem for all GMX users, using a pop fetcher + SA when receiving mails from somebody sending from a GMX account using a dialup IP. It does not make a difference if the sender uses SMTP or GMX webfrontend for sending. In both cases SA doesn't find out that the user authenticated before using GMX.

Wouldn't it be possible to trust GMX server if it is the first hop in the received headers and it's not listed as an MX?. AFAICT GMX is known to authenticate all users who are relaying through them and the MX servers don't allow relaying (I did a fast telnet-check).

If the headers of a mail send through the webfrontend is of any help - let me know.

thanks,


Tom

I just want to confirm the problem reported by Tom Fernandes.

to summarize: SA believes that all e-mails send via GMX through a MUA (not the GMX web interface) are SPF_FAIL, because the original IP of the computer running the MUA differs from the allowed SPF range for GMX

I am not sure whether this is an error in SA or an error in the internal email manipulation by GMX

PS: I use SMTP authentication when connecting to the GMX SMTP server