Bug 2744 - internal_networks support
Summary: internal_networks support
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Rules (show other bugs)
Version: 2.60
Hardware: Other Linux
: P1 normal
Target Milestone: 2.70
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-11-09 11:58 UTC by Michel Bouissou
Modified: 2004-02-08 12:25 UTC (History)
0 users



Attachment Type Modified Status Actions Submitter/CLA Status
internal_networks support patch None Justin Mason [HasCLA]
Test message causing DYNABLOCK FP text/plain None Michel Bouissou [HasCLA]
Test output with debug text/plain None Michel Bouissou [HasCLA]
bugfix on top of patch 1563 patch None Justin Mason [HasCLA]

Note You need to log in before you can comment on or make changes to this bug.
Description Michel Bouissou 2003-11-09 11:58:52 UTC
Hi there, 
 
According to The Fine Manual, 
<<trusted_networks ip.add.re.ss[/mask] ... (default: none) 
What networks or hosts are 'trusted' in your setup. Trusted in this case means 
that relay hosts on these networks are considered to not be potentially 
operated by spammers, open relays, or open proxies. DNS blacklist checks will 
never query for hosts on these networks.>> 
 
So I follow The Fine Manual, and put in my trusted_networks the IP of several 
SMTP servers of big ISPs, which I believe to be well-configured, not 
open-relays, not operated by spammers, from which I want to receive most mail, 
and thus are unnecessary checking in blacklists. 
 
Unfortunately, this has an undesired side-effect: 
 
When a user of these ISPs sends me mail, mail follows the following path: 
Dialup connection of user at BIGISP (Not trusted) => BIGISP SMTP server 
(Trusted) => My own server. 
 
In this case, I often get a RCVD_IN_DYNABLOCK hit because the sender's machine 
has been found in dynablock, probably because the algorithm checks the last 
UNTRUSTED machine that sent mail to the fist TRUSTED machine. 
 
Which, in such a situation, gives obviously an undesired result... 
 
Comments ?
Comment 1 Justin Mason 2003-11-09 18:42:13 UTC
Agh.  You're right.

I think the solution is to modify the -lastuntrusted lookup behaviour to include
trusted relays; so if a mail is relayed through *any* machine -- whether trusted
or untrusted -- that's OK and it won't hit dynablock.  (currently, it only
checks for a relay through an untrusted server, hence this bug.)

It's perfectly reasonable to trust the other ISP's relays, so just having those
in the trusted list should not cause Dynablock FPs.
Comment 2 Justin Mason 2003-11-09 18:42:36 UTC
changing milestone -- easy to fix if we all agree
Comment 3 Michel Bouissou 2003-11-09 22:53:02 UTC
Justin Mason wrote: 
> I think the solution is to modify the -lastuntrusted lookup behaviour 
> to include trusted relays; so if a mail is relayed through *any* machine 
> -- whether trusted or untrusted -- that's OK and it won't hit dynablock. 
 
Doing this would miss dynablock checks for all direct-to-MX spam sent thru 
one's secondary MX (as the secondary MX will relay to the primary, the dialup 
that sent direct-to-MX spam to the secondary MX wouldn't be 
dynablock-checked). This would be bad because a *lot* of direct-to-MX spam is 
sent thru secondary MXes (maybe spammer thinks secondary MXes are less 
protected ?). 
 
But the "trusted_network" system is much useful, as one often receives most of 
his legitimate mail from the 4-5 biggest ISPs in his country. So "trusting" 
them saves a lot of useless DNSBL queries, and saves much useless load onto 
the DNSBL servers as well. 
 
So I believe we need a supplementary option "my_mxes" separate from the 
trusted_networks. Machines on trusted_networks wouldn't be DNSBL checked, but 
only the machines that relay to one of "my_mxes" AND are not "trusted" would 
then be dynablock-checked. 
 
If "my_mxes" is not set, machines that relay to the last SMTP server should be 
checked (if not in the "trusted" list). 
 
Comments ? 
Comment 4 Duncan Findlay 2003-11-11 20:12:28 UTC
This sounds complicated... perhaps should wait until 2.62?
Comment 5 Justin Mason 2003-11-11 22:00:54 UTC
I think Michel is right -- a supplementary version of 'trusted_networks'
specifically for networks you *run*.  ie.:

trusted_networks = networks you trust will not send spam, you trust they will
report the correct rDNS data

my_mxes = the "external edge" of your network, where it meets the net

trusted_networks is "milder" than my_mxes; but my_mxes is required to support
Dynablock tests.

BTW I think I prefer "internal_networks" as a be a better name than "my_mxes". ;)
Comment 6 Justin Mason 2003-11-13 20:34:06 UTC
Created attachment 1563 [details]
internal_networks support

OK, here's the patch to add 'internal_networks' as discussed.

it's a bit of a big change, unfortunately:

 Conf.pm      |   43 ++++++++++++++++++++++++--
 EvalTests.pm |   97
+++++++++++++++++++++++++++++++++++++++--------------------
 Received.pm  |   58 ++++++++++++++++++++++++++++++++---
 3 files changed, 159 insertions(+), 39 deletions(-)

Here's some log of me explaining it to Dan:

(16:15:30) justinmason23: one thing though, is the Dynablock/trusted_networks
problem
(16:16:13) justinmason23: I think the only way to fix that properly is to add
an 'internal_networks' analog to 'trusted_networks', make it very conservative
(no inference), and use that for Dynablock checks
(16:17:28) justinmason23: because the trusted-network stuff combined with
Dynablock is causing big FPs when it makes the wrong guesses
(16:17:28) Dan: so, dialup checks would only activate if internal_networks is
set?
(16:17:48) justinmason23: no, it'd use dialup checks -- but it'd never consider
any host internal past the host the scanner is running on
(16:18:22) justinmason23: so users who have a network where SA is not running
on an external MX, will miss the dynablock hits
(16:18:30) justinmason23: unless they set 'internal_networks' correctly
(16:19:39) Dan: how is that different than trusted_networks ?
(16:19:48) justinmason23: trusted_networks does inference
(16:20:01) justinmason23: also it's used to trim out hosts from the list of DNS
checks
(16:20:18) justinmason23: people are setting it to include their friends' ISPs
as well, which is throwing off Dynablock
(16:21:01) justinmason23: basically, TN is being used for meaning *both* "my
networks" and "hosts I trust not to forge mail"
(16:21:12) justinmason23: this double meaning is the problem
(16:22:07) Dan: ah, extending too far
(16:22:19) justinmason23: yeah
(16:22:26) justinmason23: the use of it for Dynablock checks is what causes the
problem
(16:22:30) Dan: I misunderstood because I was thinking of the OTHER dialup
problem.
(16:22:39) Dan: you also have to address internal dialup hosts.
(16:23:02) justinmason23: there's an OK workaround for that though
(16:23:15) justinmason23: either (a) the host doesn't scan mail from their own
dialup networks
(16:23:28) Dan: I think the solution is to only do dynamic/dialup based on
internal networks... if unset, do nada... no inference
(16:24:15) justinmason23: or (b) the admin configures the dialup networks as
being part of internal_networks and SA handles not running the dynablock check
(16:24:28) justinmason23: yep, that's what I'm suggesting
(16:24:48) Dan: no inference and don't even run them if internal_networks is
unset
(16:25:11) justinmason23: hmm
(16:25:56) justinmason23: Well, if we did that then most people would lose the
dynablock hits, even though it's only a very small number of people who are
running into the "dynablock hitting our own dialup pool" problem
(16:26:35) Dan: I think the score is so high we have to do it.
(16:27:19) justinmason23: I think we could use "just this host" for the default
internal_networks setting for most people, then just the people who *have*
their own dialup pools have to set it for that problem case
(16:28:02) Dan: that would be ok, I guess
(16:28:58) justinmason23: that would mean that most people wouldn't lose their
hit rate
Comment 7 Duncan Findlay 2003-11-13 22:08:34 UTC
-0
It's a pretty big change in a point release, and would come close to justifying
a GA run (I agree that would be unecessary, but not by all that much).

If it's not buggy, go for it, but at this point, it's not really tested, and
it's a configuration file change in a point release.
Comment 8 Michel Bouissou 2003-11-13 22:48:33 UTC
I'll test this one and let you know. 
 
[root@totor 5.6.1]# patch -p1 < 
/home/michel/A_Installer/SpamAssassin/2.60_patches/trusted_networks.patch 
patching file Mail/SpamAssassin/Conf.pm 
Hunk #1 succeeded at 223 (offset -2 lines). 
Hunk #2 succeeded at 468 (offset -2 lines). 
Hunk #3 succeeded at 1352 (offset -3 lines). 
Hunk #4 succeeded at 1396 (offset -3 lines). 
patching file Mail/SpamAssassin/EvalTests.pm 
Hunk #1 succeeded at 696 (offset -4 lines). 
Hunk #2 succeeded at 919 (offset -74 lines). 
Hunk #3 succeeded at 1149 (offset -74 lines). 
Hunk #4 succeeded at 1189 (offset -74 lines). 
Hunk #5 succeeded at 1212 (offset -74 lines). 
Hunk #6 succeeded at 1299 (offset -74 lines). 
patching file Mail/SpamAssassin/Received.pm 
Hunk #2 succeeded at 84 (offset -13 lines). 
Hunk #3 succeeded at 221 (offset -29 lines). 
Hunk #4 succeeded at 521 (offset -87 lines). 
Hunk #5 succeeded at 919 (offset -100 lines). 
 
Comment 9 Michel Bouissou 2003-11-13 22:49:59 UTC
Should my own MXes be in both internal_networks and trusted_networks, or only 
in trusted_networks ? 
Comment 10 Justin Mason 2003-11-13 23:49:43 UTC
they should be in both -- trusted_networks should contain internal_networks.

the idea is that internal_networks is *just* your nets and MXes; trusted can
contain other people's mailservers and networks (if you trust them not to
originate spam or be running a subverted machine).
Comment 11 Michel Bouissou 2003-11-13 23:57:35 UTC
Yes, but I believe that one usually "trusts" his "internal_networks". 
 
So putting your own machines both in "trusted_networks" and 
"internal_networks" is allright, but maybe a bit of unnecessary hassle. 
 
Putting your own machines only in "internal_networks" should be enough, and 
the system should determine that "internal" machines are automatically 
"trusted" as well. 
 
(The tests that currently check for "trusted" should check for "trusted or 
internal", that would do the trick) 
 
It would just make configuration easier... 
 
Comment 12 Michel Bouissou 2003-11-14 00:02:06 UTC
The patch doesn't seem to work. I just got another DYNABLOCK FP: 
 
X-Spam-RBL-Results: <dns:32.90.8.80.dynablock.easynet.nl?type=TXT>  
	["Dynamic/Residential IP range listed by easynet.nl DynaBlock -  
	http://dynablock.easynet.nl/errors.html"] 
X-Spam-Trusted-Relays: [ ip=80.67.174.41 rdns=samizdat.net  
	helo=slut.samizdat.net by=totor.bouissou.net ident= intl=0 ] [  
	ip=193.252.22.28 rdns=smtp3.wanadoo.fr helo=mwinf0304.wanadoo.fr  
	by=slut.samizdat.net ident= intl=0 ] 
X-Spam-DCC: Etherboy: totor.bouissou.net 1002; Body=1 Fuz1=1 Fuz2=1 
X-Spam-Status: No, hits=-103.4 required=5.0 tests=AWL,BAYES_00, 
	RCVD_IN_DYNABLOCK,USER_IN_WHITELIST autolearn=no version=2.60 
X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on  
	totor.bouissou.net 
X-Spam-Possible-Lang: fr 
X-Spam-Level:  
X-Spam-Untrusted-Relays: [ ip=80.8.90.32  
	rdns=ca-bordeaux-18-32.w80-8.abo.wanadoo.fr helo=something  
	by=mwinf0304.wanadoo.fr ident= intl=0 ] 
 
Comment 13 Michel Bouissou 2003-11-14 00:36:09 UTC
Another DYNABLOCK FP, and another issue as well: It seems that the last relay 
that sent the mail to my MX is ignored by SA, as it doesn't appear at all in 
SA relays list. 
Possibly because this machine gave an IDENT information that, formatted by the 
receiving qmail, cause SA could not understand the IP format...? 
 
Let's see: 
 
In local.cf, I have: 
 
trusted_networks 213.228.0/24 
 
The e-mail Received: headers are: 
 
Received: from postfix3-2.free.fr (foobar@213.228.0.169) 
  by totor.bouissou.net with SMTP; 14 Nov 2003 08:05:50 -0000 
Received: from asterix.laurier.org (lns-p19-8-82-65-66-244.adsl.proxad.net 
[82.65.66.244]) 
	by postfix3-2.free.fr (Postfix) with ESMTP id 7BACDC372 
	for <michel@bouissou.net>; Fri, 14 Nov 2003 09:05:49 +0100 (CET) 
 
SA diags are: 
 
X-Spam-RBL-Results: <dns:244.66.65.82.dynablock.easynet.nl?type=TXT>  
	["Dynamic/Residential IP range listed by easynet.nl DynaBlock -  
	http://dynablock.easynet.nl/errors.html"] 
X-Spam-Trusted-Relays:  
X-Spam-DCC: Etherboy: totor.bouissou.net 1002; Body=1 Fuz1=1 Fuz2=1 
X-Spam-Status: No, hits=-103.2 required=5.0 tests=AWL,BAYES_00, 
	RCVD_IN_DYNABLOCK,USER_IN_WHITELIST autolearn=no version=2.60 
X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on  
	totor.bouissou.net 
X-Spam-Possible-Lang: fr 
X-Spam-Level:  
X-Spam-Untrusted-Relays: [ ip=82.65.66.244  
	rdns=lns-p19-8-82-65-66-244.adsl.proxad.net helo=asterix.laurier.org  
	by=postfix3-2.free.fr ident= intl=0 ] 
 
As you can see, the "postfix3-2.free.fr (foobar@213.228.0.169)" relay, that 
should appear in the "trusted relays" section, doesn't show up. SA has missed 
it. 
 
(I had already seen this "missed relay" issue before applying today's patch, 
so it's not the patch that caused this) 
 
Please note that the "Received: from postfix3-2.free.fr 
(foobar@213.228.0.169)" format is the usual Received: line format used by 
qmail. "foobar@" appears when the remote SMTP server answered and "IDENT" 
request giving "foobar" as username. 
 
qmail Received format is: 
 
Received: from (<remote_machine_rDNS>|unknown) 
\((<IDENT_username@>)?<IP_address>\) by <receiving_hostname> with <protocol>; 
<timestamp> 
 
Comment 14 Michel Bouissou 2003-11-14 00:45:48 UTC
Sorry, I was incomplete 
 
qmail Received format is:  
  
Received: from (<remote_machine_rDNS>|unknown) (\(HELO 
<helo_name_given_by_remote_if_different_from_rDNS>\))? 
\((<IDENT_username@>)?<IP_address>\) by <receiving_hostname> with <protocol>;  
<timestamp> 
 
This can make for example: 
 
Received: from unknown (HELO feux01a-isp) (213.199.4.210) 
  by totor.bouissou.net with SMTP; 1 Nov 2003 07:05:19 -0000 
 
or 
 
Received: from x1-6-00-04-bd-d2-e0-a3.k317.webspeed.dk 
(benelli@80.167.158.170) 
  by totor.bouissou.net with SMTP; 5 Nov 2003 23:18:42 -0000 
 
or 
 
Received: from adsl-207-213-27-129.dsl.lsan03.pacbell.net (HELO merlin.net.au) 
(Owner50@207.213.27.129) 
  by totor.bouissou.net with SMTP; 10 Nov 2003 06:30:34 -0000 
 
Comment 15 Theo Van Dinter 2003-11-14 14:43:46 UTC
-0   I have to agree with duncf, it's too much for a point release.
Comment 16 Justin Mason 2003-11-14 19:36:45 UTC
Michel -- that Received header issue is best dealt with under a separate bug --
here it is:
http://bugzilla.spamassassin.org/show_bug.cgi?id=2759

Folks, I think I agree the internal_networks thing is best left for 2.70. 
Although I am worried about sites that'll see FPs heavily from Dynablock without it.

Maybe we should *ignore* trusted_networks when trying to use dynablock; always
use the entire Received chain for that rule.
Comment 17 Michel Bouissou 2003-11-15 00:36:31 UTC
Justin Mason wrote: 
> Folks, I think I agree the internal_networks thing is best left for 2.70. 
> Although I am worried about sites that'll see FPs heavily from Dynablock 
> without it. 
 
Too bad, the patch for internal_networks being already written... 
 
It already correctly identifies internal servers (I see intl=0 or intl=1 
display correctly in mail headers). 
There probably remains a little bug in the "which machine to DYNABLOCK test" 
routine, but that should be easier to fix than taking another direction now ? 
 
Anyway, I can't use the "trusted_networks" system as long as this is not 
fixed, as I get 100% DYNABLOCK FPs for all the customers of big ISPs that I 
would "trust"... 
Comment 18 Michel Bouissou 2003-11-16 03:10:14 UTC
I have taken a look at Justin's patch, trying to undestand why I still get FPs 
on DYNABLOCK checks, even with the new "internal_networks" system. I must 
admit I didn't completely understand the current underlying logic, at it seems 
rather complex, and I'm not very familar with Perl. 
 
"dialup RBLs" tests are useful only for catching direct-to-MX spam, and what 
we can catch with this is: 
- Spam sent by a dialup/dynamic IP address which is OUT of our own network or 
ISP's dynamic IP pool, directly to our own network's or ISP's MX. 
 
I believe that the logic for testing "dialup RBLs" should be the following. 
 
We should check an IP against a "dialup RBL" only if: 
 
- The "Received: ... by" (receiving machine) is either in our 
"internal_networks", OR is the last "Received:" (in case we haven't set 
internal_networks, or have forgotten one of our MXes in it, we can always be 
sure that the last Received: is by one of our "own legitimate" MXes, which is 
a safe inference) 
=> Because these checks make sense only at the edge of our network. 
 
- The "Received: from" (sending machine) is: 
- external (not in internal_networks) 
- AND untrusted (not in trusted_networks) 
- AND its rDNS isn't in the same domain as the "Received: ... by" 
- AND its IP is not in the same /16 as the "Received: ... by" 
 
(These last 2 tests for people that will set their own ISP's MXes as 
"internal". They probably won't know / won't set the complete dialup pools of 
their own ISP as "internal" or "trusted", but we wouldn't want legitimate mail 
sent by the same ISP's customer thru its MX to trigger a DYNABLOCK test. In 
such a case, customer of ISPs that don't properly define rDNS for their own 
dynamic IP pools may trigger DYNABLOCK tests, but we can't do much for ISPs 
which network is wrotten. That's their own problem to define rDNS properly.) 
 
Did I miss something in there ? 
 
Comment 19 Justin Mason 2003-11-17 22:58:40 UTC
no, that's pretty much correct.  It's complicated a little because 
the logic is split between 2 functions:

1. Received.pm, the header parser and trust-determining part
2. EvalTests.pm, which picks what IPs to check from the "untrusted" set

To deal with the 'IP is not in the same /16 as the "Received: ... by"' test, the
latter would have to be extended to check the relay's IP against the IP of the
next relay in the list.  (that's "next" in terms of time, not list order.) 

I don't think checking the rDNS is worthwhile though as that is spoofable.
Comment 20 Justin Mason 2003-11-17 22:58:57 UTC
no longer in [review] state
Comment 21 Michel Bouissou 2003-11-18 06:38:34 UTC
So could you find out why you proposed patch for internal_networks doesn't 
work as expected ? Do you think it would be hard to fix given the current 
status of this patch ? 
 
I'd love to get it working, even if the patch isn't part of the official 
distribution for now... 
Comment 22 Justin Mason 2003-11-19 23:32:04 UTC
Michel, could you attach a message that FPs with the internal_networks patch,
and a debug log from "spamassassin -D -t < message"?
Comment 23 Michel Bouissou 2003-11-20 00:02:07 UTC
Yes, I will attach: 
- A test message (removed real contents from it, but it's enough for FP'ing). 
- Test output. 
 
I made the test with 
./test-received < test-rcvd6.txt > test-output.txt 
 
Where test-received is: 
#! /bin/bash 
cat | spamassassin -D received=255 2>&1 | egrep -v "^(debug: 
(tokenize:|bayes|(Current |Final )?PATH|[Rr]unning in|using \"|Score set)| 
Razor-Log:|nov [0-9][0-9] [0-9][0-9])" | egrep -v 
"(Razor2|helper-app|dccproc|[Ll]anguage)" 
 
(just for keeping the output short and only get what we need). 
 
In this example, my local.cf says "trusted_networks 213.228.0/24" 
 
So this is correct: 
X-Spam-Trusted-Relays: [ ip=213.228.0.169 rdns=postfix3-2.free.fr helo= 
        by=totor.bouissou.net ident=foobar intl=0 ] 
 
And this is correct as well: 
X-Spam-Untrusted-Relays: [ ip=82.65.66.244 
        rdns=lns-p19-8-82-65-66-244.adsl.proxad.net helo=asterix.laurier.org 
        by=postfix3-2.free.fr ident= intl=0 ] 
 
But what is not correct is that 82.65.66.244 got DYNABLOCK-Checked where it 
was relaying to 213.228.0.169 (which is not internal), resulting in: 
X-Spam-RBL-Results: <dns:244.66.65.82.dynablock.easynet.nl?type=TXT> 
        ["Dynamic/Residential IP range listed by easynet.nl DynaBlock - 
        http://dynablock.easynet.nl/errors.html"] 
X-Spam-Status: No, hits=-2.3 required=5.0 tests=BAYES_00,RCVD_IN_DYNABLOCK 
        autolearn=no version=2.60 
 
Comment 24 Michel Bouissou 2003-11-20 00:03:30 UTC
Created attachment 1578 [details]
Test message causing DYNABLOCK FP
Comment 25 Michel Bouissou 2003-11-20 00:04:43 UTC
Created attachment 1579 [details]
Test output with debug

This gives both debug output, and contents of the message once processed
Comment 26 Justin Mason 2003-11-20 11:19:56 UTC
Created attachment 1580 [details]
bugfix on top of patch 1563

ok, Michel, please try this one.
Comment 27 Justin Mason 2003-11-20 11:21:58 UTC
BTW that applies on top of patch 1563 -- it's cumulative.
Comment 28 Michel Bouissou 2003-11-20 11:47:20 UTC
Looks better at first sight. At least it doesn't give a DYNABLOCK FP on the 
test messsage I sent you this morning anymore. 
 
I'll keep it running like this, and check coherence as I receive new mail. 
 
Thanks :-) 
Comment 29 Michel Bouissou 2003-11-21 01:06:00 UTC
I've tested the "corrected patch" on a variety of received mail, sent directly 
to my primary MX or thru my secondary, with messages that must hit "DYNABLOCK" 
and others that must not. 
 
It seems that everything now works plain good in the internal_networks / 
trusted_networks system. 
 
So I think this patch should be integrated into SA. 
 
+1 :-) 
 
Comment 30 Justin Mason 2003-11-21 10:34:10 UTC
Subject: Re: [SAdev]  trusted_networks don't behave as expected 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


>So I think this patch should be integrated into SA. 

Excellent news!

All that remains now is determining if the devs want to wait for
2.70, since it is adding a new conf item.   However, I think it
should go into 2.61, since:

  1. it fixes a widespread, FP-causing issue with DYNABLOCK.  This
  is causing a *lot* of FPs.

  2. it is backwards compatible; users who haven't set trusted_networks or
  have set that but not set internal_networks will get sane defaults.

  3. we've added conf items to point releases in the past, specifically
  the bayes expiry ones for 2.5x.

- --j.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Exmh CVS

iD8DBQE/vlqcQTcbUG5Y7woRAmigAJ9mImwJlOTc2Th4L+5+9kHkAOKzzgCguSJO
lb0JVHDH1rkby7lpakOgYlA=
=wKMY
-----END PGP SIGNATURE-----

Comment 31 Michel Bouissou 2003-11-21 10:58:33 UTC
Justin Mason wrote: 
> However, I think it should go into 2.61, since: 
> 
>  1. it fixes a widespread, FP-causing issue with DYNABLOCK.  This 
>  is causing a *lot* of FPs. 
 
...and DYNABLOCK scores quite high... 
 
>  2. it is backwards compatible; users who haven't set trusted_networks or 
>  have set that but not set internal_networks will get sane defaults. 
> 
>  3. we've added conf items to point releases in the past [...] 
 
I completely share your opinion about this. This fix fixes a serious bug and 
can cause no harm, so it should be integrated asap. 
 
There's an issue however which I couldn't test and for which I don't know how 
the system will behave: People that might set their own ISP's SMTP servers as 
"internal" and "trusted" (people getting mail from their ISP using fetchmail 
for example). 
 
In this case there might possibly remain a risk that users of the same ISP 
sending mail from the ISP's dialup pool directly to the ISP's MX, could still 
get DYNABLOCK FP's. 
 
I'm not sure about this, but it could be, if there is no comparison done 
between the domain of the sender machine (from rDNS) and the domain of the 
receiving MX ? 
 
I don't really know how to fix this, besides: 
- Not putting your own ISP's server in the "internal networks" 
- Or adding their complete dialup pools as "internal" as well ? 
- Or adding domain comparison between sender machine and receiving MX ? 
 
Cheers. 
Comment 32 Justin Mason 2003-11-21 11:22:50 UTC
Subject: Re: [SAdev]  trusted_networks don't behave as expected 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


>There's an issue however which I couldn't test and for which I don't know how 
>the system will behave: People that might set their own ISP's SMTP servers as 
>"internal" and "trusted" (people getting mail from their ISP using fetchmail 
>for example). 
> 
>In this case there might possibly remain a risk that users of the same ISP 
>sending mail from the ISP's dialup pool directly to the ISP's MX, could still 
>get DYNABLOCK FP's. 
> 
>I'm not sure about this, but it could be, if there is no comparison done 
>between the domain of the sender machine (from rDNS) and the domain of the 
>receiving MX ? 
> 
>I don't really know how to fix this, besides: 
>- Not putting your own ISP's server in the "internal networks" 
>- Or adding their complete dialup pools as "internal" as well ? 
>- Or adding domain comparison between sender machine and receiving MX ? 

I think the domain comparison will probably fail in a lot of cases
anyway; consider ISPs that have a different domain for their dialup
ranges and their mailservers.  (some do, e.g. earthlink.net.)

IMO the best answer here is to add the complete dialup pools as internal.

- --j.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Exmh CVS

iD8DBQE/vmYEQTcbUG5Y7woRAtvIAJ9NBXw2dIihwJ6ik3JNH/53xtd0hgCgiN7e
MJ0tOOkwq3aFfabWzeLZ1Ak=
=urYw
-----END PGP SIGNATURE-----

Comment 33 Michel Bouissou 2003-11-21 15:46:14 UTC
Justin Mason wrote: 
> I think the domain comparison will probably fail in a lot of cases 
> anyway; 
 
Sure. But it might help avoiding a significant number of FPs for people who 
would configure their own ISP's MXes as "internal" without thinking of 
configuring their ISP's dialup pools as internal as well... 
 
> consider ISPs that have a different domain for their dialup 
> ranges and their mailservers.  (some do, e.g. earthlink.net.) 
 
<kidding> 
Well, I receive tons of spam from their netblocks, I don't mind if they 
trigger any possible test ;-) 
</kidding> 
 
> IMO the best answer here is to add the complete dialup pools as internal. 
 
Sure, but I don't know many people who know the comprehensive list of their 
ISP's dialup pools (at least, I don't know mine and don't care much ;-) 
 
Maybe specifying "internal_networks" as IP ranges _or_ domain names could 
help. Of course, rDNS can sometimes be forged, but I don't believe many 
spammers make the effort of faking their rDNS record before sending spam to 
each and every domain they use to spam... 
 
Anyway, the current patch already gives a big improvement; I was just thiking 
of possible ways to improve it. 
 
Cheers. 
 
Comment 34 Justin Mason 2003-12-08 22:21:31 UTC
ok, I think the other devs don't want to see this happening in the 2.60 series,
so I'll punt it to 2.70.  However, people who run into the issue (ISPs, that
is), can always apply the patches here in the meantime to their 2.6x installations.
Comment 35 Michel Bouissou 2003-12-09 03:43:51 UTC
Too bad, where it works perfectly, fixes an FP-causing bug, and wouldn't harm 
any user upgrading, even if not paying attention to the new feature and not 
configuring internal_networks. 
 
I don't really understand the reason for postponing the release of an existing 
fix, even if this fix adds a new "feature" that cannot harm. 
 
But I've seen SA 2.61 is already released, so I guess it's a bit late for 
discussing this issue. I hope to see it in 2.62 rather than have to wait for 
2.70 anyway. 
 
Cheers. 
Comment 36 Theo Van Dinter 2003-12-09 08:20:51 UTC
Subject: Re: [SAdev]  trusted_networks don't behave as expected

On Tue, Dec 09, 2003 at 03:43:52AM -0800, bugzilla-daemon@bugzilla.spamassassin.org wrote:
> I don't really understand the reason for postponing the release of an existing 
> fix, even if this fix adds a new "feature" that cannot harm. 

Ignoring this specific issue, maintenance releases are just that ...
Adding features isn't maintenance, so it's best to wait for the next
non-maintenance release.

Comment 37 Justin Mason 2004-02-08 21:25:29 UTC
ok, this has been in 2.70 for a while.  marking FIXED