Updated to:

=item dns_options opts   (default: empty)                                     

Provides a (whitespace or comma -separated) list of options applying to       
DNS resolving. Available options are 'rotate' and 'dns0x20' (without quotes). 
Option name may be negated by prepending a 'no' (e.g. 'norotate') to          
counteract previously enabled option. The last setting in configuration       
files prevails. By default options 'rotate' and 'dns0x20' are disabled.       
                                    
Option 'rotate' [...]
           
Option 'dns0x20' enables randomization of letters in a DNS query label        
according to draft-vixie-dnsext-dns0x20, decreasing a chance of collisions    
of responses (by chance or by a malicious intent) by increasing spread        
as provided by a 16-bit query ID and up to 16 bits of a port number,          
with additional bits as encoded by flipping case (upper/lower) of letters     
in a query. The number of additional random bits corresponds to the number    
of letters in a query label. Should work reliably with all mainstream         
DNS servers - do not turn on if you see frequent info messages                
"dns: no callback for id:" in the log, or if RBL or URIDNS lookups            
do not work for no apparent reason.                                           


trunk:

Bug 6338: Use of Bit 0x20 in DNS Labels to Improve Transaction Identity
(adds 'dns_options dns0x20', allows negation of dns options, off by default)
Sending        lib/Mail/SpamAssassin/Conf.pm
Sending        lib/Mail/SpamAssassin/Dns.pm
Sending        lib/Mail/SpamAssassin/DnsResolver.pm
Transmitting file data ...
Committed revision 911235.

preserve entropy across Razor call,
reveal less of a random accumulator at a time
Sending        lib/Mail/SpamAssassin/DnsResolver.pm
Sending        lib/Mail/SpamAssassin/Plugin/Razor2.pm
Transmitting file data ..
Committed revision 911411.

Doesn't this just change the query and not check it is valid on return? Or is the return check already done elsewhere?

> Doesn't this just change the query and not check it is valid on return?
> Or is the return check already done elsewhere?

It is checked by existing code in poll_responses: it tries to associate
a 'signature' (i.e. $self->_packet_id($packet) ) of a received reply
with a 'signature' of a transmitted query, they must match exactly.
If they do not, we end up with a "dns: no callback for id:" message.

This 'signature' is constructed in _packet_id() by concatenating the:
id, query-string, type, and class. (The port number matching is done
by the Net::DNS and a network layer). The 'signature' of a query section
in a reply packet must match exactly (no case-insensitivity) to the
query section of the transmitted request. The query-string (e.g. a
case-permuted host name) is thus an integral part of the matching,
any changes in case of its letters would break the match.

Awesome. Thanks.

Now that I think of it, even the current (3.2, 3.3) code relies on a query
section of a reply packet matching exactly the query packet. This happens
to work on all mainstream DNS servers, but there is no guarantee for this
in a form of a RFC requirement.  In essence, we are already depending on
poor-man's form of a dns0x20, just without any additional entropy.

The 'dns_options dns0x20' could just as well default to true, without
breaking anything that isn't already broken.

Does anybody feel we need to lift the requirement for an exact (case-for-case)
match when dns0x20 option is NOT enabled? Some poor soul on a cheap home
router/firewall/dns-server may be affected by this without knowing why
his SA DNS queries sometimes fail.

(In reply to comment #6)
> Now that I think of it, even the current (3.2, 3.3) code relies on a query
> section of a reply packet matching exactly the query packet. This happens
> to work on all mainstream DNS servers, but there is no guarantee for this
> in a form of a RFC requirement.  In essence, we are already depending on
> poor-man's form of a dns0x20, just without any additional entropy.
> 
> The 'dns_options dns0x20' could just as well default to true, without
> breaking anything that isn't already broken.
> 
> Does anybody feel we need to lift the requirement for an exact (case-for-case)
> match when dns0x20 option is NOT enabled? Some poor soul on a cheap home
> router/firewall/dns-server may be affected by this without knowing why
> his SA DNS queries sometimes fail.

If I understand you correctly, currently we require an exact match and that could be causing some cheap-home DNS proxies, etc. to be having problems?

I'd say let's add a dbg that outputs the non-match if we can and if that generates a lot of user questions, we open a different bug and address.

KAM

I'd like to let this "bake" in trunk for a little bit more, btw, before putting it in 3.3.x, if that's ok.

(In reply to comment #6)
> Does anybody feel we need to lift the requirement for an exact (case-for-case)
> match when dns0x20 option is NOT enabled? Some poor soul on a cheap home
> router/firewall/dns-server may be affected by this without knowing why
> his SA DNS queries sometimes fail.

Are you talking about a DNS server responding with a canonicalized, all lower case version?

Anyway, the "cheap home router" scenario is exactly what's most likely to fail ALWAYS for large DNS BLs, as apposed to "sometimes" due to response mis-match -- the DHCP assigned forwarder ISP's DNS is most likely to be blacklisted by major DNS BLs, and that user won't get any hits for them.

> I'd like to let this "bake" in trunk for a little bit more, btw,
> before putting it in 3.3.x, if that's ok.

Sure, I haven't touched the target milestone.

> If I understand you correctly, currently we require an exact match and that
> could be causing some cheap-home DNS proxies, etc. to be having problems?

Exactly. Or maybe with some proprietary/vendor DNS server used by
some ISP or a company.

> I'd say let's add a dbg that outputs the non-match if we can and if that
> generates a lot of user questions, we open a different bug and address.

Issuing a dbg() or info() on a mismatch would probably be a good idea.
Unfortunately it is not trivial to implement - I guess that all queries
would need to be stored twice, once with exact keys, and once with
lowercased keys - as we depend or a perl hash for reverse mapping of a
reply packet back to our query. I'll think about it.

> Are you talking about a DNS server responding with a canonicalized,
> all lower case version?

Yes, likely, or all-caps, or maybe passing back whatever comes from
the authority server.

The quoted draft states:

 6.1. Several popular authoritative DNS implemenations including ISC BIND
 (versions 4, 8, and 9), Nominum ANS, Akamai AKADNS, Neustar UltraDNS,
 Verisign Atlas, NLNetLabs NSD, PowerDNS, and DJBDNS were tested.  All
 copied the question name exactly, bit for bit, from the request into the
 response.

 6.2. Operational testing has revealed a small set of rare and/or private
 label authoritative DNS implementations who modify the 0x20 bits in
 question names while copying the question section from the request to
 the response.  Usually this modification is to set the 0x20 bit, thus
 converting a domain name to be all-lower-case (0x61..0x7A, e.g., a-z).


> Anyway, the "cheap home router" scenario is exactly what's most likely to
> fail ALWAYS for large DNS BLs, as apposed to "sometimes" due to response
> mis-match -- the DHCP assigned forwarder ISP's DNS is most likely to be
> blacklisted by major DNS BLs, and that user won't get any hits for them.

Not sure I follow. What I meant by 'sometimes' is that if a queried label
happens to be e.g. all-lowercase, a query may succeed, but may fail with a
mixed-case query. In our case, a queried label may come from a mail header
section (like From, To, ...), or from a rule specifying a RBL/URIBL.

(In reply to comment #6)
> Now that I think of it, even the current (3.2, 3.3) code relies on a query
> section of a reply packet matching exactly the query packet. This happens
> to work on all mainstream DNS servers, but there is no guarantee for this
> in a form of a RFC requirement.  In essence, we are already depending on
> poor-man's form of a dns0x20, just without any additional entropy.
> 
> The 'dns_options dns0x20' could just as well default to true, without
> breaking anything that isn't already broken.
> 
> Does anybody feel we need to lift the requirement for an exact (case-for-case)
> match when dns0x20 option is NOT enabled? Some poor soul on a cheap home
> router/firewall/dns-server may be affected by this without knowing why
> his SA DNS queries sometimes fail.


trunk (3.4.0):

Bug 4260: rewrite DNS code to use a single socket, event-based model;
Bug 6338: Use of Bit 0x20 in DNS Labels to Improve Transaction Identity;

Remove case sensitivity on DNS label in a query section
when 'dns_options dns0x20' is NOT used - RFC 1035 does NOT require
DNS server to preserve case in a query section of a reply, even
though in practice all mainstream DNS servers do preserve it.
In effect, code before this patch was half-way between dns0x20
and nodns0x20 setting.

Sending lib/Mail/SpamAssassin/DnsResolver.pm
Committed revision 928955.

(In reply to comment #10)
> > Anyway, the "cheap home router" scenario is exactly what's most likely to
> > fail ALWAYS for large DNS BLs, as apposed to "sometimes" due to response
> > mis-match -- the DHCP assigned forwarder ISP's DNS is most likely to be
> > blacklisted by major DNS BLs, and that user won't get any hits for them.
> 
> Not sure I follow. What I meant by 'sometimes' is that if a queried label
> happens to be e.g. all-lowercase, a query may succeed, but may fail with a
> mixed-case query. In our case, a queried label may come from a mail header
> section (like From, To, ...), or from a rule specifying a RBL/URIBL.

I was referring to your concern in comment 6 about the exact case match with respect to "some poor soul on a cheap home router", who might be affected.

My point being, that the cheap hardware usually just forwards DNS to the ISP's DNS. Which is likely to be blocked by major DNS BLs. Thus, unless the poor soul runs his own caching nameserver locally, he won't be affected by case mismatches due to the cheap hardware, but actually not get any blacklisted response off of the major DNS BLs.

closing, implemented and in use in 3.4.0 for a year now