Created attachment 3953 [details]
sample email that should not have been marked with DKIM_POLICY_SIGNSOME

No Domainkey policy has been published by this domain:

[mcdonalddj@mcdonalddj-dc Desktop]$ dig _policy._domainkey.appletab.com txt

; <<>> DiG 9.4.1 <<>> _policy._domainkey.appletab.com txt
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13435
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

The message is most likely spam, but not because of DKIM.

> DKIM_POLICY_SIGNSOME hits on every message when there is no domainkeys
> policy in place.

That is a correct behaviour.


> sample email that should not have been marked with DKIM_POLICY_SIGNSOME
> No Domainkey policy has been published by this domain

RFC 4870:

   _domainkey   IN TXT "t=y; o=-; n=notes; r=emailAddress"

   All tags are optional.  The current valid tags are as follows:
   [...]
      o = Outbound Signing policy ("-" means that this domain signs all
          email; "~" is the default and means that this domain may sign
          some email with DomainKeys).
   [...]
   If the policy TXT record does not exist, recipient systems MUST
   assume the default values.

There is not yet an equivalent policy publishing mechanism for DKIM.
Since the DKIM plugin also implements the DomainKeys, and as using
the old DK policy publishing mechanism is the current practice,
I believe this behaviour of the DKIM plugin may remain as it is
for the time being.

  Mark

I interpret these comments as meaning that this works as designed, so I'm
closing this. Please reopen if I'm wrong about that.

I actually don't like it.  I noticed it before but forgot to look into it.

If the rule is going to hit on every message, unless it hits another DKIM policy
rule, there's no point to the rule.

I would like to revert to only triggering the rule if there is reason to believe
that the sender actually uses DKIM.

> If the rule is going to hit on every message, unless it hits
> another DKIM policy rule, there's no point to the rule.
> I would like to revert to only triggering the rule if there
> is reason to believe that the sender actually uses DKIM.

Please see Bug 5332.

The policy records are pretty much useless today, nobody wants
to claim anything more than 'signs some mail', which is
a default anyway. I recommend setting the score of the
following rules to zero (instead of .001):
  DKIM_POLICY_SIGNALL, DKIM_POLICY_SIGNSOME, and DKIM_POLICY_TESTING.
This will avoid one useless DNS lookup.

if you guys both agree that it would make more sense to have those rules scored
0, I'll +1 ;)

>I would like to revert to only triggering the rule if there is reason to believe
>that the sender actually uses DKIM.

I wouldn't mind seeing DKIM_POLICY_SIGNSOME with a non-zero score if there were
explicit DKIM policies published.  But the existing Mail::DKIM module would need
to distinguish between no published policy and a default policy actually published.

> I would like to revert to only triggering the rule if there is reason
> to believe that the sender actually uses DKIM.

I forgot to comment on that: revert to what? I don't think
this has changed, at least not within the SA::Plugin::DKIM
(and I doubt it has changed in Mail::DKIM).

> I wouldn't mind seeing DKIM_POLICY_SIGNSOME with a non-zero score if there
> were explicit DKIM policies published.  But the existing Mail::DKIM module
> would need to distinguish between no published policy and a default policy
> actually published.

The DKIM_POLICY_SIGNSOME must not distinguish between a defaulted
and an explicit 'sign some' policy. If there is a need for such
a distinction, a new rule with a different name must be devised.
This may be harder to implement than it appears, because fetching
a policy is opaque within a Mail::DKIM perl module, and the
SA plugin has no direct access to this information.

I doubt it would be worth the effort, at least until the SSP
proposal reaches a usable state (which will not happen soon).
Until then the only useful source of information is a
'hardcoded policy' for each sender - in a form of SA rules
to attribute score points to unverified mail from popular
sender domains known to be signing their mail.

(In reply to comment #8)
> I forgot to comment on that: revert to what? I don't think
> this has changed, at least not within the SA::Plugin::DKIM
> (and I doubt it has changed in Mail::DKIM).

When Jason first pulled the code out of dkim-proxy so that we could use it in SA
as Mail::DKIM the module didn't report "sign some" if there was no record that
actually indicated that.  If it had have had its current behaviour I would have
dropped the rule from the plugin from day one.

There's a *huge* difference between *reporting* that a domain signs some and
*treating* a domain as it signs some (the RFC default).  IMHO the Mail::DKIM
plugin is taking the RFC way too literally.  At a minimum it needs to
distinguish between a published and defaulted sign some policy.


> The DKIM_POLICY_SIGNSOME must not distinguish between a defaulted
> and an explicit 'sign some' policy. If there is a need for such
> a distinction, a new rule with a different name must be devised.

We mustn't do anything.  There's nothing wrong with using the rule name
"DKIM_POLICY_SIGNSOME" to indicate that an explicit sign some policy was found.
 The rule description can even say, "an explicit DKIM sign some policy was found".


> This may be harder to implement than it appears, because fetching
> a policy is opaque within a Mail::DKIM perl module, and the
> SA plugin has no direct access to this information.

Yeah, I know, and I really think that this is a short coming of Mail::DKIM.


(In reply to comment #6)
> if you guys both agree that it would make more sense to have those rules scored
> 0, I'll +1 ;)

Yeah, I'm +1 on zeroing the score for the rule.  As it stands now it's useless.


(added Jason to bug CC list)

(In reply to comment #9)
>IMHO the Mail::DKIM
>plugin is taking the RFC way too literally.  At a minimum it needs to
>distinguish between a published and defaulted sign some policy.

I agree some way of distinguishing would be helpful. I'll try to get that in the
next version.
Jason

Thanks Jason!

And here is a third +1 for zeroing the rule. Let's get that done so this can be
closed for 3.2.1

[dos@cyan 3.2]$ svn ci -m "bug 5485: zero score DK/DKIM_POLICY_SIGNSOME rules
since they'll always fire due to defaults (unless there's an explicit SIGNALL
policy)"
Sending        rules/50_scores.cf
Transmitting file data .
Committed revision 544604.
[dos@cyan 3.2]$ cd ../rules/branches/3.2
[dos@cyan 3.2]$ svn ci -m "bug 5485: zero score DK/DKIM_POLICY_SIGNSOME rules
since they'll always fire due to defaults (unless there's an explicit SIGNALL
policy)"
Sending        3.2/50_scores.cf
Transmitting file data .
Committed revision 544605.
[dos@cyan 3.2]$

> zero score DK/DKIM_POLICY_SIGNSOME rules

To turn off a DNS lookup for a policy record, it is necessary
that all DKIM_POLICY_* rules are zeroed, otherwise not much
has been achieved.

For a meager 0.001 score points there is hardly a justification for
an extra DNS lookup for almost every mail (verified mail excluded),
especially since it is done synchronously by Mail::DKIM and not
in parallel with other RBL and similar DNS lookups.

If someone wants to assign more score points to DKIM_POLICY_TESTING
or to DKIM_POLICY_SIGNALL, or to make some metarule out of them,
he is welcome of course (hopefully knowing that it will cost an extra
query, with results that are only applicable to DK and not to DKIM).

(In reply to comment #14)
> > zero score DK/DKIM_POLICY_SIGNSOME rules
> 
> To turn off a DNS lookup for a policy record, it is necessary
> that all DKIM_POLICY_* rules are zeroed, otherwise not much
> has been achieved.

Yeah I'm aware of and have considered that.  What was achieved was preventing
DKIM_POLICY_SIGNSOME from firing on nearly every mail.  It's confusing and
pointless.  My intention is to re-enable it as soon as Mail::DKIM provides a way
to tell the difference between an actual policy and the default.

The other rules, on the other hand, only trigger when there's actually a DKIM
policy in place.  I'd like to have them recorded for re-use in mass-checks. 
Disabling them by default pretty much precludes them from being re-used.


> For a meager 0.001 score points there is hardly a justification for
> an extra DNS lookup for almost every mail (verified mail excluded),
> especially since it is done synchronously by Mail::DKIM and not
> in parallel with other RBL and similar DNS lookups.

If you're looking to save or justify the DNS lookups you might as well zero
score DKIM_VERIFIED and DKIM_SIGNED too.  With scores of +/-0.001 you're not
going to be able to justify them either... or, worse, the CPU time it takes to
verify the signature.

This is why the whitelisting feature of the plugins (SPF/DKIM/DK) was designed
to be usable with all of the DKIM/whichever rules disabled.  If someone finds
that they can only afford to do a DKIM/whichever lookup if it's going to be
really/directly useful to them (as opposed to later potential for re-use in a
scoring run or other rule development) then they can disable all of the rules
and whitelisting will still work.  If they're not going to use the whitelisting
then they might as well just disable the plugin altogether.


> If someone wants to assign more score points to DKIM_POLICY_TESTING
> or to DKIM_POLICY_SIGNALL, or to make some metarule out of them,
> he is welcome of course (hopefully knowing that it will cost an extra
> query, with results that are only applicable to DK and not to DKIM).

Conversely they're just as easily turned off.  I believe that they'll become
more useful when we start publishing rules that expect popular domains to be
signing their mail (it'd be nice to make sure that say eBay is publishing a sign
all policy so that they can unilaterally remove that policy without SA still
expecting eBay to be signing their mail).

> zero score DK/DKIM_POLICY_SIGNSOME rules

Hmmm, is zeroing a score for a rule supposed to disable any such rule?
Somehow I get the impression this works for plain, RBL, URI, etc rules,
but not for plugin eval rules. I still see hits on DKIM_POLICY_SIGNSOME,
BOTNET_IPINHOSTNAME, BOTNET_CLIENT and similar, explicitly reporting 0
points, even though I have their score set to 0.

In Check::run_eval_tests() I don't see any checks on score, like
there are in other do_*_tests routines. I think this used to work
in 3.1.8 as expected.

> Hmmm, is zeroing a score for a rule supposed to disable any such rule?
> Somehow I get the impression this works for plain, RBL, URI, etc rules,
> but not for plugin eval rules.

Opened as a new Bug 5485.

(In reply to comment #17)
> Opened as a new Bug 5485.

That's a circular reference! I think you mean Bug 5519.

> That's a circular reference! I think you mean Bug 5519.

Right, thanks.