Major Description
Email deliverability is a tricky topic and yesterday I got one more example for this...
(The trick is to de able to improve deliverability while still being secure)
The issue
The overall scenario outline is:
GIVEN two local users alice@me.org and bob@me.org on a james based instalation AND alice@external.org on another email supplier AND alice@external.org forwards its mail to alice@me.org. AND this server preserves and resend as is the SMTP transport envelope WHEN bobn@me.org sends an email to alice@external.org THEN james server rejects the incoming mail from external.org as it requires authentication to use local addresses in MAIL FROM AND a bounce would be generated by external.org for the original sender as the email could not be forwarded.
Example of such "external mail providers": orange.fr SFR professional services.
See fwd1.png
Why authentication is currently required in such cases
In order to maximise delivrability, James accepts any email for local users on port 25 (spam filtering may be applied as part of the reception chain)
However a common attack is email spoofing: a remote attacker with not whatsoever access sends an email pretending to be a local user. Without further checks he could submit email as a local user without being authenticated.
To prevent this behaviour James implements `verifySenderIdentity` configuration parameter in smtpServer.xml configuration file. If using a local mail from the current behaviour is to actualy mandate authentication.
Anybody else than the user could thus NOT be using this MAIL FROM.
This prevents the following schario: fwd2.png
However this causes the issue discussed above.
Related deliverability concerns
Receiving external forwards orginating from external mail servers
bob@external1.org sends a mail to alice@external2.org which forwards mails as described above to alice@me.org.
On the last hop james accepts the email as the MAIL FROM (bob@external1.org) is a remote one: verifySenderIdentity is not triggered for external senders.
On the spam filtering side:
- SPF checks fails (as external2.org did send a mail in external1.org name which it is not allowed to do
- But DKIM tests pass
- Which should be enough for the spam filtering stack not to flag the message too violently...
This current behaviour is considered lenient enough. White listing could further be added for trusted domains if need be.
Handling correctly forwards emition
We tackled the very same issue on the emition side: the goal was to improve james forwarded email deliverability.
GIVEN bob@me.org forwards its email to bob@external.org
WHEN toto@other.org sends a mail to bob@me.org
THEN me.org would rewrite the MAIL FROM field and use bob@me.org when forwarding to bob@external.org
Thus SPF check passes.
See https://issues.apache.org/jira/browse/JAMES-3944
Proposed solution
We could experiment turning off sender verification hop on MAIL FROM
Instead we would verify the DKIM signature of the email before spooling it.
That way:
- We would have signed legitimate traffic forwarded back to us and would accept it
- And we would still reject spoofing attempts as an attacker cant craft a DKIM signature.
GIVEN that dkim signature of outgoing email is not setted up by default this behaviour shall be disabled by default.
Proposed changes
Implement a VerifySenderIdentity SMTP message hook that:
GIVEN an unauthenticated sender
It performs a verification of non relaying (ie all rcpts are local)
And would verify the DKIM signature of the mail.
Path to the DKIM public key needs to be provided.
This shall be contributed as part of the DKIM extention.
Additional concerns
Given the Mail Exchange service (MX) and Mail Submission Agent (MSA) being collocated - the default james architecture,
Then a common misconfiguration is not to configure TLS for SMTP and just have regular users just try to send mails unauthenticated through SMTP.
This behaviour is classical from MacOS native client.
Current `verifySenderIdentity` behavior rejects such misconfigured client from the MAIL FROM. Which is convenient. Fail fast:
- The error is more explicit: AUthentication required
- No message payload had been transfered.
Ideally we would like to preserve this behaviour.
The way to achieve this would be to have a "relaxed" mode to verifySenderIdentity instead of "strict" and "disabled"
This relaxed mode would enforce the current behaviour if it determise the other party to be a MX. We can base ourselves on the structure of the EHLO sent: is this a valid domain?
- If the heuristic determines this is a MX we accept the mail that would be dkim tested
- if the heuristic fails, we reject the mail from this step as today as non MX clients are misconfigured local users.
Note that classifying a local user (heuristic false positive) only delay the inevitable error: acceptable, and a heuristic false negative is no worse than the current issue described.
The configuration of verifySenderIdentity would then accept the following value:
- true. Unchanged existing option. Verify strictly the sender identity. Same as strict.
- false. Unchanged existing option. Do not verify sender identity at all. Same as disabled.
- strict. Verify strictly the sender identity. Same as true.
- disabled. Verify strictly the sender identity. Same as false.
- relaxed. Verify strictly the sender identity if the remote party is determined to not be a MX. Same as false. Further checks are required, like inspecting DKIM signing, in order to prevent spoofing.
