It appears that implicit SMTP (SSL) is not supported. Rather only explicit SMTP (SSL) is supported.
This is due to the fact that log4net delegates mail to the System.Net.Mail.SmtpClient object and Microsoft doesn't support it.
The bug request is to fix the smtp appender to support both flavors.
Marked as bug instead of improvement since some providers only support port 465 (SMTP/SSL).
In my case, using port 465 with EnableSSL. This is why my send timed out. It was waiting for the server to respond to it’s STARTTLS command, while the server was waiting for the client to start a secure sockets connection.
Differences between implicit and explicit SMTP SSL:
With Explicit SSL, the communication (which basically is a sockets connection) starts unencrypted on port 25 or 587 as a regular SMTP conversation, then switches to TLS (encrypted channel) using the SMTP STARTTLS command, after which it authenticates and starts sending the email. This is the kind of SSL that the .NET SmtpClient understands, and the only kind it understands. It is described in this RFC (which actually talks about FTP, but the thing at issue here is the actual SSL connection, not the protocol – FTP or SMTP – running on top of it).
With Implicit SSL, the connection starts out the whole conversation over SSL, i.e. it is encrypted from the get go. Commonly, port 465 is used for this. Implicit SSL is NOT covered by any RFC, it is NOT a standard, and the .NET SmtpClient does NOT understand it.