Outlook/OE fail to authenticate on port 465/587

[Matt Simerson writes:]

M$ strangeness. Today a client called because he couldn't send email any longer. After troubleshooting for a few minutes and verifying his settings, I configured OE on XP and was able to replicate the problem. Essentially what happens is that OE fails to send outgoing mails (via port 587) when SSL is enabled. I am fairly certain that this used to work.

Using these very same servers, Mail.app (Mac OS X) and Thunderbird both work just fine. In Mail.app you enable the SSL checkbox and it negotiates TLS encryption for the connection. In Thunderbird you must select TLS as the encryption method, and it too works just fine. OE does not. :-(

With all the changes to my servers lately, I thought possibly this was a problem of my own making. So I configured an account on my old deprecated mail server, which is still housing one lonely email client and OE won't send via that server either unless the SSL checkbox is disabled. I haven't touched that server in a long, long time.

Thinking possibly this was a SSL certificate issue, I downloaded the Root Certificates update in XP but that didn't solve it either. Finally, I configured stunnel on port 465. It negotiates a SSL connection and tunnels it to port 587 on the real server. I pointed OE at that and now it works. Apparently somewhere along the way, something in OE changed. Maybe I got a new version with IE7 that made the difference? Has anyone else noticed this?

Since I was messing with Thunderbird, I added a How to configure Thunderbird page

[Tor Willy Austerslått replied:]

I noticed the same thing after migrating about 300 users from a very old toaster (version 2-something) to a 5.0x-toaster. This was in early February. After the hostname started resolving to the new box, some OE users couldn't submit mail on port 587 with SSL enabled.

It was a known problem in OE6 on port 25: if you advertise both AUTH and STARTTLS to it, it will ignore STARTTLS and try to authenticate without encryption. The end result is that OE6 tries to honour its own settings and use an encrypted session, but no encypted session was ever started by the server. Either that, or the server expects OE to do a STARTTLS before doing anything else. Thus, it bombs out with an error message. OE can in any case only do STARTTLS on port 25.

If you set OE up to do SMTP on anything but port 25, it expects to do SMTPS. The trouble with SMTPS is that it *used* to happen on port 465, but IANA has for the last year or so reserved port 465 for a different protocol. Outlook (from Office) 2003 and newer supposedly has no problems.

So what's a poor mailadmin supposed to do when everybody and their uncle wants to use clients that are broken by design?

Also see: http://www.tu-harburg.de/rzt/mailing/inkompatibel.shtml

How to run qmail-smtpd on port 465

In general, I don't recommend running smtps on port 465, but if you need to, it is very easy to do. Simply shut down your supervised services, duplicate the smtp-submssion supervised directory, and edit the run file, changing the port from "submission" to "smtps" and setting the SMTPS variable. Then symlink your new smtps directory into your services directory, start up your services and test.

cd /var/qmail/supervise
services stop && cp -R submit smtps && services start
vi smtps/run smtps/log/run **
ln -s /var/qmail/supervise/smtps /service

And then test.

** alterations required

% diff -u /service/submit/run /service/smtps/run
--- /service/submit/run	Fri Mar 16 21:13:59 2007
+++ /service/smtps/run	Fri Mar  2 13:35:54 2007
@@ -8,11 +8,11 @@
 PATH=/var/qmail/bin:/usr/local/vpopmail/bin:/usr/local/bin:/usr/bin:/bin
 export PATH

+export SMTPS=1
-export NOBADHELO=""

if [ ! -f /var/qmail/control/rcpthosts ]; then
	echo "No /var/qmail/control/rcpthosts!"
	echo "Refusing to start SMTP listener because it'll create an open relay"
	exit 1
fi
-exec softlimit -m 25600000 tcpserver ... -u 89 -g 89 0 submission qmail-smtpd ...
+exec softlimit -m 25600000 tcpserver ... -u 89 -g 89 0 smtps qmail-smtpd ...