Email is unreliable. So should we face it or fix it?

Despite what Dilbert Comic Strips may teach you, our job as security professional is to enable information services – not prevent them.

The bad guys do evil: we try to prevent it (or clean-up after) so that users can continue and use systems as if there is no evil in the world. If IT security had a Hippocratic oath, it would probably be along those lines.

Here’s a recent example. This morning I got a call from my credit card company asking me if I’d done some transactions that seem suspicious. I hadn’t, and so they will cancel the transactions (and unfortunately, cancel my credit card and send me a new one). I’m not going to stop using my credit card, and will probably completely forget about this incident. I didn’t lose any money, and the inconvenience was minimal: this is all thanks to the people that chase up the credit card fraud and enable customers around the world to use their cards despite countless attacks on credit card users, some (as my example shows) successful.

Things are not so simple in the email war front. When SMTP was introduced, it described a simple, reliable, scalable system for communication. Almost 30 years after that, we stripped email of some of its most important features. By we, I mean the IT security world. In fact, we’re slowly doing to SMTP what TSA is doing to air travel.

First, the major feature of SMTP: sending and receiving emails. This is probably our biggest failure today: There is no guarantee you will be able to send or receive emails. In fact, if you communicate with the external world, it is almost guaranteed that you will not receive a certain percentage of your emails, and that some emails you send will not arrive. Sure, there are legitimate reasons: we need to protect from spam, viruses and phishing. But the bottom line is that SMTP was designed to reliably deliver an email from point A to point B. Today, we send an email and then call to verify it was received (or send a second email which mysteriously arrives after the first one was blocked).

Next, we kill useful SMTP features. Remember the days when you got an email ‘bounce’ when mistyping the email recipient’s address? Forget about it; those days are long gone. I’m not sure what Spamcop’s exact mission statement is, but it might as well be “make email unuseful”. They have outlawed email bounces (which, by the way, are required by the SMTP RFC) and continued to take out all auto-responders.

Remember read-receipt? Gone. The postal service had this feature in 1841, but we can’t have it in 2010. Do you want to know if a certain email exists? You can’t.  Want to send email directly from your computer without using a mail relay? A non-starter. Ever heard of email fragmentation? This is an awesome feature of SMTP but don’t waste time learning it – it won’t work on the Internet today (and this time we share some of the blame).

Look at HTTP. You click on a link, and you get to the page. If you get an error, you know it’s the web site’s fault. An attack on NCSA’s httpd server is one of the first documented buffer overflow attacks, and yet attacks on modern HTTP servers are practically non-existent. SQL injection and XSS are everywhere and yet users surf dynamic pages all the time without being blocked. We’re doing a good job fixing up HTTP without being a “Mordac”. Too bad we couldn’t do it with SMTP.

Is there hope for SMTP? I think there is. Last decade the doctors were ready to pull the plug on email: spam and viruses were so frequently in the users’ inbox that email was on the verge of being unusable: You had to spent a noticeable percentage of your day clicking the ‘del’ button. These days are over: you rarely see spam in your inbox today, and if you’re like me, you get more irritating chain letters from family members you can’t block (hi mom) than shady ads for pills.

This war can be won. We just need to remember the Hippocratic oath for the IT security world and enable reliable communication again.

  • graemel

    Yes, SMTP is a mess and needs a ground up re-design, but your attack on Spamcop is factually incorrect. They do not outlaw either bounces or auto-responders, but badly configured implementations of them.

    Spamcop have no objections to bounces that occur during the SMTP transaction, where the receiving server responds with a 4xx or 5xx code and then the sending server notifies its user that the mail was not sent.

    What they do object to are badly configured servers which end the SMTP transaction with a 200 code and then at a later time, decide that they’re not going to deliver the mail and send a rejection to the envelope sender of the original mail.

    You’ve obviously never been bombarded with 100,000 delayed bounces from a server accepting deliveries from a spammer conducting a dictionary attack and then sending bounces at a later time to your address with was forged as the sender. Up to 95% of email is spam and spammers *always* forge the envelope sender address. If you’re doing delayed bounces, all you’re doing is reflecting your spam problem onto an innocent 3rd party.

    The same applies to auto-responders/Out Of Office Replies. If they’re rate limited and/or address book limited, they’re not a problem and don’t get objections. If they’re insecurely implemented, all they do is reflect spam attacks to innocent 3rd parties.

    Good luck with re-designing the email protocol, it’s not an easy problem to solve. However, please ensure that you build it in such a way that you don’t reflect your spam to innocent users.

  • Assaf

    It might be just a matter of time until email is completely outdated and all you’ll have are facebook messages and twitter DMs and the like. It’ll all be done with various http based APIs and through service providers that expose a generic interface to facebook/gmail/twitter, etc. So just give it time and SMTP will solve its own problems by simply being forgotten.

  • Aviram

    @graemel: I don’t think I am factually incorrect at all.

    “What they do object to are badly configured servers which end the SMTP transaction with a 200 code and then at a later time, decide that they’re not going to deliver the mail and send a rejection to the envelope sender of the original mail.”

    I believe you’ve just defined an auto-responder.

    And yes, I’ve been bombarded with 10,000′s of OoO emails (try posting to a large mailing list like bugtraq). It’s not fun. But if the answer is disabling a useful feature that’s widely used, we failed.

  • Aviram

    @Assaf: Good point.

  • Alvin securiteamdotcom

    You must have gone to a different networking school than I did.

    Several times you used the word ‘RELIABLE’ in reference to email.


    The actual technical definition of email in networking terminology is UNRELIABLE.


    You also discus the delivery receipt mechanism. That mechanism was not originally intended for use outside an organization. It was designed to function as an in house mechanism.