My 2/01/09 Missoulian column
Last week, I covered how e-mail protocols – Internet mail access protocol, post office protocol and Web-based e-mail – work “under the hood.” I also mentioned simple mail transfer protocol, or SMTP, which is the way e-mail gets sent. There’s a long electronic dialog that happens when you receive e-mail with those protocols or send an e-mail with SMTP, and the details of that conversation have much to do with understanding how other parts of the Internet work.
Protocols are important concepts: They are modes of computer and network “speech” with explicit rules that all computers must adhere to when they talk to one another. If one computer is set to speak a different protocol, it won’t be able to talk to any other computers on the network if they expect to talk in a protocol common to all.
And protocols – by the definition of the word – are standardized to ensure there is nothing left to chance. Everything that happens is explicitly instructed in the same language and in the same electronic “grammar.”
If you set up your e-mail program for IMAP and the mail server at your Internet service provider only speaks POP, you’re out of luck. You won’t be getting e-mail anytime soon and you will get different “can’t contact to the mail server” error messages that those protocols use to say something is wrong.
Protocols are more strict than plain languages. We can say “Hey, send my e-mail,” or, “Dude! Send!,” and we know what we mean. But SMTP has never heard – and probably will never understand – the word “Dude,” so your e-mail program has to say “Send my e-mail” in a mutually understandable way.
SMTP works this way: you open a new e-mail, address it, and fill in the “subject” line and maybe the “cc:” or “bcc:” lines as well (which stand for “carbon copy” and “blind carbon copy,” holdover terms from the days way before e-mail.) Then you type your message, attach a file, cut and paste Web links, etc. When it’s ready to go, and you click “send.” Only then does SMTP come into play.
What happens is your e-mail program says, “Hey, it’s time to send some e-mail.” And it starts a dialog with the SMTP server at your ISP based on the exacting SMTP protocol. I say dialog because it really is a kind of talk back and forth.
Here’s a paraphrase of the important parts of the process:
A split second after you click “send,” your e-mail program says, “Hey, SMTP server, are you there?” And it answers, “Yeah, I’m here.”
And it goes on from there: “OK, are you ready to receive an e-mail?” And if the reply is “Yes, I’m ready,” they go on with the transfer.
But sometimes the server answers, “No, I’m busy. Hang on for a few seconds.”
“OK, I’ll wait, too,” your e-mail program says.
Then they’ll try again:
“OK, are you ready?”
“Yeah, I’m ready.”
“OK, here’s an e-mail” … from so and so.
“OK, got that.”
“Ready for the e-mail recipient?”
“Yeah, I’m ready.”
“OK, here it is.”
“OK, got that.”
“Ready for the text?”
“I’m ready. Send it.”
“OK, sending …”
And when your e-mail program finds that it has reached the end of the e-mail, it tells the SMTP server that it’s done, and says it is hanging up without further ado. The server says “OK, we’re done,” and that’s that.
The dialog isn’t very complicated. It’s a step by step communication at a simple and precise level. Computers are like that. Everything has to be exactly outlined or you’re not going to get the expected result.
You might think of an e-mail protocol as someone who is trying to exactly describe how to do something to someone else by using mutually identical vocabulary and grammar to be perfectly clear. That’s what SMTP does. It gets e-mail sent as fast and efficiently as possible, even if the conversation is long and boring.
The SMTP server actually does more things under the hood. It sends you those cryptic error messages you get when you send an e-mail with an address that’s not valid, or when another server is off-line, or your e-mail has been bounced back because the server thinks it’s spam.
Another important thing to know about SMTP is that when you send an e-mail, you don’t need your own “phone line” or “pipe” to the mail server or the Internet as a whole.
It might seem that way with all of the back and forth conversation that goes on, but it’s not required. And if it was, the Internet as whole simply couldn’t exist as it does today. Too many separate “lines” would be required for the hundreds of billions of e-mails that are sent each year, not to mention all the Web traffic and other types of communication that goes on.
With SMTP, your e-mail program opens up communication with the mail server only when it needs to talks to the server, and then drops the connection when it’s done. So with SMTP, all the back and forth involved in sending an e-mail can take place with one connection over a second or two, if both your mail program and the SMTP server have the time to handle it.
But sometimes there will be idle time with either the server or your e-mail program waiting to reconnect and complete the e-mail transfer. It’s the same with receiving e-mail with POP or IMAP: There is an extensive back and forth conversation concerning things like, “Hey, did you already download this e-mail?” and, “Refresh the message list and don’t show the messages I just deleted,” and on and on.
So in the seconds after you click the “send” button and watch the cursor on the screen spin, or the sand in hourglass slip through, or the beach ball roll around, that’s what is happening: “Hello, SMTP server, are you there?” “Yeah, I’m here …
Mail servers field hundreds or thousands of requests a second, and sometimes a server will crash when it gets too many requests. I suppose we can imagine the mail server with a slightly cranky voice.
It might feel overloaded, always having to say, “Yeah, I’m ready …,” while your e-mail program is always ready and waiting. But SMTP is just doing its job, and while your e-mail might have to to wait its turn, it will get sent.
The different e-mail protocols exist for a reason, and without them, e-mail – as well as the rest of the Internet – just wouldn’t be speaking the same language and we’d all be out of luck.