A brief history
I have been setting up Internet email access for organizations since the 1980’s. Back then when I first started using email, the Internet community was comprised of universities, large companies, and government agencies, including the military, as the whole thing started as a defense project, It was a tiny community compared to now.
Many of the basics of how email is transmitted is still the same. The protocol used to send messages between computers, Simple Mail Transfer Protocol (SMTP) is still in use today. It would be a few more years before MIME, or Multipurpose Internet Mail Extensions, would be developed. So, sending photos and other media was not supported though there were ways to do so which were complicated and of course, really slow.
From the late 80’s and through the 90’s, my various job roles (being a typical techie, they changed often) often entailed configuring Unix servers for various organizations. Before the world rushed in during the mid 90s, configuring email on a server was a relatively simple affair. The word spam still meant that gelatinous meat product.
Without spammers and fewer hackers around, configuring email was a matter of specifying the various parameters required by the SMTP protocol. The hardest part was that many organizations, though they would register a domain for the purpose of being able send and receive email, were not actually directly connected to the Internet. There was another protocol, UUCP (Unix to Unix Copy) that was used over dialup modem connections. To transfer email, such organizations would relay their email to another organization that was connected directly. Sometimes there could be two or three UUCP “hops” between one’s local server and the Internet proper. Anyway, there was a program that is still around but less used, sendmail, that had to be configured so that the machine could properly route email via those hops. It often took a couple of hours of cussing and fussing to get it right.
Now that everyone is directly connected more or less to the Internet, the transfer aspects of the configuration are quite easy. All of the complexity we have now is related to security—preventing malware infections, filtering spam, and foiling direct hacking. For this discussion of explaining the basics, I won’t go into the details of all that is required. I can say that setting up a reliable email server capable of securely routing email for an organization is something that has become so complicated and difficult that only organizations with deep pockets should attempt to take it on. Over the last ten years I have set up email servers for the purpose of trying to provide email service for my clients. I divested of my last attempt several months ago. Because of all of the security issues with which to contend, it is just too labor intensive.
How it works
At a fundamental level, to understand how email works, all you really need to know is how snail mail works. When you send a letter via USPS, you put the recipients address on the envelop and then your return address. The return address is important so that the recipient has an address to which to replay and so that the post office can return the message if it cannot be delivered. Electronic mail duplicates these functions exactly.
When you send an email from your computer or from your web mail account (gmail.com, for example), your message is put into a virtual envelop with the sender and recipient info on the outside. The first thing the computer must too in order to send message is inspect the recipient address for the destination domain and then determine how to route the message to that domain.
The mechanism it uses to determine how to do that is the Domain Name Service, or DNS (See a How to Register a Domain for a discussion of how DNS works). Every registered domain on the Internet has a DNS “database” that contains the addresses of the servers the respective organization has provisioned to send and receive Internet traffic. There is an “A” record that contains the IP address to which web traffic should be routed. The record required to route email is a mail exchange or MX record. To send a message, the computer, or more specifically the program called the “Mail Transfer Agent” finds the MX record for the recipient’s domain. Here is an MX record for tnotw.com.
tnotw.com. 86400 IN MX 20 mailsec.protonmail.ch.
The last part of that record, mailsec.protonmail.ch. is what the MTA needs to route the messuage. It knows that to route mail to the domain tnotw.com, it must relay the message to the host that is named mailsec.protonmail.ch.
The next step to send the message is for the MTA to establish an SMTP connection with mailsec.protonmail.ch. To do that it will do an A record lookup on mailsec.protonmail.ch to get its IP address (176.119.200.129). With that it will establish a connection with the MTA at that address to transfer the message.
Since I am using Proton Mail as my email provider (hence letting them do all the work of setting up a secure mail server), the MTA at mailsec.protonmail.ch will put the message in my inbox. If the message was addressed to a user that does not exist in the tnotw.com, the MTA would bounce the message back to sending MTA which would then put a bounced mail message in the sender’s inbox.
With the mail message placed in the inbox on proton.me, I can log in and use the proton.me “webmail” user interface to read the message, reply to it, or compose and send other messages.
If I don’t want to log into my email provider every time I want to check my mail, I can configure an email client (mail user agent or MUA) on my laptop, phone, or Commodore 64 (just kidding) to retrieve the message from proton.me. That allows me to process all of my email locally on my device, which is usually faster and easier.
There are two protocols that are used to retrieve messages from a mail server. One is post office protocol (POP) and the other is Internet Message Access Protocol (IMAP).
POP has mostly been obsoleted by IMAP. POP works by simply downloading all messages not previously downloaded into a local folder. Email clients can be configured to remove the messages downloaded from the POP server or not.
IMAP works a little differently in that it provides a view of your mail folders that are physically resident on the IMAP server. When you view your inbox or other folders, the message subjects and senders are listed. When you click on an individual message to read it, that is when the message gets downloaded for you to view. But, a copy is always kept on the server. When messages are deleted (moved to Trash), or moved to other folders, the folders on the server are what actually changes.
So, the big advantage that IMAP has over POP is that you won’t lose all of your mail if you lose your computer. It also makes it possible to access your email from different devices. It is what allows you to check your mail from either your phone or your laptop.
This article describes the basic mechanics of email routing. As mentioned above, the presence of spammers and hackers has turned something that is relatively simple in concept into a rather jumbled soup of acronyms-SPF, DMARC, DKIM etc which I hope to cover in future articles.
If you are having problems with your current email provider, or you are trying to get a new organization on-line, we have been at this for a while and can probably help. Request and consultation.