Tackling Spam
The next step was to figure out what do with the spam itself. A team was formed and real-time black lists
(RBL's) were researched extensively. A set of 14 reliable lists was produced. The black lists use the
IP address of a mail server and simply report on the IP's recent spam activity to fake "seed" addresses.
A new program was created to sort mail by performing these 14 black list tests on every email before
delivery to the real mail servers.
If we were going to significantly cut volume to the real servers, the flagged messages would have to be held
rather than delivered. A management system was created for support so that the held messages could be searched
for any missing email reported by customers and released. This support system was also given the transaction
logs on all email so that support personnel could positively identify failures outside our own system, which
were frequent with other ISP's buried in the same struggles. Months of use proved that our black lists were
nearly 100% reliable and the support system's primary use became researching mail problems in other companies.
Although we had managed to positively identify a large volume of spam, we were still far from success. Our own
inboxes had shrunk from thousands of messages to hundreds of messages, but were still fairly unmanageable. The
next phase of the project was to move something we had been doing in our email clients to the servers. We knew
who we were trying to receive email from because they were in our address book. We had all set up filters in
our email clients to move known senders off into separate folders so we could find them. Combined with
observations of instant messaging systems, we realized that the key to stopping spam was to stop caring about
identifying the spam itself and start caring about identifying the spammers.
By definition, this turned out to be just about anyone not in our address book. The only question that remained
was how do you get to know new addresses? This is an easy answer. If we sent an email to an address, it was
reasonable that we would want email back from the same address. If we spotted a message in our email client's
inbox that hadn't been moved to a known sender's folder, we just needed to add them to our address book. Every
single other piece of email from any other sender was absolute garbage. We knew that we could free ourselves
from the spam if we just made the mail servers perform these tasks for us.
[ Next: Sender Authentication - The Final Frontier ]
|