|
|
 |
Relentless Spam-Eating Engines in Our Internet Universe
by Michael J. vanZwieten, MCSE, CNA
|
A quick history . . . .
In an Internet galaxy, close, close by . . . “spam” was starting to get out of control.
As you might have guessed, spam – also commonly referred to as unsolicited commercial e-mail, unsolicited bulk e-mail or junk e-mail – is most definitely not new, as the first-ever spam was sent in 1978. Everyone with an e-mail account has at one point in time received a spam. Some of you may have received more than just a couple! In fact, according to statistics from McAfee.com, 7.3 billion spams are sent out each day across the world; that number is expected to reach around 14.5 billion per day by 2006.
About a decade ago, e-mail server administrators started noticing that an extraordinarily high number of spam e-mails were originating from open relays. An “open relay” is an e-mail server that is configured to relay mail from any origination address to any destination address. Open relays typically are misconfigured e-mail servers, either due to a mistake made during the actual server configuration or due to a lack of knowledge on how to properly secure an e-mail server. Spammers typically search the Internet far and wide for such open relays. Then they send out hundreds of thousands, even millions, of spams using someone else’s equipment and bandwidth.
|
|
A “closed relay,” on the other hand, is an e-mail server that is configured to only accept mail based on certain criteria, such as the destination address or the originating IP address. This would prevent a spammer from relaying e-mail through the e-mail server because it didn’t originate from the e-mail server itself.
Citing whitepapers from both Philip Jacob and Philip Howard (“The Spam Problem: Moving Beyond RBLs”), e-mail server administrators were getting fed up with the spam problem, and started blocking inbound e-mail from these open relays as they noticed abuse on an ever-increasing basis. On occasion, these administrators would send a test message to themselves via the open relay that they had just discovered. If they happened to receive the test message back in their mailbox, they could deduce with a pretty high level of certainty that the system in question was an open relay, since they were basically able to send messages through it to anybody in the world. Sometimes, the e-mail administrator would contact the operator of the open relay and ask him or her to close it, in order to cut down the amount of spam emanating from their server. If the relay operator refused to comply or did not respond to the request in an appropriate time frame, the e-mail administrator would simply blacklist the server in question as a known source of spam run by an “irresponsible” manager. The server would stay on this blacklist either until the administrator retested it at a later time and found it to be closed, or until the “irresponsible” manager finally came to his or her senses.
|
|
This method of spam blocking was fine and good; however, the amount of work to maintain these lists became burdensome, mostly because the e-mail administrators had to manually share these lists with each other. The next step in the evolution of blackholes was a drastic one! The e-mail administrators decided to create a distributed and automated system for sharing these lists using a DNS (Domain Name Server) model. This type of system is called an RBL (Realtime Blackhole List), or “Blackhole” for short. The reason it was nicknamed a “Blackhole” is because the e-mail emanating from e-mail servers that were marked on this Blackhole list would disappear into nothingness on the e-mail servers that used the list. As people reported the IP addresses of open relays that were possibly, or most definitely, being used to send out spam messages, the RBL system would initiate a testing process. A test e-mail message would be sent to itself via the open relay on the IP address in question. If the RBL system received the message back, the IP address in question would automatically be added to the blacklist.
The number of RBLs has grown tremendously from a decade ago, but the systems differ in many ways, including the procedures in which open relays or hosts are blacklisted, list content that is targeted, policies for being removed from the blacklist, RBL technology, Web site/RBL lookup performance, server redundancy and overall accuracy. People have been known to start creating their own custom RBLs in order to fight their own spam problems, while openly sharing them with anyone who would want to use their list. Others started making a business out of this type of service so that other businesses would be able to cut down on their spam problems.
In order to block spam using RBLs, an e-mail server must have the capability of doing RBL lookups. Currently, many e-mail servers have this sort of functionality built in; however, if they do not, there are e-mail firewall packages available that run in front of your e-mail server and have this capability.
|
|
The Different Types of RBLs
It is important to understand that there are different kinds of RBLs available and most are free of charge. Some RBLs primarily focus on listing open relays, while others may focus on listing all the IP ranges that ISPs use for their dial-up customers.
ISP dial-up customers typically have no need to set up an e-mail server in order to send/receive e-mail, since that activity is usually done using their e-mail client through the ISP’s e-mail servers. A great deal of spam tends to originate from these ISPs’ dial-up ranges because many spammers create temporary accounts and bulk-mail their spam from there.
Other RBLs focus on listing host IP addresses from which spam directly originates, whether they’re a rogue server at some spammer-friendly ISP, or even the entire address block of a rogue ISP itself. Some RBLs aggressively block entire networks, or the entire IP address ranges within countries such as South Korea or Croatia.
|
|
What RBLs Try To Accomplish
RBLs were originally created to “educate” the ISPs, and forcefully show them why they should not be harboring spammers, or allowing spam to relay through their networks. An RBL forces ISPs, companies, cities and other organizations to take an active stance against spam, either by booting spammers off their servers, locking down their own servers and preventing open relays from occurring, or paying the consequences of not being able to send e-mail to various parties.
Today, RBLs still are used for very much the same purposes, but are also being used to block many different sources of unwanted e-mail, regardless of whether they are spammers or not. For example, some companies who only have a very local customer base can choose to block all e-mail coming from the entire Asian region using RBLs, which knocks out a considerable percentage of spam originating from that part of the world. Others, who may want to block e-mail from a specific ISP can use RBLs to block the entire IP address ranges that an ISP might own.
|
|
The Pros and Cons of Using an RBL
More and more organizations are starting to use RBLs, which do have some drawbacks, as you will read below. However, it still is believed that these drawbacks are outweighed by the benefits. Some RBLs are overly aggressive when adding addresses to the lists. For example, if a large city has a single misconfigured server that ends up being an open relay, which then gets placed on a blacklist, the entire mail domain of this city can be blocked, even if the city is entirely innocent of spam activity. Also, if a city’s ISP is being blacklisted for harboring spammers, and the ISP is not budging to kick the spammers off, the city’s IP range might be included in this blacklist as well. Again, this can block the entire mail domain of the city even if it is not running any open relays. Another drawback is that some of the smaller rogue RBLs make it very difficult to get off their blacklists. If you are placed on it, it could take up to six months or more to get off of it. Fortunately, not many organizations are using these smaller rogue RBLs to filter their e-mail.
According to Kym Gilhooly from Computerworld.com, good RBLs share a number of important traits. First, they establish and maintain a consistent set of criteria for putting an IP address on the list. Second, they rigorously test suspected open relays again and again to verify the integrity of their shared databases. And third, they provide an easier process for domains to notify the RBL operators that they have been mistakenly been put on the list so they can be removed from it.
|
|
Should Our City Use an RBL To Cut Down on Spam?
Here are a couple of points to consider before using RBLs:
RBL = Voluntary! – Most importantly, using an RBL is strictly voluntary. By not using an RBL, all e-mail will flow freely to and from your e-mail server, spam and all. Using one or more RBLs could cut down your amount of spam by up to 50 percent or more.
Whitelist – It is imperative that you have the capability to “whitelist” an IP address of an e-mail server. A whitelist is the opposite of a blacklist, and allows the contact’s e-mail to come through with no questions asked. Communication at times is absolutely imperative between entities. So is a whitelist.
Responsibility – Be prepared that some of the entities you deal with on a daily basis may get blacklisted at any time. Make sure you have staff/resources available to deal with these entities and possibly assist them through their struggle to secure their e-mail servers, give advice on how to contact their ISP or how to get off the blacklist, and be able to whitelist them if necessary. Creating a Web page with information regarding your spam policy is a helpful resource for those who get blacklisted (e.g.: www.flcities.com/spam). Also, support staff should be available to field questions from internal city staff if they are having issues sending/receiving e-mail to the various entities.
ISP Selection – You (or an entity you communicate with) might be placed on a blacklist based on the ISP you use. Some ISPs are spammer-friendly, while others have very strict policies against spammers and make sure that none of their customers fit this category.
|
|
Complementing the Use of RBLs
Using an RBL is not your only method of defense to cut down on spam. RBLs should be used in conjunction with other filtering methods to increase the spam-blocking ratio. There are several other methods:
Keyword Filtering – This filtering method looks for specific keywords in the message body, such as “click here to unsubscribe” or “Viagra.” Normally, e-mails flagged with keywords in them are quarantined, because there is a pretty good likelihood that some of these e-mails are false positives.
E-Mail Header Filtering – This filtering method looks at the consistency of the e-mail header, i.e. the history of how the e-mail ended up at your e-mail server, and looks for any forging of e-mail headers.
Bayesian Filtering – This filtering technique uses a statistical method to determine whether an e-mail is a spam or not. Still a very new method, this technique has to learn what good e-mail looks like, and what spam looks like. The more it learns, the more accurate it gets. This technique, when working as advertised, can filter as much as 99.5 percent of spam with zero false positives. Of all spam-filtering methods, this method alone could eliminate the need to use RBLs or any other filtering technique.
Heuristic Filtering – This filtering technique assigns scores to various keywords and phrases within the e-mail, then tallies up a final score to determine the likelihood of whether the e-mail is legitimate or a spam. |
|
Validating Message Sender – A function performed as soon as the e-mail arrives to determine if a particular e-mail coming from someone@aol.com actually corresponds with the IP address of the e-mail server that sent it.
Common Spam Technique Filtering – Many anti-spam products make use of special filters to quarantine e-mail that looks suspicious and that goes along the lines of current spam techniques. Some examples are spams that use external Web sites to construct the body of an e-mail, or e-mails that have HTML-based null comment tags in them that make the word “spam” actually appear as “spam” on your screen. Null comment tags typically are ignored by your Web browser or HTML-enabled e-mail application, such as Outlook or Outlook Express. When the e-mail comes in, the spam filter just passes by the word “spam”, and is not able to deduce that it actually says “spam.” Many filters nowadays are able to deal with this null comment tags issue.
Internal Domain and IP Blacklists – Spam that gets quarantined by your other spam filters that may not have gotten caught by the RBL filters could get added to your internal domain or IP blacklists. This is kind of like building your own little RBL, consisting of known spammer IP addresses or domain names.
Multiple RBL Selections – If you start using RBLs, it is a good idea to use a variety of the various different types of RBLs available. A combination of using open relay, dial-up range, host blocks and ISP block RBLs will yield a much higher spam-blocking ratio than using only a bunch of open relay RBLs, for instance.
|
|
In Conclusion
RBLs are an effective way to reduce the amount of spam your city receives. Although some RBLs can be fairly aggressive with the hosts they add to their lists, many are very fair and are run by professionals. Although the benefits to using RBLs seem to outweigh the drawbacks, some false positives are possible. It is important that your city takes the necessary precautions before starting to use RBLs, such as having the capability of using a whitelist, and having technical staff ready to handle any possible e-mail communication problems between your city staff and the external entities you deal with on a daily basis. If spam seems to be out of control at your city, implementing the use one or more RBLs will make a considerable dent in the number of spam e-mails received. Many of the larger ISPs, such as AOL, MSN and Hotmail, are starting to use RBLs along with other sophisticated filtering techniques to cut down on the spam going to their customers. The use of RBLs is not only highly encouraged, but almost necessary nowadays in the war against spam!
Michael J. van Zwieten, MCSE, CNA, is assistant director of the Florida League of Cities’ Department of Technology Services. For more information, he can be contacted via e-mail or at (407) 425-9142.
|
|
|
|
|
|
|
