Should I use RBLs? Are they effective?

[matt]

Grab a cut of coffee, it's SMTP lesson time.

The type of statistics you'll see in your logs regarding SMTP blocks will reflect several things of interest (that are related to your question). The first, and most obvious is how you are using the RBLs.

The most important setting in that regard is rbl_enable_soft_failure in your toaster-watcher.conf file.  If you have soft failure enabled, then when you deny a mail agent access, you are doing so with a temporary failure, which amounts to saying "try again later".  The idea behind that is that if it's a legit mail server, it gives them a chance to fix their problem and then try sending again.

I'm not in that camp. My email addresses have been plastered on web pages for  years and years, on mailing lists, usenet forums, etc. Thus, my address(es) are included in many of the spammers databases so I'm going to be a target regardless. I want to stop them dead so I set rbl_enable_soft_failure to 0, which chooses a 553 error "sorry go away, permanent error, you spammer pig" style. This tells the remote server, don't even bother trying again, it won't work.

Obviously with the latter arrangement, you'll see a lot fewer (than if using the former) RBL blocks. Many of the spammers don't honor the 553 error and try again anyhow but that's quite OK. The majority of mail servers (themselves victims of naughty people) will get the 553 and bounce the message, no longer trying to contact me. All is well. This is the very same reason why it's better to use a SMTP 553 error instead of simply packet filtering their IP.  Sure, they're blackholed, but they'll get a SMTP unreachable error and just keep trying. I want the spam traffic destined to my server to go away, not add to the cost of my system (via connectivity). When your server has thousands of mailboxes on it, these policy decisions make very real and measurable differences.

Doing a RBL lookup on a message at SMTP invocation time is cheap. Extremely cheap. It's a simple DNS query so it's very very very lightweight. I may even have it cached from the last time the spammer tried. It costs almost nothing. If I can deny a spam message there, before having to pass it to qmail-scanner (medium cost),  through the virus scanner (medium to expensive), bayesian filtering (low cost), spamassassin network tests: blacklists, DCC, pyzor, razor (low - medium), and then maildrop filtering, then you it's worth it to me.

So, reason #1:  It costs me less to use RBLs.

The next reason for RBL usage is more political. The spammers revenue (that they charge advertisers) is based on message deliveries. They maintain databases of email accounts that they can deliver to. They have developed very sophisticated ways of determining if a message got delivered (well beyond "did the SMTP server accept it") and further, if it landed in a real persons mailbox. These include delivery notifications, embedded HTML (the web server logs then reflect success), and other clever mechanisms.

Very few Mail User Agents are very clever about dealing with spam messages. Unfortunately, the ones that are (like Mail.app) are not the most popular. A lions share of users use that icky software from Redmond which is a spammers dream come true. Even if I tag and bag the spam to a special folder, they end user may peruse through it, opening the message to verify and thus letting the spammer know the message was delivered successfully. Yay, spam 'em again!  

So, if I can block the message entirely (not just from my mailbox, but from every landing on my server) then the spammer will never get back any indication of success. This increases the likelihood of being removed from the naughty miscreants list and thus reduces the likelihood of getting spammed from him again in the future.

Sounds great in theory right?  So how does it play out in real life?  I have quite a few "case studies" we'll call them, ranging from my own personal mail servers to mail systems with tens of thousands of mailboxes on them. In general, the spam to ham ratio of messages on the internet is rising, but in every case my mail servers (with RBLs), the number of RBL blocks I see is gradually decreasing. This suggests that either RBLs are becoming less effective or some other unknown factor(s) are at work.

So, let's look at my mail server as a case study. Over time my message volume on my server has continued to rise, so we should expect the level of spam messages as well as ham messages to be steadily increasing (or the number of RBL blocks to increase). The effectiveness of my spam filters has been fairly constant (>98%) for the past six months yet the number of spam messages being delivered on my mail server has steadily dropped. What accounts for the difference?

Reason #2: Keeping spam messages way from end users

About a year ago I was watching my mail server logs and noticed that quite a few of my mail users (of whom I know personally) were getting extraordinarily high levels of spam being delivered to them. I then decreased my spam tolerance by blocking mail servers without DNS (which required whitelisting a fair number of mail servers), increasing my RBL usage. Immediately after that the number of RBL hits spiked and then began a gradual decline ever since. This reduction of traffic is due almost entirely to the RBL usage as I tested the reverse DNS blocking at various times and ultimately removed it.

Approximately six months later I did another round of decreased spam tolerance and enabling the spam filtering for these users, all of whom are not very sophisticated mail users. They don't know or care about mail servers, filtering, etc. They just want their email to work. Overnight the levels of spam in their inboxes dropped to almost nothing (a condition I'm accustomed to living with). This, of course made them all very happy.  It also gave me a way to track the effectiveness of the RBLs, as I had a very accurate indicator of their effectiveness. RBLs only caught about 50% of the spam, but that's still a remarkably good achievement for something that costs so little to employ.

So, RBLs aren't a "one stop shopping" spam solution, but they are a very effective tool.

Reason #3:  Spammers hate RBLs

There is good reason why RBL operators are targets of DoS attacks. These days running a RBL means making sure you and your upstream provider are prepared to deal with massive DoS attacks, sometimes lasting over a week. These DoS attacks can cost many thousands of dollars in excess bandwidth fees, lost revenues, and related costs. This is the reason for several of the RBL operators decisions to step out of the game, including osirusoft which was one of, if not the most effective free RBL. If RBLs weren't effective, they wouldn't be targets.

Unfortunately, the spammers have millions of advertising dollars at stake. When a RBL is as effective as Osirusoft, it measurably effects their bottom line, and thus becomes a target. If I can hurt the spammers where it counts (in their wallet), then in my book, that's A Very Good Thing.

Matt