We’ve joined the Carnegie team! Find out more.
Alert Close close
Intelligence
Watching a spam filter at work

Intelligence

Watching a spam filter at work

Oct 01, 2003By Michael Stoner

I’ve been watching how my new email program identifies spam and I’m learning some interesting lessons about what gets filtered and what doesn’t.

Some background. I’ve already done a lot to get rid of the spam in my elife: I gave up an Earthlink address I had been using since the early 90s because of the spam-more than 100 messages a day. I use a single email address for all my online newsletters and mailing lists-messages from the lists I subscribe to are filtered into one mailbox, news into another. These filters help to ensure that only the most important messages reach my Inbox.

I just started using Eudora 6.0, which includes software that filters messages it identifies as spam into a folder labeled JUNK. I have the option of scanning these messages to see if there are any I want to read. Unread messages are discarded every 30 days, or when I choose to empty the trash. I can tinker with the spam filter rules, but I haven’t: I wanted to watch what gets consigned to JUNK, figuring that many users won’t adjust the settings once a filter like this is installed.

Eudora uses SpamWatch, which relies on Bayesian filtering and a dictionary to identify and filter spam. My take on the software is that it rates a B+. It has helped to keep my mailboxes spam-free.

Here’s what I notice:

**The filter pays a lot of attention to subject lines, so messages with the trigger words that spammers use (“Special,” “You Have the Power,” “Cash available”) get filtered into JUNK.

**Messages with many images, or one large image, just don’t make it through the filter. So much for all the marketers advising (say) Gourmet, Starwood Hotels, Marriott, etc., about how effective HTML email is!

**I never see messages with “Viagra” in the subject line unless I look in the JUNK mailbox.

**Messages with an “aol.com” email address tend to get filtered into JUNK.

**Some messages that I want to see are inexplicably filtered, so I do have to scan this folder to make sure I’m not missing anything.

My advice to you is that you not only need to evaluate your email strategy to ensure that your institution isn’t sending too many emails [there is an interesting discussion on CUE‑L about this right now], but you need to pay very close attention to how your email is constructed. And what I’m seeing in my JUNK mailbox reinforces my long-standing advice to keep HTML email messages very simple or to avoid HTML email entirely.

I don’t know how the email filter will handle rich-media email-though it’s something I generally avoid in any case. And it’s a good idea to remember that more and more email is filtered at the ISP level-so it won’t even reach an email filter like the one I’m using


  • Michael Stoner Co-Founder and Co-Owner Was I born a skeptic or did I become one as I watched the hypestorm gather during the dotcom years, recede, and congeal once more as we come to terms with our online, social, mobile world? Whatever. I'm not much interested in cutting edge but what actually works for real people in the real world. Does that make me a bad person?