By: Steve Outing Because I have no interest in self-censoring this column, odds are high that some of my e-mail subscribers will not receive it -- or it will end up in their "junk mail" folders. That's because throughout this column I'll use various words that are banned by lots of e-mail spam filters at Internet service provider (ISP) and corporate servers -- words like "spam," "Viagra," "sex," "breast," "penis," "porn," "lover," "opt-in," and "e-mail." I'll even write a FEW WORDS IN ALL CAPS.
If you never use such "controversial" words in your opt-in newsletters (that is, e-mail publications that subscribers have asked to receive free, or paid to receive), the odds are much higher that your subscribers will receive your messages. Use any of those words and the likelihood increases that your subscribers will go unfulfilled (and if they're paying for your e-publication, they'll probably be angry).
As an ISP law expert told me, new anti-spam efforts increasingly are "catching more dolphins in the nets along with the tuna." And the best sure-fire way not to be an unfortunate dolphin is to edit your content so that it doesn't contain words that might trigger the filters. So much for freedom of expression on the Internet.
Ironically, when I e-mailed this column to my editor at
E&P, he didn't receive it because it was blocked by spam filters in place at his company. After several unsuccessful tries, I had to e-mail my column to his personal Earthlink account.
SpamAssassin worsens the problemThis has long been a problem because some ISPs and corporations use junk e-mail filters that are not very smart -- they catch the medical e-newsletter that covers breast cancer as well as the porn spam touting "Huge breasts!" -- although it's typically affected only a small percentage of an e-newsletter's subscribers. But it has become a more serious problem recently because of the popularity of a spam solution called
SpamAssassin, which is increasing the amount of opt-in (that is, ethical, non-spam) e-mail that is being blocked along with the spam.
SpamAssassin is being used by a growing number of ISPs, including major ones such as The Well, as well as corporations. (Individuals also can use the software.) It is a collaborative, community anti-spam effort that solicits the help of programmers throughout the Internet. What makes it an effective spam-killer is that it features hundreds of filtering rules from which the software analyzes e-mail -- not just subject lines and e-mail headers, but the content of a message -- and determines if it might be spam. Rules can be contributed by any programmer, and if a SpamAssassin project administrator approves them, they are included in new releases of the software.
New filtering rules thus are being added all the time -- so a legitimate e-mail publisher can never be sure that its mailings won't trip filters powered by SpamAssassin.
SpamAssassin doesn't actually block e-mail that it identifies as spam. Rather, it's a tool used by ISPs and server administrators to spot spam; it's up to the ISP administrators to decide what to do with it, according to
Matt Sergeant, one of the leaders of the SpamAssassin project. The Well, for example, sends e-mail suspected of being spam to its users' junk e-mail areas; other ISPs may choose to automatically delete e-mail tagged as spam, or bounce it back to the sender.
One problem that the software is creating, according to
John Buckman, president of
Lyris Technologies, an e-mail software developer and e-mail list hosting company, is that many ISP and corporate server administrators are setting too low of a "threshold" for catching spam. When SpamAssassin analyzes a piece of e-mail, it assigns a "score" based on how many of its filtering rules the e-mail violates. If an ISP administrator sets a low threshold, it will end up catching more legitimate, ethical e-mail along with the spam -- and that's what's happening more and more.
A second problem, says Buckman, is that some of the SpamAssassin filter rules that are being set are "dubious." They're too broad and end up catching lots of innocent opt-in mail.
E-mail publishing consultant
Vin Crosbie concurs that the ISPs are typically the ones causing the problem. "ISPs are blocking content in a very bone-headed way," he says, looking for keywords that spammers typically use -- but that ethical publishers also may use from time to time. A more sophisticated approach is necessary.
The situation sucks!Ethical e-mail publishers don't always know that their mail is not being delivered. Some corporate servers simply delete mail that they think is spam, so unless a subscriber complains about not receiving a requested newsletter, the publisher is left in the dark about there being a problem.
More often, the publisher will receive a "bounce" that will include clues as to why an e-mail message was rejected by a subscriber's ISP. Your best bet for tracking down the reason you were blocked by a spam filter is if one of your subscribers can forward you a copy of the e-mail that may have ended up in his/her junk e-mail box. The header of such an e-mail should contain a list of filter rules that the message violated (assuming that the subscriber's ISP uses SpamAssassin).
If you are an ethical e-mail publisher and find that your deliveries are being blocked by spam filters, it's seldom easy to get unblocked. You'll need to contact the offending ISP and ask that you be put on a "whitelist" that will allow your (asked-for) mail to get through to subscribers. Most ISPs don't make this an easy thing to do, and it will be difficult to track down the right person who can make this happen. This is where working with an e-mail hosting company is useful; such firms typically have a dedicated staff member who watches for rejected mail and contacts ISPs to resolve the problem.
An increasing number of legitimate e-mail publishers are finding that their mail is triggering filter rules in SpamAssassin. Unfortunately, the SpamAssassin system relies on complaints and "who screams loudest" for changes. If a recently added filtering rule ends up snaring a lot of legitimate e-mail deliveries, then enough complaining might lead to SpamAssassin's administrators changing or eliminating the filter. But that can take a long time -- and meanwhile requested e-mail continues to be blocked.
What's the solution?An immediate solution is, from a journalism perspective, a poor one: edit your opt-in e-mail newsletter's content so that it will get through the filters. Don't use words that will trigger the filters. Rephrase sentences to avoid certain combinations. Any competent journalist will balk at such nonsense as not using the word "Viagra" in a story about men's sexual dysfunction, or "spam" in an article about junk e-mail, but this is one way around the problem.
What does not exist yet is a Web service on which publishers could submit their newsletters for screening -- and it would identify content or technical problems that would likely trigger spam filters before mailing. (The trouble with such a service is that it would be used by spammers to try and get their mail through.)
A simple way to screen is to send an individual message to a test account at an ISP that uses SpamAssassin. If it gets through, then you should be OK; since SpamAssassin does not account for quantity of e-mail, it should make it to many subscribers who use that ISP. And getting through one SpamAssassin-powered ISP filter should assure that you get through to others.
The e-mail experts I interviewed for this story suggest that the best solution is simply to keep watch for when your opt-in mailings are blocked, then complain (loudly) to the ISP. Because publishers typically don't have relationships with ISPs that block them, it's important to present an ISP with a specific complaint from one of its customers about not receiving asked-for e-mail.
Buckman also suggests that a good overall solution is for some organization to set up an Internet-wide "whitelist" that would include information on ethical publishers -- which spam filters would be obligated not to block. Such a whitelist organization should mirror the model of
TRUSTe, a Web site that guarantees that its member sites abide by tight Web and e-mail privacy guidelines.
Is there a legal angle?As spam filters like SpamAssassin widen their nets in efforts to catch more junk e-mail, and inadvertently block more opt-in e-mail as well, the argument can be made that spam filtering is causing economic harm to ethical publishers. Especially for those who charge money for e-mail subscriptions, undelivered messages can result in angry customers and canceled orders.
Is there a legal remedy available, then? The consensus appears to be No.
Paul Michael DeCicco, a San Diego-based attorney who specializes in ISP law, says that publishers are not in a position to seek remedy from an ISP because there is no formal relationship between them. It would be up to the subscriber to an e-mail service (say, one he/she paid for but was not receiving because of an ISP's spam filter) to challenge the ISP. The ISP has no duty to deliver on behalf of the publisher, DeCicco says.
A possible argument would be that the ISP's filters are interfering with the monetary relationship between publisher and subscriber -- but that argument would be a legal stretch, he says.
DeCicco also points out that ISPs are "between a rock and a hard place" on this issue. By not filtering incoming e-mail, it's possible that an ISP's customers could be hit by a virus, for example, and that carries more liability risk for the ISP than filtering for spam and in the process accidentally catching a few "good" e-mails from publishers with which it has no business relationship.
Is this really a problem?How many "good" e-mails are getting caught by spam filters? There's dispute about how serious the problem is, but Lyris' Buckman says that in the last three weeks he's seen a growing number of complaints from his customers as new ISPs have begun deploying SpamAssassin.
Examples of blocked opt-in e-mail deliveries are rampant. (Not all have to do with SpamAssassin use.) A few:
*
Mystery Lovers Bookshop sends out an e-mail newsletter. It discovered that an Oklahoma ISP was blocking e-mail from Web sites with "lover" in their URLs, including its newsletter.
* The popular Windows tips newsletter
Lockergnome recently was caught by spam filters using SpamAssassin because Lockergnome was sent out with a "Message-ID" e-mail header line that included the "#" character -- which is used sometimes by spammers, but also by ethical e-mail publishers. (This example demonstrates that technical factors sometimes can trigger spam filters.)
* The widely circulated Macintosh e-mail newsletter
TidBITS has been hit especially hard in the last year by spam filters. Rejections from subscribers' ISP servers number from the hundreds to the thousands some weeks -- based on what words the newsletter's editors use. In a recent article, "
E-mail Filtering: Killing the Killer App," TidBITS'
Geoff Duncan noted that in one issue that used the word Viagra, 2,500 deliveries were rejected. Duncan wrote, "We're starting to see signs that e-mail, often hailed as the Internet's 'killer app,' is in danger of becoming an unreliable, arbitrarily censored medium -- and there's very little we can do about it."
*
Jeff Laurie, publisher of a popular opt-in e-mail newsletter called
Sex News Daily (featuring humorous sex-related news items), reports that getting caught by spam filters is common -- since he regularly uses words that some filters deem "unsavory." (Viagra, penis, vagina, etc.)
*
Randy Cassingham, who publishes both free and paid versions of his humorous-news newsletter,
This Is True, also complains that he's been blocked for using such words. He comments, "If you can't use proper medical terminology, what
can you use? Do I really have to resort to 'pee pee' when I'm quoting a medical report? I can't imagine the troubles that health newsletters are having!"
Cassingham sums up the spam and spam-filtering dilemma nicely: "This complication is the sort of thing I've been predicting since 1996, when spam was really first starting to be a problem and I wrote the first draft of my
Spam Primer. I fear it will continue to get worse until something drastic is done about spam."
Other recent columnsKnight Ridder Digital Cedes Some Control, Wednesday, July 17
Board The Weblog Bandwagon Now, Please, Wednesday, June 26
Experts Offer Tips On Usability, Wednesday, June 12
Libraries Threaten Paid Online Archives, Wednesday, May 29
Content Publishing Systems Squash News Design, Wednesday, May 8
Have u 4gotten IM?, Wednesday, April 24
Previous columns
Get Stop The Presses! by e-mailIf you would like to get e-mail delivery of the Stop The Presses! column, there are two options:
1) Text e-mail. I send out a text e-mail message containing a brief description of the current column, along with a URL link to the actual column on the E&P Web site. To receive these regular reminders,
signup here.
2) HTML e-mail. If you prefer to receive the entire column, you can have it delivered to you as an HTML e-mail message whenever a new column is published.
Sign up here.
Got a tip? Let me know about itIf you have a newsworthy item about the online news media business, please send me a note.
Comments
No comments on this item Please log in to comment by clicking here