K-Squared Ramblings

iPhoney

Two items from @ThisIsTrue on the iPhone’s app store:

• Apple’s new low: censoring a dictionary #
• Followup on the Apple dictionary story. #

Also, the webcomic Cat and Girl was Sent from my iPhone (via @brionv) #

Email Confusion

• Why do people email me to tell me email is down? WHY? What makes them think I’ll receive the message? #
• Spammer sent me this urgent message: “We need to remove cupboard, come on!” Uh, yeah, I’ll get right on that… #

CentOS List Hijack

• Pissed off because some a-hole w/ a centos.org addr posted multiple copies of a racist antisemitic diatribe to the CentOS announcement list #
• CentOS sent an apology to lists. Said spammers forged the sender’s address to get past moderation. Look back, name doesn’t match address. #

Updates, Linking In, Inbox Cleanup

• Arg. Still can’t update Office. Wonder if it’s the IE8 beta. Time to download patches manually, I guess. #
• Hey, a Farscape podcast linked to my post on the trip to see Gigi Edgley fire twirling! #
• Serious stab at cleaning up inbox: 550+ messages down to 320. Also first stab at updating/syncing contacts (so many out of date entries) #

Spam Filters Gone Wild: This Is True

Waaay back in the dark ages of the Web (somewhere between 1994 and 1997) I discovered a weekly email newsletter called “This Is True.” It collected strange-but-true news stories from around the world, summarizing each in a short paragraph with a witty one-liner at the end. I subscribed to the free edition, and later to the full version, which had about twice as many stories. I even picked up a few of the books collecting past stories (at a con, I think, but I can’t remember which con).

Eventually I got too busy to read them, and the back-issues piled up unread, and I decided to let my subscription lapse. But earlier this year, I decided to re-up with the shorter, free version, and it’s still as good as ever.

This week’s issue included a disappointing story: even though they practice — in fact, probably helped originate — responsible list management, Yahoo is blocking them as spammers. Why? Because people are signing up for the list, then deciding they don’t want it anymore, and instead of unsubscribing, hitting the “Report as Spam” button. Yahoo has apparently taken those spam reports at face value, and blocked everyone’s copy of the newsletter.

Clearly, some people are unclear on what “spam” means. It’s not just “mail I don’t want.” It’s “mass mail I don’t want and didn’t ask for.”

That, and I’m sure some people don’t realize that their reports are being used to train everyone’s filters. I remember a co-worker explaining a few years ago that he’d trained Gmail to send the SourceForge newsletters (or something similar) straight into his spam folder. I commented that they might be using that data to train their sitewide filters, and he said something like, “I hope not.”

Using user feedback to train sitewide or network-wide (such as Cloudmark, or Akismet) filters is a powerful technique. Some people will catch the leading edge of a spam attack, and that data can be used to protect others as the attack continues. Some will check their mail sooner, and that data can be used to re-filter messages that have been received, but not yet viewed.

Unfortunately, it also can give a lot of power to people who are either unclear on the criteria being used or have an axe to grind, unless you include measures to (a) contain the impact or (b) keep track of each reporter’s reliability. I know Cloudmark factors in the reporter’s reputation, for instance. And I suspect that AOL does, at least in some cases, limit measures such as blocking to specific recipients, but I can’t be certain.

Anyway, to summarize:

• Use the Report Spam button responsibly.  If you actually subscribed to it, it isn’t spam unless they refuse to remove you from the list.
• Check out This is True.  You may laugh, you may groan, you may think, or you may get pissed off at the world — or all of the above.  It’s certainly worth a look.

(I really should have finished writing this yesterday, before someone submitted the original story to Slashdot. Posting about it to get the word out seems kind of redundant now. Heck, now that I think about it, I should have submitted the original to Slashdot. Oh, well.

Flagging (Non)-Spoofed Mail

Following up on the PayPal anti-phishing discussion of a few weeks ago, I see that PayPal is promoting a service called Iconix. You install the program on your system, and it looks at your inbox for messages that claim to be from one of its customers. It tries to verify them “using industry-standard authentication technologies such as Sender ID and DomainKeys.” Messages that pass get a lock-and-checkbox icon attached to the sender’s name, and in some cases the name is replaced by the sender’s logo.

On the tech side, it’s similar to SpamAssassin’s whitelist_from_spf and whitelist_from_dkim features. Both allow you to specify a sender to whitelist, and it will only give a message special treatment if it can verify the sender.

On the user-interface side, it’s similar to EC certificates, in that it tries to highlight a “good” class of messages rather than flag or filter out a “bad” class.

It’s not a bad idea, actually, and now that I’m surprised I haven’t seen something similar in other email clients. It’s sort of like setting up custom rings or images for images on your cell phone address book

They seem to be focused on webmail and Outlook so far, and only on Windows, but it looks like the perfect candidate for a Thunderbird extension. They do have a sign-up form to notify you when they add support for various programs and OSes, and I was pleased to see not only Thunderbird and Mac OS listed, but Linux as well. Too often, Linux gets forgotten in the shuffle to ensure compatibility with every Windows variation.

Confidential? Perhaps not…

I found a 419 scam in the spamtraps that started, in typical fashion, with an all-caps name and address, then the line:

HIGHLY CONFIDENTIAL REQUESTING

What made this funny (aside from the bad grammar) was the fact that the To: line contained over 1,200 addresses!

Ah, this is obviously some strange use of the word confidential that I wasn’t previously aware of!

Email advice: Pick a domain and stick with it!

Here’s a piece of friendly advice from a mail server admin to companies that interact with subscribers and customers via email:

Pick one domain name for your business. Just one. Don’t use any other domains in your emails, even if you want to keep order confirmations separate from promotions. If you contract out for some other company to send out a newsletter or survey to your customers, insist that they send it out using your own domain name. If you’re using DomainKeys or SPF, make sure they’re authorized or send it yourself. And don’t even think of making the links through redirection scripts, even if you really want to track which subscribers are clicking.

Why?

Two words: Spam and fraud. Read the rest of this entry »

How Thunderbird’s Scam Detection Works

Since upgrading to Mozilla Thunderbird 1.5 beta 2, I’ve seen a number of messages slapped with a warning label that “Thunderbird thinks this message might be an email scam.” It appears at the top of the message, in the same style as the junk mail notice bar or the warning that remote images have been blocked, and there’s a button to mark the message as “Not a Scam.”

There’s only one problem. Since SpamAssassin and ClamAV do such a good job of catching the phishing scams before they reach my inbox, Thunderbird has yet to catch any actual phish. But there’ve been a lot of false positives. It’s hit LiveJournal reply notices, newsletters from IEEE and Golden Key, a Spam Karma notice from my own blog, and I’ve seen it on both outbid notices and updates to saved searches from eBay.

I found myself wondering just how Thunderbird’s phishing detection decides that a message is suspicious—and how to teach it that the next LJ notice isn’t a scam.

The Thunderbird support website doesn’t seem to have been updated yet. Most of the articles I’ve found only talk about TB adding the feature, not how it works. The best information I found was this Mozillazine forum thread, which included a link to the actual code that makes the decision, in phishingDetector.js. Thunderbird looks at the following:

• Links that only use an IP address, including dotted decimal, octal, hex, dword, or some mixed encoding.
• Links that claim to go to one site, but actually go to another. (Phishers do this to fool you into going to their site. Legit mailing lists sometimes do this with redirectors for tracking purposes.)
• Forms embedded in the email. (This explains the LiveJournal notices.)

It also appears to trap text URLs containing HTML-escaped characters, which explains the Spam Karma reports. In this case the report includes a spammer’s link with ​ in the hostname. The message is plain text, so Thunderbird leaves the entity as-is when displaying it…but decodes it when it creates the link. Result: a link where the text and URL don’t match.

The easiest way to prevent it from freaking out over the next message? Add the sender to your address book. I’m not sure that’s a great idea, since a phisher could guess which addresses you have saved and spoof them, but it’s at least simple. I guess I’ll find out whether it works the next time I get a reply notice from LJ. Update: Adding the sender to your address book doesn’t seem to have any effect.

Update 2 (July 12, 2006): The comment thread’s gotten long enough that I can see people might miss this, so here’s how to disable it:

1. Open Options or Preferences (this will be under the Tools menu on Windows, Thunderbird on Mac, or Edit on Linux).
2. Click on Privacy (there should be a big padlock icon).
3. Click on the E-mail Scams tab.
4. Disable the “Check mail messages for email scams” option and click on Close.

That’s it.

Accidental Blogspam

I just got a complaint about the latest comment on Another One Bites the Dust. Apparently the previous commenter (who checked the “Subscribe to comments” box) either entered someone else’s email address or forgot visiting the site. It’s a name123@example.com-style address, so it could easily have been a typo.

Either way, the new comment notice went out, and the recipient sent me a spam complaint. I apologized and removed him from the update list, but it moves “accidental spam” from a theoretical risk to an observed problem. I’ve disabled the subscription plugin until I have a chance to figure this out.

The good news is that Subscribe to Comments 2.0 is out now, so I should be able to upgrade when I get a chance. The bad news is that it doesn’t seem to have added a confirmation step, meaning it’s still (effectively) opt-out. Sure, you have to opt-in to get it in the first place…but the fact is that anyone can opt you in just by giving your email address instead of their own.

Man or Machine?

In the old days, we used to accept email sent to any local account. This meant that various system accounts would collect outside mail instead of bouncing it. No one was reading, say, rpm@example.com, or apache@example.com, but the mailboxes were there.

Enter the dictionary attacks. An awful lot of those standard accounts are three-letter names—rpm, gdm, bin, adm, etc. Spammers trying to guess addresses made up of three initials landed on these addresses, confirmed them, and added them to their lists. The system accounts began collecting spam.

Eventually we locked things down so that only “real” accounts would accept mail from outside. But here was this steady stream of 100% spam we could use to help train our filters.

The funny thing: these days, nearly all of it is for sex-related drugs or body part enlargements. Sent to software!

(Incidentally, if you can read this sentence, don’t send mail to ramblo@hyperborea.org.)

On to step 8

Hmm, CNET reports that spammers are starting to route zombies’ mail through the ISP’s servers. (Hmm, that sounds familiar.) I don’t know about the “email meltdown” Linford warns against, but it will require a change in tactics. And so the escalation continues…

Blocking spam by source

A brief history:

1. Spammers send mail directly to victims.
2. Server admins block by source, victims complain and try to get spammers kicked off their networks.
3. Spammers relay through third-party servers to disguise their origin.
4. Server admins shut close relays, and block mail from open relays.
5. Spammers relay through trojaned zombies straight to victims.
6. Network admins block outgoing mail traffic except through their servers.
7. Spammers relay through zombies’ ISPs’ mail servers.
8. ????

We’re in the early stages of step 6, with broadband ISPs starting to block outgoing direct-to-MX mail traffic. The obvious response by spammers is, of course, Read the rest of this entry »