Spammers do A/B testing

26th September 2008 at 8:01 am 2 comments

I’m a regular reader of Linda Bustos’ excellent Get Elastic blog. Ecommerce isn’t my specialty but it’s something that affects all of us who shop online so I find it very interesting to see just how much work goes into the tiniest of details.

One of the subjects that came up recently and intrigued me was A/B testing. To perform an A/B test, an online retailer makes a change to their website and supplies the changed pages to some but not all of their visitors while the remainder see the original pages. By monitoring the statistics from both groups separately, the retailer can determine whether the change is a good thing or not. For instance you might change the colour of the “add to cart” button. A casual observer might think a change like that makes no difference at all… but test with enough people and it seems that there can actually be a measurable variation from something this small. And if you don’t A/B test then you’ll never find out. The world of online retailers appears to be full of folks fretting over whether they’d get a 0.3% increase in conversion rate if they reduced the width of their checkout button by 2 pixels. And while you might laugh, they have every right to fret – that’s money they could be making… or losing.

I was looking through my email spam folder the other day and noticed something interesting. Spammers are doing A/B testing too.

Look at this extract from my junk mail folder. This isn’t showing all my spam but it is showing an interesting subset of the spam that arrives to various usernames on a domain that I’ve owned for nearly 10 years now.

some of the spam in my mailbox

From the subject lines it’s easy to see that there are two usernames which are receiving this spam. This is backed up by the emails appearing in pairs. But the most interesting feature is that the duplicated emails frequently have different subject lines. So, for example, the baby laundry detergent spammer is sending out (say) 5,000,000 emails seeing if it’s dermatologist testing that helps make the sale, and another 5,000,000 seeing if it’s the stain removal powers (and, for all we know, another 50,000,000 emails trying out other variations). Of course it goes without saying that the links embedded in the two emails are slightly different so the spammer can check the different response rates.

Other amusing things I noticed…

  • the use of famous brands to increase recipient trust (that’s not really from Dell)
  • one spammer selling both Rolex and Viagra (might as well hit all the spam stereotypes)
  • he didn’t really have a job for me (the lying spamming b&%!#*d)

Entry filed under: Uncategorized. Tags: , .

Employment testing Canadian Do Not Call List

2 Comments Add your own

  • 1. Linda Bustos  |  26th September 2008 at 1:35 pm

    Hey John, thanks for the nice post about us πŸ™‚ It doesn’t surprise me that spammers are split testing, I wonder if it’s to see which improves conversions or which beat the spam filters, ha ha.

  • 2. Jon Jennings  |  26th September 2008 at 1:51 pm

    Thanks for the comment Linda.

    An interesting point – and one which I was going to blog about next week πŸ™‚


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed

September 2008

RSS Jon Jennings on Twitter

  • An error has occurred; the feed is probably down. Try again later.

%d bloggers like this: