Here’s a quick interesting thing. I’ve been wondering about whether or not anyone does split testing in the real world with physical things.
For those that don’t know, split testing is where you do an experiment where you change something for a proportion of people to see how they react and measure the effect. If the proportion experiencing the change react more favourably (for a given definition of favourably) then you make the change for everyone.
An example is email subject lines. Marketing and campaigning emails are frequently split tested. The subject line or message has some variation and you measure the response (how many people open the email, how many people click a link in the email). If the variation has a higher response rate then you adopt that change in the future.
I’m SURE they do this kind of thing in the real world, but I’ve been trying to think of examples.
Now, recently there was a story about Leicestershire police ignoring attempted burglaries at odd numbered houses. The BBC actually gave a fairly accurate report, as did some other places, but the headline clearly didn’t communicate the reality.
And the reality was that this was a real-world split test! Burglaries were attended, but the forensics team were previously being sent to every case. Forensics were only helping in a very small number of cases, so they decided it wasn’t worth doing, but wanted to know if the public would mind forensics NOT being sent in to every case.
So they tested public reaction with a split test. Forensics would only be sent to critical cases, and in non-critical cases, only to burglaries with odd house numbers.
Strangely fascinating. Hear more at the BBC’s More or Less podcast; about seven minutes in:
http://www.bbc.co.uk/programmes/p030bct8