So a little while back I tweeted a blog post on A/B testing a product logo. Now I love A/B testing, the thought that you can do a simple test online and actually measure the impact of minor changes in web copy hit all (well... actually some of the many) of the buttons that get me excited; human behavior, statistics, measurement, and the TRUTH!
So when I came across this blog entry that talked about how a 99 designs logo was handily trumped by a one-off custom logo it was a perfect tweet. A perfect example of what A/B testing can show. That a cheap custom logo was more effective than a 'generic' logo by 99 designs.
But this tweet would teach me two things.
1) That if you're formulating an opinion about something and you have access to an expert in that something; ask them about it.
2) Do this BEFORE you publically weigh in with your opinion.
So after tweeting this I sent the link directly to a friend who does web design for a living and she immediately responded back with a couple very valid criticisms of the original this article.
These were; the power of the first logo was that it had a bingo card which would validate to the person that clicked on the ad that they were in the right spot and that the rest of the site was clearly designed to support that original logo. In doing the A/B testing the tester had taken his original site and simply replaced the original logo with the 99 designs 'generic'!
Once that was pointed out to me it seemed clear that the A/B testing was flawed. Because, after all, the A/B test was really the web page not simply the logo. And once this was pointed out to me it seemed clear that the B page was fatally flawed by its discontinuity.
Thank you expert.
No comments:
Post a Comment