In one of my other courses we are experimenting with A/B testing on our own designs. The syllabus introduced this Wired article in which the strengths and weaknesses of the method are discussed. Just as the assigned chapters for this course the article builds on the Optimizely founder, Dan Siroker’s revelation during the Obama 2008 campaign, but it also takes a critical perspective and highlights some of the potential downsides of A/B testing. I especially think the method’s bias towards incremental improvement and its lack of lessons are worth bringing into our discussion on Friday.
One thought on “Strengths and weaknesses of A/B Testing”
Leave a Reply
You must be logged in to post a comment.
Thanks for sharing this article. It brings up some good limitations of A-B testing. It surely cannot be a method for everything. It’s interesting to see if it leads only to incremental improvements. It depends on the “variation” allowed for in the new versions to be tested–if potentially large, then there can be large changes.