How to AB test a physical product

24 Nov
November 24, 2012

Online retail and online marketing has brought with it the possibility for realtime testing of multiple user experiences in a very clean and controlled manner. If you have a website and an online marketing campaign, chances are high that you could improve your returns with some testing. And it’s easy to get started – Google Analytics has testing functionality built-in. Adwords encourages testing. Advice is plentiful if you want to find out how to make a start.

So, the online world has a lot of options. But what about the physical world? How can you AB test a product?

Usually, product testing happens before a product is released, most obviously because it’s harder to make changes after it is released. Staples in the product development and testing arena include focus groups, customer panels, interviews, and surveys. Two common limitations of these approaches are: people don’t actually do what they say they would do, and bias in the form of group-think mentality.

However, there are possibilities to take some of the “release early, release often” mentality of the online world offline. One company doing this to seemingly great effect is TCHO.

TCHO is a chocolate company making high-end, interesting flavoured chocolate bars “based on the pure flavors of cacao“. They have a Beta testing programme for testing new bars and flavours. It’s a cross between a customer panel, a genetic algorithm, and ab testing. In their own words:

Just like with software development, we initially launch our bars in a Beta tasting program that is open to the public. We continually seek feedback from our Beta Tasters to improve our formulations. Thousands of people have contributed valuable feedback to help us arrive at our final formulations. Our first formulations went through over a thousand iterations based on this feedback, before finally being released as 1.0.

You can buy (that’s right, you pay for the chocolate yourself) into the Beta programme and TCHO send you 2 bars of chocolate, A and B, each with unique serial numbers so that TYCO know exactly the recipe used to make that bar. You taste the chocolates and vote for your favourite. TCHO take this feedback and use it to weed out bad recipes and allow good ones to prosper, improving their chocolate based on direct customer feedback. New bars are produced and sent out. Rinse and repeat until you have a winning chocolate bar.

How do TCHO do this? Well, they have built this testing into their manufacturing process. Release early, release often. Test, Learn, and Improve.

And not only is it a great way to improve their products, it’s also a great way to get loads of free customer feedback (remember, customers pay to be in the Beta programme!), and loads of free word-of-mouth marketing. Who doesn’t want to be involved in helping to make amazing chocolate flavours?

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>