Jeff Random

musings and metatheory


A B Split Test Confidence Based Sample Size Calculator

a b split testing sample size calculator

One of the basic building blocks of optimization is split testing.

While the metatheory is the same regardless of what you are testing, this is an example of applied marketing theory. This tool is focused (labeled) on conversions (sales) as the datapoints, but you could just as easily use it for any other kind of success (such as opt-ins, free downloads, registrations etc).

Whether you call it AB split testing (or even A B C D E testing), using split testing software, conversion optimization, landing page optimization or any other name you still want to act on your tests as soon as possible.

The split test sample size calculator will show you how much traffic you need to send to each test segment, based on how confident you want to be in the results.

This part of the materials from the Metric & Analytical Marketing workshop at Internext-Expo. While the workshop was focused on adult website marketing, the tools & theory should be useful in almost any vertical.

Published by r8ndom, on January 24th, 2007 at 11:01 am. Filled under: Metatheory,Optimization,PostsNo Comments

Meta-Theory of Optimization

Meta-Theory of Optimization

Optimization is the discipline (art/science) which is concerned with finding the maxima and minima of functions, subject to constraints.

In Meta-Terms, Optimization does not necessarily mean finding the optimum solution to a problem. Often this is not possible (constraints), and this is where the balance (heuristics) is used to make principally accurate judgments factoring in given circumstances. Then these are tested and refined to a degree that is itself part of the Optimization, which grows into a self-referential diminishing curve.

Knowing what questions to ask (your metrics), what tests to run (your data), and what to do with the information (analysis) is the Art of Optimization.

This is a definition in progress. What do you think? Let me know

Published by r8ndom, on February 21st, 2006 at 1:36 pm. Filled under: Metatheory,Optimization,PostsNo Comments