GPWA Times Magazine - Issue 24 - April 2013

at your disposal you may only be able to test one hypothesis at a time. This breeds a whole new list of questions regarding how long you should conduct the test for. How can you tell if a current test is prov- ing significant or useless? It’s really quite simple. There are three very important things that must be considered when con- ducting your tests and analyzing the data. 1. Confidence level 2. Conversion range 3. Sample size The biggest mistake you can make – one that will automatically render your efforts useless – is to focus on only one of these elements. These three factors will make it clear as to whether A really is better than B or that there is no statistical significance between the two. Confidence level refers to “statistical con- fidence,” which measures howmany times out of 100 that a result can be expected to occur within a specified range. The num- ber you are aiming for here is a 99% confi- dence level; this means that there is only a 1% chance that the numbers are incorrect. If you were to achieve a 70% confidence level, it may seem promising but remem- ber that if you go along with that result you are choosing to accept the 30% risk that the numbers are wrong. Conversion range displays the range with- in which the actual conversion rate may lie. For example, if A produces a conver- sion range of “6.5% (±1%)” that means that the conversion rate lies somewhere between 5.5% and 7.5%. There is only a 1% margin of error, which is ideal because the smaller the margin of error the more accu- rate the numbers. Another rule of thumb is that if the two conversion rates overlap then you’ll need to retest to achieve a bet- ter result. So if B produces a conversion range of “8.5% (±2%)” that makes the conversion rate between 6.5% and 10.5%, which directly overlaps with A and ren- ders the result statistically insignificant. Sample size represents the number of visi- tors in your test and the number of con- versions that are completed. Generally speaking the bigger the sample size, the more accurate the result. If you’re unsure of how large a sample size to engage, re- fer to your variation. If the change you are testing is minor, increase the number of your sample. If it is a major change, then the sample size need not be as substantial. Note that a lot can happen within the first 100 conversions, not visits, so when you hit the 100-conversion mark it’s a good place to gauge the statistical significance. If one variant is clearly outperforming the other, then it’s a good sign that you have a winner. Fluctuations are common in the beginning, however, so adequate testing time is necessary. Decision making and moving forward At this point you have a clearer under- standing on what exactly a split test in- volves and why we utilize such testing in marketing. You know to begin with a well- structured control page in order to maxi- mize the testing potential and produce strong results. In terms of what to test, you know the varying elements that make up a page and the importance of being direct and incorporating calls to action. And even though the last section prob- ably reminded you of that statistics test you failed in high-school math, it is vital to understand the proper way to interpret data so that you can make it to this step. Making an informed decision based on the data at hand This is where all that testing pays off. By interpreting your data correctly and see- ing that there was a statistical significance between A and B you can move forward knowing you’re making an informed de- cision. For example, let’s say that you are testing the position of a signup box. Page A has it on the left-hand side and Page B has it on the right. After 100 conversions, the data clearly shows that there is a 40% increase of signups on Page B, there is no conversion range overlap, the margin of error is small and the confidence level is 99%. Tell me, where do you think you’ll be permanently placing that signup form? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tabitha Monroe is an affiliate manager and content writer with Mojo Affiliates and Olympian Group. She was swept into the gaming and entertainment world as a freelance writer and hasn’t looked back. A social media veteran, Tabitha has begun detailing her unique adventures navigating her new job in a self-titled blog, “The adventures of Tabitha Monroe.” She has a keen interest in psychology and currently resides on the Gold Coast in Australia. 53 Split Testing