Step by Step Guide for Using Adwords' Campaign Experiments (ACE) for Ad Copy Testing
Back in October, I wrote a post on how to use Google’s ACE tool for landing page testing. In December, Google launched a new feature for this innovative tool that makes ad copy testing using ACE possible. They describe this new feature in detail in their blog post linked here.
As the post explains, “In the past, you may have used ad rotation to test ad variations within an ad group. The main difference between ACE and ad rotation is that ACE allows you to decide the percentage of traffic on which you want your experiment to run – giving you more control over which ads show when”.
In addition to the benefit described above ACE will also automatically calculate whether or not statistical significance has been achieved, which is very useful to be able to see this right in the interface.
Recently I took over an account for a client who was managing paid search in house. They had some really strong ad description lines in the account that I wanted to keep running. I decided to test the existing description line vs. a new description line for each ad group.
I planned to analyze whether or not the new variations outperformed the existing variations. I could have done this by setting my ad copy to rotate evenly. Once data collected, I would then export all the ad copy into Excel and use a Pivot Table to analyze the Control ads vs. the Experiment ads. However, instead I decided to take advantage of the new ad copy feature that Google launched for ACE. ACE made it easy for me to test my ad copy and see which ad variations were statistically significant winners at the campaign level, or within individual ad groups.
Below is the step by step guide as to how I used ACE for ad copy testing:
- Once you have your campaign and ad copy uploaded, navigate to the campaign settings.
Within campaign settings scroll down to the bottom and click on the plus box. Add in the details of your experiment. If you are testing new ads vs. current winners, then you may want to set your test so that your control runs for a greater percent of the time than your experiment ads. If you expect your ad groups to get a lot of volume you can run the experiment for a smaller percent of time, for example, 10%, vs. 90% for your control. However, if you do not expect to get a lot of volume then I would choose a split closer to 50/50, so that you can get enough data for each variation to achieve statistical significance. Next, add in the date you want to launch your test and the date you would like your experiment to automatically stop running and hit ‘save’. I would set a reminder to yourself as to when you set the end date for the experiment because your experiment ads will stop running on that date. - Now that you have started an experiment, navigate to view all ads within the campaign.
Create a filter so that all of the ads you plan on setting as the Control are visible. For instance, in my example all of my new ads contained BBB in the description line, while the client’s existing description lines did not. Thus, under Filter in the drop down I selected Ad Text>does not contain> BBB and hit apply. - Only the ads that will be set as Control should be visible. Select all ads and then under status select Control only.
- Repeat step 3, but now create a filter that will only show the ads you would like to set as your experiment. In my example, I set the filter to contains >BBB. Select all visible ads and changed their status to Experiment.
- Now let your campaign run as normal and wait for data to gather. You can view your results at the campaign level to see how your Experiment ads are running as an entire group or at the ad group level.
- Viewing results at the campaign level: Currently the example below shows that a statistically significant winner has not been achieved for all the Experiment ads vs. Control ads as a group for CTR since the up and down arrow is greyed out.
- Viewing results at the ad group level: Currently the example below, shows that a statistically significant winner for CTR and clicks has been achieved for the one ad group detailed below.
The Experiment ad actually lost in this example. The one arrow down for clicks and CTR signify with 95% confidence, that the Experiment ad performs worse on these metrics than the Control.
Using ACE for testing ad copy is very easy and results and statistical significance can be seen right in the interface without any extra computation. Below are two ideas to make this tool even more useful for ad copy testing:
- The ability to export the experiment results. Right now it is not possible to pull a report that includes the experiment segments broken out. The ability to do this would be helpful to report on findings to clients, as well as to keep historical data since once you stop your experiment this data is lost.
- Automatic Alerts: It would be great if Google added the functionality to alert you when you have a winning ad copy variation or when your experiment is coming to an end. This would be helpful If you were running tests across multiple accounts, campaigns or ad groups so you do not have remember to check results and when you set the end date.
As this tool is still in Beta, I have confidence that Google will eventually add additional features to ACE. For now, I still find this tool to be incredibly useful for testing ad copy.
Posted: 01.10.11
MaheshSrinivasan:
Hi, I got the holistic view on using Adwords Campaign Experiment than the word on Google Adwords Blog. Neat and clear.
Bonnie:
@Mahesh great! I am glad you are enjoying ACE as well.
Sign Company:
I have personally been using the experiments feature for a while now. It makes it incredibly easy to fine tune your ad copy so you can find that winning formula.
Terry Whalen:
Hi Bonnie,
Thank you for writing and screen-shotting so clearly! We will give this a try.
Someday, we hope that ACE allows user to split-test between a control campaign and a test campaign with different campaign settings.
Best,
Terry
Bonnie Schwartz:
@sign company, I agree ACE is very user friendly
@Terry I’m with you 100% I would love to test day parting, sitelinks, ad optimize vs. rotate. Im glad the screen shots were helpful!!
Bonnie Schwartz:
I just noticed as a follow up to this that you can now export experiment results, which is very help if you want to send the ad copy break down to clients.
Terry Whalen:
Ah – I figured we’d run into a problem with ACE. We just found out we cannot use it to test ads if we are already using Enhanced CPC bids. I actually think enhanced cpc bidding works really well, and we use it a lot. But I do get frustrated with so many things the AdWords folks put out — much of it sounds good on paper or in conversation, but we seem to always run up against a brick wall in all our use cases. This is just one more example. And using ACE to test keyword bids – that’s such a silly idea, yet that is one of the prime uses of ACE, I think.
Bonnie Schwartz:
@Terry that is a bummer you cant use with enhanced cpc bidding. Good to know you are having good success with enhanced cpc bidding. I usually find that Google innovates their product eventually to include features that I originally had wished. I hope my wish comes true for campaigns settings and enhanced cpc bidding
Aaron Luckie:
Great post on using ACE.
Have just implemented this into a number of client accounts for ad copy testing.
Do you always tend to use the 50/50 experiment split? Additionally would you recommend 2-3 weeks for a trial period or longer?
Cheers!
Bonnie Schwartz:
@Aaron Thanks for your comment. The 50/50 split depends on the volume you expect on the account. Recently I have been running a 70/30 split for high volume ad groups so that if my test ads perform worse than the control I am not negatively impacting my numbers.
In terms of a time frame, that also depends on volume. I wait until there is stat significance before making any decisions.
Thanks!
Bonnie
Ankur:
Hey Bonnie,
Thanks for this. I’m sure you guys are now aware that you can actually download the experiment reports into excel.
However they aren’t that clear. There’s this column called “Ace Indicator” next to each metric. I have no idea what those nosrepresent. Has anyone come across this ?
Cheers,
Ankur !
Bonnie Schwartz:
@Ankur thanks for the comment and update. The ace indicator is the level of stat significance. Below is taken from Google Help:
- one represents that there is a 5% probability that the metric increased or decreased by chance rather than due to your experimental changes.
- two represents a 1% probability
- three arrows represents a 0.1% probability
Hope this helps
Ankur:
@ Bonnie this sounds fab,
Thanks, Ankur