Hello!
This is a Medium Rare template, check out more here.
Are you running Facebook Ads without testing creatives? How is that working out for you?
It’s been proven over and over again that Facebook creative is what moves the needle the most when it comes to the success of your paid social marketing campaigns. "Always be testing”: the number one rule when it comes to paid advertising and digital marketing in general.
"Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day"
- Jeff Bezos
There’s a direct correlation between testing and the success of your business in general. When you test you learn, cut out what is not working and be more consumer-centric.
However, how do you actually test properly? How can you make sure that you are conducting your test in the “right way” or evaluate your test properly.
All this and more, we are going to cover in this article. This article is perfect for you if you meet one of the following criteria:
We are going to start out by talking about the importance of testing, however, if you know that you already need to be testing constantly, then feel free to skip ahead to your most interesting section.
This is our agenda. Scroll ahead 👇🏼
It doesn’t matter what your company is all about, testing will be directly related to the growth within your company. It doesn’t matter whether it's Amazon, Tesla or Twitter an increase in growth rate could always be attributed to a certain test they have launched.
Testing lets advertisers uncover what is converting for their target audience and therefore they can then iterate and make sure to have a more appealing messaging which will then lead to a decrease in cost per action.
Often when entrepreneurs or companies say that Facebook Ads don’t work for them, they have just not tested enough variables and variants to determine what is working and double-down on that.
When people think that they are already testing, but they feel like that their performance is not improving even on a long-term basis, then the source reason for that normally is that they are trying to do too much at once. They try to test too many things and therefore then lose sight of the one specific adjustment and aren’t able to attribute that uplift to a particular adjustment.
Therefore, let’s talk about the different elements of an ad that you can test to increase user behavior and actions and how each and every one of them has a different impact on performance.
Primary Text:
Often referred to as the ad copy or body text, this is normally the first or second thing that your prospect sees, depending on whether the actual creative catches their eye first.
Facebook recommends to have it less than 125 characters, however, as with every copy here it’s about testing. You have to consider your audience as well as the point in the customer journey you are targeting this ad to, meaning retargeting or cold.
Either way, here you go with some best practices:
Ad Creative:
This is your most important element within your creative that moves the needle the most, although it is an unproven theory that videos convert better than images, this is also something that should be tested.
In your ad creative you should really get across the unique value proposition, however, make sure to keep in mind that you are not necessarily trying to sell your product here. This is what you do on your product page. With your Facebook Ad Creative, you are solely trying to spike enough desire to make sure to what we like to call “sell the click”.
There’s an avalanche of information that we could unload right now, but for now, we can boil our basic specifications down to:
Headline:
If there’s one thing that you want your prospect to take away from your ad then it should go into the headline. You want to make sure to emphasize why they should click on your ad and check out your product or brand right now.
However, keep in mind that this ad copy accounts for a very small percentage of the overall effectiveness of your ad. 25-50 characters will make sure that it won’t get cut off.
Description:
This is the text that appears right underneath your headline. It’s one of the limited places within your ad that is optional.
Your description should normally support your headline, but it can be used in a number of different ways. Just ask yourself: “What would be the last tipping point for somebody to click on my ad right now and convert?”
Here you go with some examples:
Call-To-Action-Button
If your ad has successfully done the job and spiked the interest of your prospect, then this button will determine the next action.
It has been proven in various studies that the “Shop Now”-button, although it sometimes leads to a lower CTR-rates, normally converts the most people into actual customers.
The following are the most common CTA-buttons that we at VictoryMedia use:
We see it day in, day out: We onboard a new account and take a look at their prior account performance and tests. 9 times out of ten we then notice that they have been launching a lot of tests, but without any structure.
In order to successfully grow your campaign’s performance with testing, it is inevitable that you need to prioritize your tests.
So, you need to ask yourself: “What creative element has the most impact on my overall campaign’s success?” To save you some time: It is your ad creative. Especially, when you are on a lower budget, you want to first and relentlessly test your ad creative.
Your scroll-stopper, different creative angles and ultimately your type of creative, meaning, whether UGC-content, a static image or a GIF performs better within your account and engages the most with your target audience.
Normally the first test that we launch at VictoryMedia is a “type of creative”-test. We analyse the former account performance and the best-performing creatives. Then we go ahead and test that against other types of creatives that we have at hand for this account.
Let’s use a quick example for a skincare brand: Let’s imagine that based on our qualitative research we found out that it’s important for our audience that they don’t need too much time to actually use our skincare products. We would then take that angle and craft four different types of creatives for this angle.
Based on the prior performance of their ad account: UGC-content worked really well. We would then create one UGC-creative, another product demonstration video, a comparison creative and a static image and run them against each other.
So, congrats now you know why you should and what you should be testing first, but you might be wondering: “How can I actually launch those tests?”
There are three major media buying strategies on how you can structure your testing framework, so let’s dive into each of them since they all have their pros and cons…
With this technique you simply add multiple ad creatives within one ad set that has the same targeting options.
Strictly seen this is not necessarily split-testing, since your audience will not be split and your ads will not be displayed evenly.
“You won’t have an equal distribution of traffic across all variants, but you get a good enough read of which ad is better.”
– Shamanth Rao, VP Growth & UA at FreshPlanet
Pros:
Facebook will optimize and deliver your ad based upon your CTR (click-through-rate) and CPA (cost-per-action). This will give you an optimisation at the ad level. You are also in full control, meaning, you can manually see the performance of your ad creative and make decisions based on those metrics that will not directly affect the delivery of your ad set.
Cons:
Your optimisation advantage is also your drawback. Since Facebook will distribute your ad set budget across the different ad creatives within your ad set based on their evaluation of the possible performance, you run at risk that Facebook dials into one creative pretty soon and that the other test won’t get any significant results or spend.
However, this is a manageable drawback: You can simply turn off your creative after you have successfully reached a level of statistical significance for a particular creative, so Facebook is forced to spend more money on the other creatives. You’ll then just compare the different performances after you have successfully gathered enough spend for each creative that you have initially launched.
Facebook offers you a native split-testing function where you can create a different ad set and the ad set has one different variable.
Other than with the manual way of testing your creative, Facebook now splits your audience into different groups that do not overlap, so you have a 100% accurate data set. Also, Facebook calculates the statistical significance for you, so you can make sure that your results are reliable.
With that native feature from Facebook you can test the following variables:
Pros:
The split-testing feature of Facebook is an interesting feature, especially regarding its reliability. The accuracy of your results are top-notch, however, this comes at a cost.
Cons:
Since you have to allocate a dedicated budget for it and Facebook needs to get enough results in order to be statistically significant, you’ll need to spend a decent amount of money and time in order to complete that split-test.
This goes against the general maxim of staying agile and moving on to the next test as fast as possible - therefore especially for accounts spending less than six figures per month, we tend to stay away from this methodology.
Conclusion:
There are obviously more ways to split-test your creative, however, every buyer buys differently, but those are the two main strategies that will lead to you being able to make accurate decisions based upon a data set that is not fragmented and therefore provides a level of statistical significance.
At VictoryMedia we normally opt-in for the first methodology, the manual way of adding several ad creative within one ad set. We create what we call a “creative sandbox” campaign which we use to play around with and cycle in new creatives to identify new winning creative which we can then duplicate into our actual scaling campaigns.
Setting an initial creative baseline is crucial to understand the future success of your tests and hypotheses, since you then actively put a measurable metrics behind the vague and subjective term “great creative”.
Before you can properly evaluate your creative tests and identify possible winning creatives, you need to first establish a creative baseline which basically tells you exactly what the average of your creatives' performance has been over the past.
Those metrics are set to surpass. Every creative that outperforms those metrics in the future is what we call an “A-grade” creative. In case you want to know more about creative evaluation and how to actually identify a winning creative, click Here to get to our blog article about our adjusted AIDA framework and how we evaluate our creative’s performance.
If you are already familiar with this article, then make sure to click Here to copy our Google spreadsheet on establishing your account baseline metric and then compare your future creative test results relative to this.
Before we start diving into this section - I want you to know that you could write a complete guide or book on proper hypothesis setting and testing, so please take this with a grain of salt, however, I’ll do my best to lay out the fundamentals.
Hypothesis setting is a really important part of your creative testing. After you have run your first tests and analysed their results, now it’s time to form a proper hypothesis. But what is a hypothesis in particular.
A hypothesis gets everyone aligned. It describes a problem, a proposed solution and predicts an outcome.
An example of a hypothesis would be as follows: “We have a below average CTR on our creative A, when we emphasise our CTA-button and the benefit at the end of our video, then we will see an increase in CTR.”
It helps to formulate your hypothesis in an action/consequence format: If [we do this action], then [our audience will behave this way].
Hypothesis setting is the art of combining analytical knowledge with consumer psychology. You always want to back up your creative decision-making process by data.
Basically, when you come up with a hypothesis, you come up with problems, and then you create a hypothesis on these problems and why they are happening to try to solve them to see if they are true or not.
“Statistical significance is the likelihood that the difference in conversion rates between a given variation and the baseline is not due to random chance”
It doesn’t matter whether it’s with Facebook advertising or any other type of testing and optimisation, it is crucial to measure your results always against your sample size.
For people that are not too familiar with statistical fundamentals, you can think about statistical significance as follows:
Scenario 1: You spent $10.000 on creative A and that resulted in 10.000 clicks at a CTR of 2.5% and 250 purchases which lead to an overall Return-On-Ad-Spend (ROAS) of 2.5x.
Scenario 2: You spent $100 on creative B that resulted in 200 clicks at a CTR of 5% and 5 purchases which lead to an overall Return-On-Ad-Spend (ROAS) of 5x.
Objectively speaking, you might argue that creative B outperformed creative A, however, since the sample size is a lot smaller the results of this creative can become a lot more volatile and are not as reliable.
I hope I was able to get my point across, if not make sure to check out this blog article on statistical significance to understand it thoroughly.
There are a lot of calculators and formulas out there that we also use and I am also going to share with you in a minute, however, it is important to be aware of the fact that this is just hypothetical, at least to a certain extent. Nobody can ever be a 100% certain that the creative A is going to outperform creative B forever. You'll always have to deal with a certain level of uncertainty. However, the higher your statistical significance, the lower your uncertainty.
Therefore you always need to take into consideration the variance and standard deviation from your tests. In plain english - you always need to make sure to calculate with possible errors.
There are a lot of advertisers out there that have simplified their definition of statistical significance and just set themselves a static sample size “goal” to then be able to identify this cohort.
A common principle for example in the Facebook marketing world is to run a creative test until it gets 1000 impressions. Although this does have some logic behind it, it comes with certain limitations, since those type of “static goals” do not take things like average order value or conversion rate into account.
Therefore, we at VictoryMedia have a relative sample size “goal” that is relative to the account at hand. We take average order value and the average conversion rate into consideration and based upon those metrics we are then able to set ourselves a threshold to when to complete and evaluate creative tests.
Normally, this is either until we have spent 2x the average order value on one creative or when we have a high average order value at hand >$100, then we go ahead and take the conversion rate into calculation to see how many visitors the website needs to make one sale.
After people have realised the importance of testing, they often want and also actually run a lot of tests, however, they often miss out on the necessary planning that goes into running those tests.
Every one of you that has already ran a couple of tests within your organisation probably quickly noticed that you lose sight of what you are actually testing and trying to achieve and when you don’t know what metric you are trying to improve in particular, well, then it becomes particularly difficult to measure that metric and with it the success of your test.
Therefore, every time before you run a test you want layout a rough plan and do some decent scenario planning.
We at VictoryMedia use this testing framework worksheet that we always attach to our creative tests to keep track of what we are trying to achieve with this test.
“It doesn’t matter if a test fails or wins, in both scenarios you have learned something about your creative, audience and strategy. The only scenario where you lose either way is when you don’t gather your learning from a test to then implement them going forward.”
5. Not taking a look at the right metrics
Long story short - you can have the best plans for your creative testing & the best creative solutions, however, if you don’t take a look at the right metrics that are actually correlated with the performance of your creative, then you won’t be able to make the right decision which will lead to the long-term growth of your marketing success.
We wrote a complete blog article on which metrics we use to properly evaluate our creatives’ performance and make sure that we are actually improving it.
Click here if you want to know more about our adjusted AIDA model and to see how you can track metrics that will lead to building a sustainable testing framework within your ad account.
Get a free 30 minute marketing consultation with a Facebook growth expert
Are you running Facebook Ads without testing creatives? How is that working out for you?
It’s been proven over and over again that Facebook creative is what moves the needle the most when it comes to the success of your paid social marketing campaigns. "Always be testing”: the number one rule when it comes to paid advertising and digital marketing in general.
"Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day"
- Jeff Bezos
There’s a direct correlation between testing and the success of your business in general. When you test you learn, cut out what is not working and be more consumer-centric.
However, how do you actually test properly? How can you make sure that you are conducting your test in the “right way” or evaluate your test properly.
All this and more, we are going to cover in this article. This article is perfect for you if you meet one of the following criteria:
We are going to start out by talking about the importance of testing, however, if you know that you already need to be testing constantly, then feel free to skip ahead to your most interesting section.
This is our agenda. Scroll ahead 👇🏼
It doesn’t matter what your company is all about, testing will be directly related to the growth within your company. It doesn’t matter whether it's Amazon, Tesla or Twitter an increase in growth rate could always be attributed to a certain test they have launched.
Testing lets advertisers uncover what is converting for their target audience and therefore they can then iterate and make sure to have a more appealing messaging which will then lead to a decrease in cost per action.
Often when entrepreneurs or companies say that Facebook Ads don’t work for them, they have just not tested enough variables and variants to determine what is working and double-down on that.
When people think that they are already testing, but they feel like that their performance is not improving even on a long-term basis, then the source reason for that normally is that they are trying to do too much at once. They try to test too many things and therefore then lose sight of the one specific adjustment and aren’t able to attribute that uplift to a particular adjustment.
Therefore, let’s talk about the different elements of an ad that you can test to increase user behavior and actions and how each and every one of them has a different impact on performance.
Primary Text:
Often referred to as the ad copy or body text, this is normally the first or second thing that your prospect sees, depending on whether the actual creative catches their eye first.
Facebook recommends to have it less than 125 characters, however, as with every copy here it’s about testing. You have to consider your audience as well as the point in the customer journey you are targeting this ad to, meaning retargeting or cold.
Either way, here you go with some best practices:
Ad Creative:
This is your most important element within your creative that moves the needle the most, although it is an unproven theory that videos convert better than images, this is also something that should be tested.
In your ad creative you should really get across the unique value proposition, however, make sure to keep in mind that you are not necessarily trying to sell your product here. This is what you do on your product page. With your Facebook Ad Creative, you are solely trying to spike enough desire to make sure to what we like to call “sell the click”.
There’s an avalanche of information that we could unload right now, but for now, we can boil our basic specifications down to:
Headline:
If there’s one thing that you want your prospect to take away from your ad then it should go into the headline. You want to make sure to emphasize why they should click on your ad and check out your product or brand right now.
However, keep in mind that this ad copy accounts for a very small percentage of the overall effectiveness of your ad. 25-50 characters will make sure that it won’t get cut off.
Description:
This is the text that appears right underneath your headline. It’s one of the limited places within your ad that is optional.
Your description should normally support your headline, but it can be used in a number of different ways. Just ask yourself: “What would be the last tipping point for somebody to click on my ad right now and convert?”
Here you go with some examples:
Call-To-Action-Button
If your ad has successfully done the job and spiked the interest of your prospect, then this button will determine the next action.
It has been proven in various studies that the “Shop Now”-button, although it sometimes leads to a lower CTR-rates, normally converts the most people into actual customers.
The following are the most common CTA-buttons that we at VictoryMedia use:
We see it day in, day out: We onboard a new account and take a look at their prior account performance and tests. 9 times out of ten we then notice that they have been launching a lot of tests, but without any structure.
In order to successfully grow your campaign’s performance with testing, it is inevitable that you need to prioritize your tests.
So, you need to ask yourself: “What creative element has the most impact on my overall campaign’s success?” To save you some time: It is your ad creative. Especially, when you are on a lower budget, you want to first and relentlessly test your ad creative.
Your scroll-stopper, different creative angles and ultimately your type of creative, meaning, whether UGC-content, a static image or a GIF performs better within your account and engages the most with your target audience.
Normally the first test that we launch at VictoryMedia is a “type of creative”-test. We analyse the former account performance and the best-performing creatives. Then we go ahead and test that against other types of creatives that we have at hand for this account.
Let’s use a quick example for a skincare brand: Let’s imagine that based on our qualitative research we found out that it’s important for our audience that they don’t need too much time to actually use our skincare products. We would then take that angle and craft four different types of creatives for this angle.
Based on the prior performance of their ad account: UGC-content worked really well. We would then create one UGC-creative, another product demonstration video, a comparison creative and a static image and run them against each other.
So, congrats now you know why you should and what you should be testing first, but you might be wondering: “How can I actually launch those tests?”
There are three major media buying strategies on how you can structure your testing framework, so let’s dive into each of them since they all have their pros and cons…
With this technique you simply add multiple ad creatives within one ad set that has the same targeting options.
Strictly seen this is not necessarily split-testing, since your audience will not be split and your ads will not be displayed evenly.
“You won’t have an equal distribution of traffic across all variants, but you get a good enough read of which ad is better.”
– Shamanth Rao, VP Growth & UA at FreshPlanet
Pros:
Facebook will optimize and deliver your ad based upon your CTR (click-through-rate) and CPA (cost-per-action). This will give you an optimisation at the ad level. You are also in full control, meaning, you can manually see the performance of your ad creative and make decisions based on those metrics that will not directly affect the delivery of your ad set.
Cons:
Your optimisation advantage is also your drawback. Since Facebook will distribute your ad set budget across the different ad creatives within your ad set based on their evaluation of the possible performance, you run at risk that Facebook dials into one creative pretty soon and that the other test won’t get any significant results or spend.
However, this is a manageable drawback: You can simply turn off your creative after you have successfully reached a level of statistical significance for a particular creative, so Facebook is forced to spend more money on the other creatives. You’ll then just compare the different performances after you have successfully gathered enough spend for each creative that you have initially launched.
Facebook offers you a native split-testing function where you can create a different ad set and the ad set has one different variable.
Other than with the manual way of testing your creative, Facebook now splits your audience into different groups that do not overlap, so you have a 100% accurate data set. Also, Facebook calculates the statistical significance for you, so you can make sure that your results are reliable.
With that native feature from Facebook you can test the following variables:
Pros:
The split-testing feature of Facebook is an interesting feature, especially regarding its reliability. The accuracy of your results are top-notch, however, this comes at a cost.
Cons:
Since you have to allocate a dedicated budget for it and Facebook needs to get enough results in order to be statistically significant, you’ll need to spend a decent amount of money and time in order to complete that split-test.
This goes against the general maxim of staying agile and moving on to the next test as fast as possible - therefore especially for accounts spending less than six figures per month, we tend to stay away from this methodology.
Conclusion:
There are obviously more ways to split-test your creative, however, every buyer buys differently, but those are the two main strategies that will lead to you being able to make accurate decisions based upon a data set that is not fragmented and therefore provides a level of statistical significance.
At VictoryMedia we normally opt-in for the first methodology, the manual way of adding several ad creative within one ad set. We create what we call a “creative sandbox” campaign which we use to play around with and cycle in new creatives to identify new winning creative which we can then duplicate into our actual scaling campaigns.
Setting an initial creative baseline is crucial to understand the future success of your tests and hypotheses, since you then actively put a measurable metrics behind the vague and subjective term “great creative”.
Before you can properly evaluate your creative tests and identify possible winning creatives, you need to first establish a creative baseline which basically tells you exactly what the average of your creatives' performance has been over the past.
Those metrics are set to surpass. Every creative that outperforms those metrics in the future is what we call an “A-grade” creative. In case you want to know more about creative evaluation and how to actually identify a winning creative, click Here to get to our blog article about our adjusted AIDA framework and how we evaluate our creative’s performance.
If you are already familiar with this article, then make sure to click Here to copy our Google spreadsheet on establishing your account baseline metric and then compare your future creative test results relative to this.
Before we start diving into this section - I want you to know that you could write a complete guide or book on proper hypothesis setting and testing, so please take this with a grain of salt, however, I’ll do my best to lay out the fundamentals.
Hypothesis setting is a really important part of your creative testing. After you have run your first tests and analysed their results, now it’s time to form a proper hypothesis. But what is a hypothesis in particular.
A hypothesis gets everyone aligned. It describes a problem, a proposed solution and predicts an outcome.
An example of a hypothesis would be as follows: “We have a below average CTR on our creative A, when we emphasise our CTA-button and the benefit at the end of our video, then we will see an increase in CTR.”
It helps to formulate your hypothesis in an action/consequence format: If [we do this action], then [our audience will behave this way].
Hypothesis setting is the art of combining analytical knowledge with consumer psychology. You always want to back up your creative decision-making process by data.
Basically, when you come up with a hypothesis, you come up with problems, and then you create a hypothesis on these problems and why they are happening to try to solve them to see if they are true or not.
“Statistical significance is the likelihood that the difference in conversion rates between a given variation and the baseline is not due to random chance”
It doesn’t matter whether it’s with Facebook advertising or any other type of testing and optimisation, it is crucial to measure your results always against your sample size.
For people that are not too familiar with statistical fundamentals, you can think about statistical significance as follows:
Scenario 1: You spent $10.000 on creative A and that resulted in 10.000 clicks at a CTR of 2.5% and 250 purchases which lead to an overall Return-On-Ad-Spend (ROAS) of 2.5x.
Scenario 2: You spent $100 on creative B that resulted in 200 clicks at a CTR of 5% and 5 purchases which lead to an overall Return-On-Ad-Spend (ROAS) of 5x.
Objectively speaking, you might argue that creative B outperformed creative A, however, since the sample size is a lot smaller the results of this creative can become a lot more volatile and are not as reliable.
I hope I was able to get my point across, if not make sure to check out this blog article on statistical significance to understand it thoroughly.
There are a lot of calculators and formulas out there that we also use and I am also going to share with you in a minute, however, it is important to be aware of the fact that this is just hypothetical, at least to a certain extent. Nobody can ever be a 100% certain that the creative A is going to outperform creative B forever. You'll always have to deal with a certain level of uncertainty. However, the higher your statistical significance, the lower your uncertainty.
Therefore you always need to take into consideration the variance and standard deviation from your tests. In plain english - you always need to make sure to calculate with possible errors.
There are a lot of advertisers out there that have simplified their definition of statistical significance and just set themselves a static sample size “goal” to then be able to identify this cohort.
A common principle for example in the Facebook marketing world is to run a creative test until it gets 1000 impressions. Although this does have some logic behind it, it comes with certain limitations, since those type of “static goals” do not take things like average order value or conversion rate into account.
Therefore, we at VictoryMedia have a relative sample size “goal” that is relative to the account at hand. We take average order value and the average conversion rate into consideration and based upon those metrics we are then able to set ourselves a threshold to when to complete and evaluate creative tests.
Normally, this is either until we have spent 2x the average order value on one creative or when we have a high average order value at hand >$100, then we go ahead and take the conversion rate into calculation to see how many visitors the website needs to make one sale.
After people have realised the importance of testing, they often want and also actually run a lot of tests, however, they often miss out on the necessary planning that goes into running those tests.
Every one of you that has already ran a couple of tests within your organisation probably quickly noticed that you lose sight of what you are actually testing and trying to achieve and when you don’t know what metric you are trying to improve in particular, well, then it becomes particularly difficult to measure that metric and with it the success of your test.
Therefore, every time before you run a test you want layout a rough plan and do some decent scenario planning.
We at VictoryMedia use this testing framework worksheet that we always attach to our creative tests to keep track of what we are trying to achieve with this test.
“It doesn’t matter if a test fails or wins, in both scenarios you have learned something about your creative, audience and strategy. The only scenario where you lose either way is when you don’t gather your learning from a test to then implement them going forward.”
5. Not taking a look at the right metrics
Long story short - you can have the best plans for your creative testing & the best creative solutions, however, if you don’t take a look at the right metrics that are actually correlated with the performance of your creative, then you won’t be able to make the right decision which will lead to the long-term growth of your marketing success.
We wrote a complete blog article on which metrics we use to properly evaluate our creatives’ performance and make sure that we are actually improving it.
Click here if you want to know more about our adjusted AIDA model and to see how you can track metrics that will lead to building a sustainable testing framework within your ad account.
Are you running Facebook Ads without testing creatives? How is that working out for you?
It’s been proven over and over again that Facebook creative is what moves the needle the most when it comes to the success of your paid social marketing campaigns. "Always be testing”: the number one rule when it comes to paid advertising and digital marketing in general.
"Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day"
- Jeff Bezos
There’s a direct correlation between testing and the success of your business in general. When you test you learn, cut out what is not working and be more consumer-centric.
However, how do you actually test properly? How can you make sure that you are conducting your test in the “right way” or evaluate your test properly.
All this and more, we are going to cover in this article. This article is perfect for you if you meet one of the following criteria:
We are going to start out by talking about the importance of testing, however, if you know that you already need to be testing constantly, then feel free to skip ahead to your most interesting section.
This is our agenda. Scroll ahead 👇🏼
It doesn’t matter what your company is all about, testing will be directly related to the growth within your company. It doesn’t matter whether it's Amazon, Tesla or Twitter an increase in growth rate could always be attributed to a certain test they have launched.
Testing lets advertisers uncover what is converting for their target audience and therefore they can then iterate and make sure to have a more appealing messaging which will then lead to a decrease in cost per action.
Often when entrepreneurs or companies say that Facebook Ads don’t work for them, they have just not tested enough variables and variants to determine what is working and double-down on that.
When people think that they are already testing, but they feel like that their performance is not improving even on a long-term basis, then the source reason for that normally is that they are trying to do too much at once. They try to test too many things and therefore then lose sight of the one specific adjustment and aren’t able to attribute that uplift to a particular adjustment.
Therefore, let’s talk about the different elements of an ad that you can test to increase user behavior and actions and how each and every one of them has a different impact on performance.
Primary Text:
Often referred to as the ad copy or body text, this is normally the first or second thing that your prospect sees, depending on whether the actual creative catches their eye first.
Facebook recommends to have it less than 125 characters, however, as with every copy here it’s about testing. You have to consider your audience as well as the point in the customer journey you are targeting this ad to, meaning retargeting or cold.
Either way, here you go with some best practices:
Ad Creative:
This is your most important element within your creative that moves the needle the most, although it is an unproven theory that videos convert better than images, this is also something that should be tested.
In your ad creative you should really get across the unique value proposition, however, make sure to keep in mind that you are not necessarily trying to sell your product here. This is what you do on your product page. With your Facebook Ad Creative, you are solely trying to spike enough desire to make sure to what we like to call “sell the click”.
There’s an avalanche of information that we could unload right now, but for now, we can boil our basic specifications down to:
Headline:
If there’s one thing that you want your prospect to take away from your ad then it should go into the headline. You want to make sure to emphasize why they should click on your ad and check out your product or brand right now.
However, keep in mind that this ad copy accounts for a very small percentage of the overall effectiveness of your ad. 25-50 characters will make sure that it won’t get cut off.
Description:
This is the text that appears right underneath your headline. It’s one of the limited places within your ad that is optional.
Your description should normally support your headline, but it can be used in a number of different ways. Just ask yourself: “What would be the last tipping point for somebody to click on my ad right now and convert?”
Here you go with some examples:
Call-To-Action-Button
If your ad has successfully done the job and spiked the interest of your prospect, then this button will determine the next action.
It has been proven in various studies that the “Shop Now”-button, although it sometimes leads to a lower CTR-rates, normally converts the most people into actual customers.
The following are the most common CTA-buttons that we at VictoryMedia use:
We see it day in, day out: We onboard a new account and take a look at their prior account performance and tests. 9 times out of ten we then notice that they have been launching a lot of tests, but without any structure.
In order to successfully grow your campaign’s performance with testing, it is inevitable that you need to prioritize your tests.
So, you need to ask yourself: “What creative element has the most impact on my overall campaign’s success?” To save you some time: It is your ad creative. Especially, when you are on a lower budget, you want to first and relentlessly test your ad creative.
Your scroll-stopper, different creative angles and ultimately your type of creative, meaning, whether UGC-content, a static image or a GIF performs better within your account and engages the most with your target audience.
Normally the first test that we launch at VictoryMedia is a “type of creative”-test. We analyse the former account performance and the best-performing creatives. Then we go ahead and test that against other types of creatives that we have at hand for this account.
Let’s use a quick example for a skincare brand: Let’s imagine that based on our qualitative research we found out that it’s important for our audience that they don’t need too much time to actually use our skincare products. We would then take that angle and craft four different types of creatives for this angle.
Based on the prior performance of their ad account: UGC-content worked really well. We would then create one UGC-creative, another product demonstration video, a comparison creative and a static image and run them against each other.
So, congrats now you know why you should and what you should be testing first, but you might be wondering: “How can I actually launch those tests?”
There are three major media buying strategies on how you can structure your testing framework, so let’s dive into each of them since they all have their pros and cons…
With this technique you simply add multiple ad creatives within one ad set that has the same targeting options.
Strictly seen this is not necessarily split-testing, since your audience will not be split and your ads will not be displayed evenly.
“You won’t have an equal distribution of traffic across all variants, but you get a good enough read of which ad is better.”
– Shamanth Rao, VP Growth & UA at FreshPlanet
Pros:
Facebook will optimize and deliver your ad based upon your CTR (click-through-rate) and CPA (cost-per-action). This will give you an optimisation at the ad level. You are also in full control, meaning, you can manually see the performance of your ad creative and make decisions based on those metrics that will not directly affect the delivery of your ad set.
Cons:
Your optimisation advantage is also your drawback. Since Facebook will distribute your ad set budget across the different ad creatives within your ad set based on their evaluation of the possible performance, you run at risk that Facebook dials into one creative pretty soon and that the other test won’t get any significant results or spend.
However, this is a manageable drawback: You can simply turn off your creative after you have successfully reached a level of statistical significance for a particular creative, so Facebook is forced to spend more money on the other creatives. You’ll then just compare the different performances after you have successfully gathered enough spend for each creative that you have initially launched.
Facebook offers you a native split-testing function where you can create a different ad set and the ad set has one different variable.
Other than with the manual way of testing your creative, Facebook now splits your audience into different groups that do not overlap, so you have a 100% accurate data set. Also, Facebook calculates the statistical significance for you, so you can make sure that your results are reliable.
With that native feature from Facebook you can test the following variables:
Pros:
The split-testing feature of Facebook is an interesting feature, especially regarding its reliability. The accuracy of your results are top-notch, however, this comes at a cost.
Cons:
Since you have to allocate a dedicated budget for it and Facebook needs to get enough results in order to be statistically significant, you’ll need to spend a decent amount of money and time in order to complete that split-test.
This goes against the general maxim of staying agile and moving on to the next test as fast as possible - therefore especially for accounts spending less than six figures per month, we tend to stay away from this methodology.
Conclusion:
There are obviously more ways to split-test your creative, however, every buyer buys differently, but those are the two main strategies that will lead to you being able to make accurate decisions based upon a data set that is not fragmented and therefore provides a level of statistical significance.
At VictoryMedia we normally opt-in for the first methodology, the manual way of adding several ad creative within one ad set. We create what we call a “creative sandbox” campaign which we use to play around with and cycle in new creatives to identify new winning creative which we can then duplicate into our actual scaling campaigns.
Setting an initial creative baseline is crucial to understand the future success of your tests and hypotheses, since you then actively put a measurable metrics behind the vague and subjective term “great creative”.
Before you can properly evaluate your creative tests and identify possible winning creatives, you need to first establish a creative baseline which basically tells you exactly what the average of your creatives' performance has been over the past.
Those metrics are set to surpass. Every creative that outperforms those metrics in the future is what we call an “A-grade” creative. In case you want to know more about creative evaluation and how to actually identify a winning creative, click Here to get to our blog article about our adjusted AIDA framework and how we evaluate our creative’s performance.
If you are already familiar with this article, then make sure to click Here to copy our Google spreadsheet on establishing your account baseline metric and then compare your future creative test results relative to this.
Before we start diving into this section - I want you to know that you could write a complete guide or book on proper hypothesis setting and testing, so please take this with a grain of salt, however, I’ll do my best to lay out the fundamentals.
Hypothesis setting is a really important part of your creative testing. After you have run your first tests and analysed their results, now it’s time to form a proper hypothesis. But what is a hypothesis in particular.
A hypothesis gets everyone aligned. It describes a problem, a proposed solution and predicts an outcome.
An example of a hypothesis would be as follows: “We have a below average CTR on our creative A, when we emphasise our CTA-button and the benefit at the end of our video, then we will see an increase in CTR.”
It helps to formulate your hypothesis in an action/consequence format: If [we do this action], then [our audience will behave this way].
Hypothesis setting is the art of combining analytical knowledge with consumer psychology. You always want to back up your creative decision-making process by data.
Basically, when you come up with a hypothesis, you come up with problems, and then you create a hypothesis on these problems and why they are happening to try to solve them to see if they are true or not.
“Statistical significance is the likelihood that the difference in conversion rates between a given variation and the baseline is not due to random chance”
It doesn’t matter whether it’s with Facebook advertising or any other type of testing and optimisation, it is crucial to measure your results always against your sample size.
For people that are not too familiar with statistical fundamentals, you can think about statistical significance as follows:
Scenario 1: You spent $10.000 on creative A and that resulted in 10.000 clicks at a CTR of 2.5% and 250 purchases which lead to an overall Return-On-Ad-Spend (ROAS) of 2.5x.
Scenario 2: You spent $100 on creative B that resulted in 200 clicks at a CTR of 5% and 5 purchases which lead to an overall Return-On-Ad-Spend (ROAS) of 5x.
Objectively speaking, you might argue that creative B outperformed creative A, however, since the sample size is a lot smaller the results of this creative can become a lot more volatile and are not as reliable.
I hope I was able to get my point across, if not make sure to check out this blog article on statistical significance to understand it thoroughly.
There are a lot of calculators and formulas out there that we also use and I am also going to share with you in a minute, however, it is important to be aware of the fact that this is just hypothetical, at least to a certain extent. Nobody can ever be a 100% certain that the creative A is going to outperform creative B forever. You'll always have to deal with a certain level of uncertainty. However, the higher your statistical significance, the lower your uncertainty.
Therefore you always need to take into consideration the variance and standard deviation from your tests. In plain english - you always need to make sure to calculate with possible errors.
There are a lot of advertisers out there that have simplified their definition of statistical significance and just set themselves a static sample size “goal” to then be able to identify this cohort.
A common principle for example in the Facebook marketing world is to run a creative test until it gets 1000 impressions. Although this does have some logic behind it, it comes with certain limitations, since those type of “static goals” do not take things like average order value or conversion rate into account.
Therefore, we at VictoryMedia have a relative sample size “goal” that is relative to the account at hand. We take average order value and the average conversion rate into consideration and based upon those metrics we are then able to set ourselves a threshold to when to complete and evaluate creative tests.
Normally, this is either until we have spent 2x the average order value on one creative or when we have a high average order value at hand >$100, then we go ahead and take the conversion rate into calculation to see how many visitors the website needs to make one sale.
After people have realised the importance of testing, they often want and also actually run a lot of tests, however, they often miss out on the necessary planning that goes into running those tests.
Every one of you that has already ran a couple of tests within your organisation probably quickly noticed that you lose sight of what you are actually testing and trying to achieve and when you don’t know what metric you are trying to improve in particular, well, then it becomes particularly difficult to measure that metric and with it the success of your test.
Therefore, every time before you run a test you want layout a rough plan and do some decent scenario planning.
We at VictoryMedia use this testing framework worksheet that we always attach to our creative tests to keep track of what we are trying to achieve with this test.
“It doesn’t matter if a test fails or wins, in both scenarios you have learned something about your creative, audience and strategy. The only scenario where you lose either way is when you don’t gather your learning from a test to then implement them going forward.”
5. Not taking a look at the right metrics
Long story short - you can have the best plans for your creative testing & the best creative solutions, however, if you don’t take a look at the right metrics that are actually correlated with the performance of your creative, then you won’t be able to make the right decision which will lead to the long-term growth of your marketing success.
We wrote a complete blog article on which metrics we use to properly evaluate our creatives’ performance and make sure that we are actually improving it.
Click here if you want to know more about our adjusted AIDA model and to see how you can track metrics that will lead to building a sustainable testing framework within your ad account.
Are you running Facebook Ads without testing creatives? How is that working out for you?
It’s been proven over and over again that Facebook creative is what moves the needle the most when it comes to the success of your paid social marketing campaigns. "Always be testing”: the number one rule when it comes to paid advertising and digital marketing in general.
"Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day"
- Jeff Bezos
There’s a direct correlation between testing and the success of your business in general. When you test you learn, cut out what is not working and be more consumer-centric.
However, how do you actually test properly? How can you make sure that you are conducting your test in the “right way” or evaluate your test properly.
All this and more, we are going to cover in this article. This article is perfect for you if you meet one of the following criteria:
We are going to start out by talking about the importance of testing, however, if you know that you already need to be testing constantly, then feel free to skip ahead to your most interesting section.
This is our agenda. Scroll ahead 👇🏼
It doesn’t matter what your company is all about, testing will be directly related to the growth within your company. It doesn’t matter whether it's Amazon, Tesla or Twitter an increase in growth rate could always be attributed to a certain test they have launched.
Testing lets advertisers uncover what is converting for their target audience and therefore they can then iterate and make sure to have a more appealing messaging which will then lead to a decrease in cost per action.
Often when entrepreneurs or companies say that Facebook Ads don’t work for them, they have just not tested enough variables and variants to determine what is working and double-down on that.
When people think that they are already testing, but they feel like that their performance is not improving even on a long-term basis, then the source reason for that normally is that they are trying to do too much at once. They try to test too many things and therefore then lose sight of the one specific adjustment and aren’t able to attribute that uplift to a particular adjustment.
Therefore, let’s talk about the different elements of an ad that you can test to increase user behavior and actions and how each and every one of them has a different impact on performance.
Primary Text:
Often referred to as the ad copy or body text, this is normally the first or second thing that your prospect sees, depending on whether the actual creative catches their eye first.
Facebook recommends to have it less than 125 characters, however, as with every copy here it’s about testing. You have to consider your audience as well as the point in the customer journey you are targeting this ad to, meaning retargeting or cold.
Either way, here you go with some best practices:
Ad Creative:
This is your most important element within your creative that moves the needle the most, although it is an unproven theory that videos convert better than images, this is also something that should be tested.
In your ad creative you should really get across the unique value proposition, however, make sure to keep in mind that you are not necessarily trying to sell your product here. This is what you do on your product page. With your Facebook Ad Creative, you are solely trying to spike enough desire to make sure to what we like to call “sell the click”.
There’s an avalanche of information that we could unload right now, but for now, we can boil our basic specifications down to:
Headline:
If there’s one thing that you want your prospect to take away from your ad then it should go into the headline. You want to make sure to emphasize why they should click on your ad and check out your product or brand right now.
However, keep in mind that this ad copy accounts for a very small percentage of the overall effectiveness of your ad. 25-50 characters will make sure that it won’t get cut off.
Description:
This is the text that appears right underneath your headline. It’s one of the limited places within your ad that is optional.
Your description should normally support your headline, but it can be used in a number of different ways. Just ask yourself: “What would be the last tipping point for somebody to click on my ad right now and convert?”
Here you go with some examples:
Call-To-Action-Button
If your ad has successfully done the job and spiked the interest of your prospect, then this button will determine the next action.
It has been proven in various studies that the “Shop Now”-button, although it sometimes leads to a lower CTR-rates, normally converts the most people into actual customers.
The following are the most common CTA-buttons that we at VictoryMedia use:
We see it day in, day out: We onboard a new account and take a look at their prior account performance and tests. 9 times out of ten we then notice that they have been launching a lot of tests, but without any structure.
In order to successfully grow your campaign’s performance with testing, it is inevitable that you need to prioritize your tests.
So, you need to ask yourself: “What creative element has the most impact on my overall campaign’s success?” To save you some time: It is your ad creative. Especially, when you are on a lower budget, you want to first and relentlessly test your ad creative.
Your scroll-stopper, different creative angles and ultimately your type of creative, meaning, whether UGC-content, a static image or a GIF performs better within your account and engages the most with your target audience.
Normally the first test that we launch at VictoryMedia is a “type of creative”-test. We analyse the former account performance and the best-performing creatives. Then we go ahead and test that against other types of creatives that we have at hand for this account.
Let’s use a quick example for a skincare brand: Let’s imagine that based on our qualitative research we found out that it’s important for our audience that they don’t need too much time to actually use our skincare products. We would then take that angle and craft four different types of creatives for this angle.
Based on the prior performance of their ad account: UGC-content worked really well. We would then create one UGC-creative, another product demonstration video, a comparison creative and a static image and run them against each other.
So, congrats now you know why you should and what you should be testing first, but you might be wondering: “How can I actually launch those tests?”
There are three major media buying strategies on how you can structure your testing framework, so let’s dive into each of them since they all have their pros and cons…
With this technique you simply add multiple ad creatives within one ad set that has the same targeting options.
Strictly seen this is not necessarily split-testing, since your audience will not be split and your ads will not be displayed evenly.
“You won’t have an equal distribution of traffic across all variants, but you get a good enough read of which ad is better.”
– Shamanth Rao, VP Growth & UA at FreshPlanet
Pros:
Facebook will optimize and deliver your ad based upon your CTR (click-through-rate) and CPA (cost-per-action). This will give you an optimisation at the ad level. You are also in full control, meaning, you can manually see the performance of your ad creative and make decisions based on those metrics that will not directly affect the delivery of your ad set.
Cons:
Your optimisation advantage is also your drawback. Since Facebook will distribute your ad set budget across the different ad creatives within your ad set based on their evaluation of the possible performance, you run at risk that Facebook dials into one creative pretty soon and that the other test won’t get any significant results or spend.
However, this is a manageable drawback: You can simply turn off your creative after you have successfully reached a level of statistical significance for a particular creative, so Facebook is forced to spend more money on the other creatives. You’ll then just compare the different performances after you have successfully gathered enough spend for each creative that you have initially launched.
Facebook offers you a native split-testing function where you can create a different ad set and the ad set has one different variable.
Other than with the manual way of testing your creative, Facebook now splits your audience into different groups that do not overlap, so you have a 100% accurate data set. Also, Facebook calculates the statistical significance for you, so you can make sure that your results are reliable.
With that native feature from Facebook you can test the following variables:
Pros:
The split-testing feature of Facebook is an interesting feature, especially regarding its reliability. The accuracy of your results are top-notch, however, this comes at a cost.
Cons:
Since you have to allocate a dedicated budget for it and Facebook needs to get enough results in order to be statistically significant, you’ll need to spend a decent amount of money and time in order to complete that split-test.
This goes against the general maxim of staying agile and moving on to the next test as fast as possible - therefore especially for accounts spending less than six figures per month, we tend to stay away from this methodology.
Conclusion:
There are obviously more ways to split-test your creative, however, every buyer buys differently, but those are the two main strategies that will lead to you being able to make accurate decisions based upon a data set that is not fragmented and therefore provides a level of statistical significance.
At VictoryMedia we normally opt-in for the first methodology, the manual way of adding several ad creative within one ad set. We create what we call a “creative sandbox” campaign which we use to play around with and cycle in new creatives to identify new winning creative which we can then duplicate into our actual scaling campaigns.
Setting an initial creative baseline is crucial to understand the future success of your tests and hypotheses, since you then actively put a measurable metrics behind the vague and subjective term “great creative”.
Before you can properly evaluate your creative tests and identify possible winning creatives, you need to first establish a creative baseline which basically tells you exactly what the average of your creatives' performance has been over the past.
Those metrics are set to surpass. Every creative that outperforms those metrics in the future is what we call an “A-grade” creative. In case you want to know more about creative evaluation and how to actually identify a winning creative, click Here to get to our blog article about our adjusted AIDA framework and how we evaluate our creative’s performance.
If you are already familiar with this article, then make sure to click Here to copy our Google spreadsheet on establishing your account baseline metric and then compare your future creative test results relative to this.
Before we start diving into this section - I want you to know that you could write a complete guide or book on proper hypothesis setting and testing, so please take this with a grain of salt, however, I’ll do my best to lay out the fundamentals.
Hypothesis setting is a really important part of your creative testing. After you have run your first tests and analysed their results, now it’s time to form a proper hypothesis. But what is a hypothesis in particular.
A hypothesis gets everyone aligned. It describes a problem, a proposed solution and predicts an outcome.
An example of a hypothesis would be as follows: “We have a below average CTR on our creative A, when we emphasise our CTA-button and the benefit at the end of our video, then we will see an increase in CTR.”
It helps to formulate your hypothesis in an action/consequence format: If [we do this action], then [our audience will behave this way].
Hypothesis setting is the art of combining analytical knowledge with consumer psychology. You always want to back up your creative decision-making process by data.
Basically, when you come up with a hypothesis, you come up with problems, and then you create a hypothesis on these problems and why they are happening to try to solve them to see if they are true or not.
“Statistical significance is the likelihood that the difference in conversion rates between a given variation and the baseline is not due to random chance”
It doesn’t matter whether it’s with Facebook advertising or any other type of testing and optimisation, it is crucial to measure your results always against your sample size.
For people that are not too familiar with statistical fundamentals, you can think about statistical significance as follows:
Scenario 1: You spent $10.000 on creative A and that resulted in 10.000 clicks at a CTR of 2.5% and 250 purchases which lead to an overall Return-On-Ad-Spend (ROAS) of 2.5x.
Scenario 2: You spent $100 on creative B that resulted in 200 clicks at a CTR of 5% and 5 purchases which lead to an overall Return-On-Ad-Spend (ROAS) of 5x.
Objectively speaking, you might argue that creative B outperformed creative A, however, since the sample size is a lot smaller the results of this creative can become a lot more volatile and are not as reliable.
I hope I was able to get my point across, if not make sure to check out this blog article on statistical significance to understand it thoroughly.
There are a lot of calculators and formulas out there that we also use and I am also going to share with you in a minute, however, it is important to be aware of the fact that this is just hypothetical, at least to a certain extent. Nobody can ever be a 100% certain that the creative A is going to outperform creative B forever. You'll always have to deal with a certain level of uncertainty. However, the higher your statistical significance, the lower your uncertainty.
Therefore you always need to take into consideration the variance and standard deviation from your tests. In plain english - you always need to make sure to calculate with possible errors.
There are a lot of advertisers out there that have simplified their definition of statistical significance and just set themselves a static sample size “goal” to then be able to identify this cohort.
A common principle for example in the Facebook marketing world is to run a creative test until it gets 1000 impressions. Although this does have some logic behind it, it comes with certain limitations, since those type of “static goals” do not take things like average order value or conversion rate into account.
Therefore, we at VictoryMedia have a relative sample size “goal” that is relative to the account at hand. We take average order value and the average conversion rate into consideration and based upon those metrics we are then able to set ourselves a threshold to when to complete and evaluate creative tests.
Normally, this is either until we have spent 2x the average order value on one creative or when we have a high average order value at hand >$100, then we go ahead and take the conversion rate into calculation to see how many visitors the website needs to make one sale.
After people have realised the importance of testing, they often want and also actually run a lot of tests, however, they often miss out on the necessary planning that goes into running those tests.
Every one of you that has already ran a couple of tests within your organisation probably quickly noticed that you lose sight of what you are actually testing and trying to achieve and when you don’t know what metric you are trying to improve in particular, well, then it becomes particularly difficult to measure that metric and with it the success of your test.
Therefore, every time before you run a test you want layout a rough plan and do some decent scenario planning.
We at VictoryMedia use this testing framework worksheet that we always attach to our creative tests to keep track of what we are trying to achieve with this test.
“It doesn’t matter if a test fails or wins, in both scenarios you have learned something about your creative, audience and strategy. The only scenario where you lose either way is when you don’t gather your learning from a test to then implement them going forward.”
5. Not taking a look at the right metrics
Long story short - you can have the best plans for your creative testing & the best creative solutions, however, if you don’t take a look at the right metrics that are actually correlated with the performance of your creative, then you won’t be able to make the right decision which will lead to the long-term growth of your marketing success.
We wrote a complete blog article on which metrics we use to properly evaluate our creatives’ performance and make sure that we are actually improving it.
Click here if you want to know more about our adjusted AIDA model and to see how you can track metrics that will lead to building a sustainable testing framework within your ad account.
.png){kind=icon}
.jpg)
