What is CRO testing? A step-by-step guide to running efficient tests

CRO tests improve conversion rates

The most obvious reason why CRO testing is important is because it can help you improve conversion rates. The information you gain from CRO tests is incredibly valuable and lets you target highly specific audiences more accurately.

Case in point: Knowing where your customers are in their buying journey and targeting them accordingly can boost conversion rates by as much as 73%.

CRO tests lead to insights (even when they’re inconclusive)

Knowledge is power—and you can gather plenty of it about your audience from a CRO test, no matter the ultimate outcome.

Landing page traffic data alone can paint a pretty impressive picture, covering geographic location, device type, timezone, and traffic source, to name a few. However, understanding who is converting and what motivates them is invaluable.

Just think about how much energy and data are needed to develop accurate, useful buyer personas in a traditional, non-CRO approach—we’re talkin’ major investment in some cases.

This is why an “inconclusive” CRO test should still be considered a success. Even if it doesn’t confirm your hypothesis, you’ve still got a wealth of information you can use in your next landing page or marketing campaign. 

Find untapped gold mines in the “micro surfaces” in your conversion funnel

Every conversion funnel has tons of “micro surfaces” ripe for optimization.

You might be wondering, “What in the world is a micro surface?” but you can relax as it’s not just another Gen Z term you need to learn. When building a landing page or marketing funnel, micro surfaces are usually elements and sections that don’t get the same level of attention as the hero image, above-the-fold copy, and CTA buttons.

Elements like:

• Form titles and copy
• Thank you pages
• Thank you email subject lines
• Link text in confirmation emails
• Calls to action button text
• Ecommerce payment options

On their own, these pieces probably won’t have a massive impact on conversion rates—but when you start to dig in, test, and optimize these small sections as part of your CRO testing activities, they can drive impressive results.

Recommended reading: Mastering ecommerce conversion rate optimization: expert tips and inspirational examples

Try things out and test assumptions (without taking big risks)

Make no doubt about it—there’s some risk in any marketing activity.

The truth is, any campaign or initiative has a chance of falling short—and that can be stressful. CRO tests allow you to manage and mitigate risk by testing and trying out ideas and assumptions in a more controlled environment. 

Here’s an example: 

You’re investing in a PPC campaign to drive traffic to a landing page. It’s a strong page, but you’re not 100% certain you’ve nailed the messaging that will inspire your audience to convert. The headline and above-the-fold copy you have is okay, but you feel it could be better.

CRO testing lets you test drive different messages for you to see how they work in a controlled environment, without dramatically changing your overall campaign.

Another common example would be an email campaign.

Most email marketing tools give you a straightforward way to test your subject line before the email is sent to the bulk of your list. You can come up with two variations, then send them to 5% of your list. Whichever variation performs the best becomes the subject for the remaining 95%.

If the test doesn’t make a huge impact, no big deal! You haven’t risked your entire project by making massive changes, so you can continue experimenting and testing your ideas without worrying about your main campaign. 

Establish your own benchmarks to test against

We mentioned earlier the median conversion rate for all industries is 4.3%, a stat we explored in our Conversion Benchmark Report.

But what does “good” look like for your industry and audience? 

The truth is, it varies greatly.

For example, the conversion rate for a cybersecurity firm averages 2.0%, while an ecommerce subscription site has an average conversion rate of 8.0%. If you work in food and catering, you’re looking at a 9.8% average. Average conversion rates can vary by industry quite a bit.

Knowing what you are benchmarking your own success against is critical. It helps you develop meaningful KPIs and metrics to accurately measure success and growth, ensuring your efforts are worthwhile.

Through CRO testing, you can establish your own baselines for your landing pages, and then test against those same baselines to try to beat them.

What are the different types of CRO tests?

There are four common types of CRO tests, particularly as they relate to experimenting with your landing pages and conversion funnels.

Split tests or A/B tests

A/B testing, or split testing, is a type of CRO test where you create two variations of your landing page, show them to different parts of your audience, and see which one works best. 

When A/B testing, you’ll start by choosing a control page (your unchanged, original page) and a challenger that differs in a key area. These split tests are a great way to make the most of your marketing efforts when you’re focusing on one variable at a time. With Unbounce’s A/B testing tool, this entire process can run as smooth as butter, too.

A/B/n tests

A/B/n testing follows the same process as an A/B test but with more variations. The “n” in the name represents that undefined number of variants, so you could have an A, B, C, D, and even an E all in the same test. No matter the number of variants, these tests typically focus on comparing a single element.

Multivariate tests

Multivariate testing assesses multiple combinations of page elements at once to determine which combination has the best conversion rate. While it may look like A/B testing at first, the key difference is that each variation may have multiple differences from the original page. These tests rely on a lot of traffic for effective testing and are best suited for established pages that you want to refine further. 

Traffic segmentation

Traffic segmentation is a way of optimizing page performance based on user attributes. For example, you could segment traffic based on geographic location, traffic source (like an ad or social post), the device used to access the page, or additional behaviors they take once they’re on the page. 

You can learn a lot about your audience from this data, letting you further tweak and tailor your page to better reach them. And better yet, you can also run traffic segmentation experiments with the click of a button using our AI-powered Smart Traffic tool. Check out the experiment we ran on a few of our own landing pages that led to a 50% increase in conversions.

Smart Traffic Experiment: How to Get a 50% Conversion Lift

How to run a CRO test

Before digging into any CRO testing, you need to put your marketing lab coat on and get things set up. Make sure you’ve got your tools ready to go and ensure your control group is all set.

Step 1: Brainstorm test ideas

Start with the basics.

What’s the problem you’re trying to solve?

Before generating a formal test hypothesis, identify the challenge you face and brainstorm ideas on how to overcome it. Record them, too—they may spark even more ideas and theories! 

Next, it’s research time. Look internally at any past tests you’ve run, user journey data, and analytics insights. Then, look at your competitors. Have they faced similar challenges? What sort of tactics and strategies are they using to achieve success? Is it working for them? Record all this information and fill in what blanks you can to flesh out your ideas further. 

Step 2: Plan your experiment and define your goals

You’ve got your test idea ready to go—now it’s time to design your experiment. 

This is where you’ll start to formalize your efforts. First, create a document to act as the main source of information on your test, and loop in any stakeholders who need to be involved. 

Your test document should include the following: 

• Your goal: What do you want to accomplish with this CRO test? What is the ultimate conversion goal you’re looking to influence?
• Hypothesis: What do you think will happen with this test? This helps you be as specific as possible—include the current state of your CRO efforts, what you’re testing, how you’re measuring it, and any predictions you have about the results. 
• Test design: What type of test are you running? What pages will it run on? How many variants do you plan on running? How long will the test run for? What are you measuring in this test, and will there be any additional metrics? What results do you predict, and are there any special considerations that should be taken into account? 
• Test results: After the test is done, be sure to include any details about it in this document too. 

Step 3: Build the variants for your test

With all the paperwork out of the way, it’s time to have some fun designing and building the variants your test is going to run on. Put your creative hat on and start building (or start from a template to save time and boost your chances of building a winner).

Focus on conversion-centric design, which means your page should: 

• Create focus for your target audience
• Build a structure that guides your audience to act
• Stay consistent with the ads users are clicking on
• Show benefits that will resonate with your customers
• Draw attention to most important elements
• Build trust by including social proof
• Reduce friction, making it easier for customers to take action

From there, build your landing pages with Unbounce’s drag-and-drop editor before setting up internal tracking.

When you’re ready to launch, run a few QA tests to ensure everything works smoothly. If everything checks out, you’re good to go.

Step 4: Launch your CRO test

Launch your landing page variants to start your CRO test. Once they’re live, take a moment to click through each page yourself as a final check. It’s never a bad idea to do one last QA.

Next, double-check your analytics tools (i.e. Google Analytics) to ensure everything is connected and running smoothly. If your landing page data isn’t showing up in Google Analytics, troubleshoot what’s going on. It may just be a simple integration hiccup.

In Unbounce, all you need to do to connect Google Analytics to your landing pages is add your Google Analytics website tracking ID to the Unbounce Script Manager. 

(Source)

Don’t forget to keep your team in the know the test is running to prevent any confusion. This helps prevent any mixups over metrics that other team members may be tracking and reporting on.

Once your test is live, start to monitor the results and the amount of visits your page is getting to look for signs of statistical significance. As your test reaches a statistical significance, you’re now ready to review your findings.

Step 5: Review the results

First thing’s first, record the data you’ve collected in your experiment document from the planning phase, then start to analyze it. Keep in mind, this is sometimes easier said than done—analyzing data can be tedious, especially as you try to determine what it means for your audience and your efforts. 

That’s why it’s so critical to establish benchmarks and KPIs early on.

Instead of sifting through all the data, you can focus on the specific metrics you were measuring and the impact the test had on them.

From there, ask yourself a few more questions:

• Did the test confirm your hypothesis?
• Did you discover anything new or surprising from your test?

Whether you have a high-performing landing page on your hands or not, the information you gather is immensely valuable. If your control page won, excellent—you’re on the right track! If one of the variant challengers delivered better results, that’s the one you’ll want to implement moving forward.

Step 6: Act on your findings

If there was a clear winner in your CRO test—you have a new control page. That’s the easy part. From there, the job still isn’t done, though. There’s one more crucial step that far too many marketing teams unfortunately skip.

Step 7: Carry your takeaways into new tests

After you’ve implemented your changes on the first page, carry your findings forward. That means running more tests, testing even more pages, and coming up with new hypotheses. This iterative process helps you continuously refine your conversion rate optimization strategy to reach new heights, make data-driven decisions, and drive further business.

Our mantra at Unbounce is clear: Always be testing.

Conversion rate optimization is an ongoing process. There’s always room for improvement, and you should always be on the hunt for insights into user behavior.

CRO test examples and ideas

We’ve talked about what CRO tests look like, why they’re important, and how to run them. But what elements can you test, and what does it look like in action? Let’s start with a real-world example of how powerful CRO testing can be. 

Going is a small business dedicated to helping people travel the world without breaking the bank. They give users access to an alert service for great flight deals at their local airport. The challenge: how do you let people know this awesome service exists?

The company wanted to better understand their audience and reach them in a way that truly resonated. Working with Unbounce, Going used the A/B testing tool to run a simple test.

They tried two different CTAs on their homepage:

• “Sign up for free”
• “Trial for free.” 

This one simple test led to a 104% month-over-month increase in their trial start rate. 

Beyond testing out different CTA text, though, there are plenty of things you can test to drive an increase in conversion rates or click-through rates.

• Headlines and copy: Play around with different headlines, subheads, and copy to tweak your messaging based on what you know about your audience. 
• Graphic elements and layout: Try out different imagery, layouts, and graphic treatments to see what gets more attention. 
• Forms: How long are the forms you’re using? Try testing the number of fields in the forms on your pages, or tweak the field labels and text to try and reduce friction. 
• Social proof: What testimonials and reviews can you highlight on your landing page? Even three small testimonials can earn a page up to 34% more conversions than a page without them. 
• Trust signals: If your business has any certifications, accreditations, or guarantees, it’s worth testing to see what role they play in closing business. 
• Credit card requirements: If you offer free trials, you can experiment with an “opt in” vs an “opt out” model where users either enter credit info at signup or only when they choose to upgrade.

Recommended reading: How to Score Double-Digit Conversion Rates—A Marketing Hero’s Journey

CRO testing tools

Conversion rate optimization tools help you increase conversions by driving more traffic to your site, increasing the number of actions taken, helping you make informed decisions, arming you with user feedback from real users, and much more.

To help you get started, we published a comprehensive list of 32 different CRO tools you can add to your marketing tech stack.

If we want to get more in the weeds, there are a few subcategories to know:

• A/B testing tools: These tools let you try, test, and iterate with easy-to-use A/B testing. If you’re curious, we have a list of the just the best A/B testing tools too.
• Analytics tools: Analytics tools help you evaluate the impact of your work and understand your data better. 
• Landing page tools: Landing page tools help you build pages that serve your different purposes. 
• Lead capture tools: These tools help you collect visitor info, in turn making it easier to understand the customer journey. 
• SEO tools: SEO tools are designed to make it easier to drive organic traffic to your pages, letting you see how you rank in search engines compared to your competitors. 
• User survey tools: Sometimes, the best way to conduct customer research is by asking your users or customers directly. Customer survey tools let you do just that. 
• User behavior analytics tools: Closely related to analytics, user behavior tools like heat maps let you see exactly where users are going on your pages.
• User testing tools: Getting users to test out new products and features, or experience new designs and messaging is another great way to conduct user research. These tools make it easier to connect with users to that end.
• Customer review tools: After a customer completes a purchase, asking them to leave a review can give you authentic social proof to use in your future tests.
• Session recording tools: What if you could watch how a visitor interacts with your landing pages over their shoulder? There are tools for that.
• Form analytics tools: The more data you can collect on how users interact with the elements on your page, the better—form fields included.

6 common CRO testing mistakes to watch for

While we can’t stress enough that an inconclusive test shouldn’t be considered a failure, some common CRO testing mistakes can lead to suboptimal results. 

Mistake #1: Testing without a plan

We talked about the importance of planning and preparing for a CRO test, but we’ll reiterate it here. Jumping into tests without a plan is a recipe for disaster. Take the time to plan out what you’ll be testing and when you’ll be testing it to avoid wasting time and effort. 

Mistake #2: Running big tests out of the gate

Contrary to popular wisdom, you don’t need to go big or go home. As exciting as it is to dive headfirst into all things CRO testing, if you’re just starting out with CRO, a big test out of the gate likely won’t get the results you’re after. 

The reason for this is simple:

You likely don’t have enough existing data to support a test on a larger scale. 

To be clear, there’s nothing wrong with big tests. That said, starting with smaller tests can help you validate your hypotheses and build from there. Soon, you’ll have the baselines you need to tackle the bigger projects.

Mistake #3: Stopping tests early

Time is a valuable resource, especially in marketing. Because of this, we find far too many marketing teams acting too soon when running CRO tests. They want to shift their focus to the next project or the next test, so they call the current test a winner or loser before it’s actually had enough time (and data) to become statistically significant.

Generally speaking, aim to run your CRO tests for at least two weeks to maximize the likelihood of good, accurate data that will benefit your CRO efforts. This isn’t a non-negotiable, and the actual timeframes will vary from one test to the next, but two weeks is a good rule of thumb.

Mistake #4: Not defining a primary metric

If you don’t know your primary metric or primary conversion goal, it will be hard to determine if your test was successful. As you begin planning and researching your test, figure out if what you’re measuring is the thing that will have the most impact on your efforts going forward. 

Mistake #5: Using too small a sample size

Running granular tests on pages with very little traffic is an all-too-common mistake.

The low traffic volume will make it difficult to get reliable results because the test likely won’t have enough data for high confidence levels. You’ll either have to stop the test or run it for longer. Instead, focus on high-traffic pages and let them run until you have the data you need. 

Mistake #6: Chasing winners

If every test you run confirms your hypothesis, then you’re missing out on the deeper value of CRO, and your hypotheses should probably just be incorporated into your processes. Instead, push your tests to try out new ideas and concepts, and really experiment. This will help you better understand your audience and build a more successful sales funnel. 

CRO testing FAQs

How long should you run the CRO test?

The honest answer is that there’s no one-size-fits-all “best” timeframe for a CRO test.

How long your CRO test should run will always depend on the unique variables associated with your specific test. Variables like how much traffic you’re getting, how large of an improvement you’re looking to achieve, how strong your current conversion rate is, and more.

Generally speaking, you should expect to run your test for at least two weeks. If you’re using Unbounce, the best approach would be to monitor the confidence percentages within your dashboard. These percentages help to calculate statistical significance for you so you don’t have to worry about getting the math wrong or calling a test too early.

How do you assess CRO testing results? 

When assessing CRO test results, the conversion rate is usually the primary KPI to determine how the test went. The exact metrics you use to assess CRO results will depend on your overall goals and what you’re measuring for. Other meaningful metrics include bounce rate, average time on page, and revenue generated.

What is the difference between A/B testing and CRO testing? 

Here’s the simplest way to think about it:

A/B testing is a tactical technique, while CRO testing is the larger process under which this technique falls. This is similar to how online advertising is a larger process, while Instagram video ads are a tactical technique within the broader bucket.