The BPI Network Journal

How to Get the Most From Your Website A/B Testing

By Ramel Levin, Vice President of Marketing, ClickTale

In today's conversion-driven digital markets, A/B testing is a fundamental tool that takes the guesswork out of website optimization. It is a great way to quickly try to improve the design and performance of your web pages.

This short article discusses three specific issues that can improve your A/B testing and lead to better website optimization:

• Data-driven versus user experience-driven testing
• Audience segmentation
• Testing procedure

Before You Start

A/B testing is a valuable tool that enables the comparison of two versions of a web page to identify which of them elicits more responses, causes more visitors to subscribe to a service, or leads to more conversions. But how do you choose which pages to test? And on each page, what elements should you test?

You can start with best practices. Most web analysts will recommend testing calls to action, the content placed above or below the fold, and the structure of a registration form because all of these can lead to quick wins and have a strong impact on conversion.

You can also make a hypothesis about which areas of a page may be causing frustration or abandonment. This process can be aided by in-page web analytics, which monitor your visitors' behavior on the page (including where they are clicking, hovering or abandoning a funnel) and provide insights into which page elements to A/B test.

1.     Data-Driven Versus User Experience-Driven Tests

Data-driven A/B testing relies on hard numbers, including on the mathematical analysis of how people use your webpages, click different elements and interact with every section. It is based on statistics, calculations, responses to questionnaires and/or feedback forms that you ask your visitors to complete. Data-driven A/B testing is structured, stable and easy to measure.

User experience (UX)-driven tests, on the other hand, are based on hundreds or thousands of theories involving issues like font type, font size, color, position of the call to action, page design and more. In many ways, user experience is related to creativity and personal opinion, which are very hard to measure.

Data-driven A/B testing should be your goal. Gather as much significant data as possible, structure it to ensure high-quality results, test and analyze it, and draw conclusions. The result of the A/B test of your home page may lead to the conclusion that no changes are required. However, if the test was conducted scientifically using large amounts of data and relying on tools that provide you with insights into your visitors' behavior, then you will know the results of the test are valid.

2.     Segment Your A/B Test Audience

Today, more and more companies personalize their web pages to more accurately deliver their messages and campaigns to different target audiences. To accomplish this, it is important to be able to segment visitors. For example, you can segment the target population of an A/B test by new versus returning visitors. If you sell a service to which your customers are already subscribed, you do not want to offer them something they already have, but rather show them a page with other supplementary products, a training video, the page of your support organization or information about how they can maximize the benefits of your service.

This methodology is also known as targeted A/B testing; you do not test with everybody…only with specific segments of the target population. As author Avinash Kaushik states in Web Analytics 2.0, "Absolutely nothing is more important in analytics than segmentation. Why? Because monoliths don’t come to our websites. I know that seems like such a shock. Yet most of our reporting and analysis happens at an aggregate level…Different types of visitors come to your website. And they all come with different intentions, with different problems or solutions, and with different personas. This unique phenomenon mandates that you have a very effective and persistent segmentation strategy as part of your web-analytics process."

Other popular segmentation options for A/B testing include:

• Geographical location—This is useful if you're considering localizing your website or if you wish to further outline the services you provide to specific locations (for example, shipping).
• Source—This method enables you to segment your visitors according to what brought them to your website (e.g. PPC campaigns, social channels, email campaigns or organic web search). It is very useful when you're trying to measure the success of a campaign or when deciding on your company's marketing budget.
• Previous behavior—You can target your message or campaign based on whether a visitor has previously shopped on your site, visited but not shopped, or has items in the shopping cart from a previous visit. It also allows you to assess the user experience and make improvements where necessary.

If you want to test other behavioral aspects like how visitors respond to a color or message, you can run a multivariate test or factorial experiment to test different combinations of different sections or elements of a page.

In summary, segmentation provides an infinite number of possibilities for tests that you can run, further perfecting your optimization cycle.

3.     Test, Test and Retest

Many businesses make the mistake of A/B testing a web page, analyzing the results, making the changes (if the results of the A/B test show that changes are indeed needed), and then moving on to another page.

This should not be the case, however. After running the A/B test, you may learn that your original hypothesis was incorrect, that the results are inconclusive, or that both versions of a tested page had the same conversion rate. This does not mean you should stop testing; it only means that you should go back to the drawing board, study your visitors' behavior, make another hypothesis and test again. This is the only way to ensure a continued process of optimization.

Testing is a cyclic and iterative process. You can run many tests on different sections of the same webpage or test a single element (for example, a call to action) across several pages. Moreover, user preferences change over time, the devices they use to interact with your website are constantly advancing, and you want to make sure that they always have an optimum experience and that they keep converting. This can only be achieved through continuous testing.

What's Next?

After collecting the data, segmenting it, conducting the test, analyzing the results and making the changes, you can move on to understanding why one version performed or behaved better than another. This can be achieved with in-page analytics tools like heat maps, visitors' recordings and form analytics. These provide actual insights into your visitors' behaviors when they interact with your pages, explaining why they reacted differently to each version.

The visualization of people's actual behavior online will help you draw conclusions that you can implement across all website pages, so that you can enhance your visitors' experience, increase conversion and improve your bottom line.

Author Bio:

As Vice President of Marketing, Ramel Levin is responsible for the ClickTale brand, marketing communications, acquisition and retention. He has more than 20 years of experience in the high-tech development and marketing industries. Prior to joining ClickTale, he was the Marketing Director at Panaya, where he led the company's entrance into new markets in addition to growing their existing markets. Earlier in his career, Levin was Marketing Director at ARX and was also the founder of WorldMate, where he led a mobile bootstrap from inception to a multi-million-dollar revenue company. He holds an MBA from Hebrew University and a bachelor’s degree in psychology and computer science from Bar-Ilan University, both in Israel.

Share

Could Business Reorganization Harm Customer Relations?

By Nicholas C. Watkis, Principal Consultant at Contract Marketing Service

Change is the only constant. Whether considering the states of the universe, weather patterns or ecological systems, all are in a state of continuous change and development where even the rate of change itself changes, creating very dynamic systems.

Business and commerce generally are similarly dynamic in their evolution and change. New market opportunities arise as others decline, resulting from the continual alteration and development of customers’ needs and desires, driven by fashion, economics and technology.

Businesses must therefore modify and develop their organizations to match their capabilities with customer requirements in a cost effective manner to maximize their profits.

The objective of every business is to continually produce and maximize profits for the long-term benefit of owners, staff and customers. For the manager of commercial operations, the objective is to maximize profitable income and make it sustainable for the long term while minimizing costs and the use of assets. But in order to achieve this, businesses may need to adapt their organizations and operations in order to maintain and increase their profitable income.

More often than not, organizational change in businesses is driven by a perceived need to cut costs, often resulting from reduced profit. But profits come from income, so reorganizing to cut costs may not be the answer when failures in customer satisfaction, a decline in demand or increased competitor activity may be the underlying cause.

While recognizing that business organizations need to change and develop in order to meet the changing circumstances and conditions of their markets, it is important to understand that the way such changes are made may have far-reaching and unforeseen consequences.      

Before embarking on change, it is important to establish why it should be necessary, what it is intended to achieve, and how it will benefit the organization. Is the requirement for radical process or evolutionary development? Experience shows that evolutionary development is generally more effective than the radical answer, which should rarely be used and almost as a last resort as it frequently produces confusion, inefficiency and demoralization.

Changes in organization and operations are intended to produce consequences that may entail greater efficiency, cost effectiveness and ultimately more profitability. However, the most important consideration is how such changes will affect the customer.

It should be remembered that alterations to a business’ operations and organization may be disadvantageous to the customer which, in a competitive market, may result in reduced demand and reduced income. At the same time, reformation in a business organization and operation may provide opportunities to increase market penetration as well as income. Any adverse effects on customers and their demand resulting from changes to a business’ organization and operations must be avoided or minimized. Therefore, before commencing structural or operational changes, commercial managers need to ask the following:

• How will this change affect the customers?
• Will the customers perceive the change as beneficial or detrimental to them?
• What will be the positive consequences of the change?
• What will be the negative consequences of the change?
• Can the results of the change be quantified?
• If not quantifiable, how are the benefits to be measured?
• How much will the changes cost in time and money?
• Who will benefit from the changes?

Only when these questions have been satisfactorily answered should organizational and operational changes be considered. When they are, the objectives should be clearly understood, and the results should be quantifiable. “Best practice” suggests that as far as possible, staff should be encouraged to suggest where changes and improvements can be made as they are often better placed to know how these may be made and how to implement them successfully.

Change and development are essential if businesses are to continue to prosper for their long-term future. But change and development must not be made at the expense of the customer and should aim to increase customer satisfaction in order to maintain and increase profitable income.

While companies need to make profit to justify their existence, they make it by providing products and services that customers want to buy. When companies make changes that are perceived by the customer to have adverse effects on the product or service they seek, they will go elsewhere and take their money with them. 

Author Bio:

Nicholas Watkis is an experienced specialist in marketing management performance with extensive experience in executive management and organization for business development across a wide variety of B2B markets and organizations.
He is a professionally qualified Fellow of the Chartered Institute of Marketing, as well as a Certified Management Consultant of the Institute of Consulting. Watkis is also the author of articles on marketing management performance, which are published internationally.

Share

DOUBLE YOUR VALUE

By Mark C. Thompson and Bonita Buell-Thompson

Value is a perception—not a fact—and everybody has different ideas about what’s most valuable. We develop these ideas early based on both our experiences and inherent inclinations. In doing research for our book, ADMIRED: 21 Ways to Double Your Value, we discovered a number of techniques that can be used to determine what’s valuable to your boss and coworkers. Below, this article outlines nine steps that will help to dramatically increase your value.

1. Two Interests are Better Than One

First and foremost, most people treat themselves as a commodity—something that can be replicated and that isn’t unique or has value that’s more or less equivalent to that of other commodities like it.

Instead, think about how you can combine several of your special interests, skills and passions into a role in your company. Steve Jobs is a great example of this. He was an orphan who started Apple in his foster parents’ garage, but he dropped out of engineering school when it failed to inspire him, not wanting to waste his folks’ money on something he didn’t feel was special or unique. Instead, he started taking art classes, particularly calligraphy, where he discovered what technology was missing most: a sense of beauty and ease of use. 

Jobs’ ability to combine his dual passions for engineering and art into exquisite industrial design is legendary. Rather than following a linear path toward a career or job, you’re better off combining your multiple passions to make what you do more unique. 

When Bonita Thompson started out in HR 30 years ago, she combined her love for organizational behavior with her interest in statistics. She became a compensation specialist, which at the time was valued at a premium over being an HR generalist. It was very rare to find an HR person who loved both people and math, so Thompson was able to capitalize on that. She then went on to earn an MBA with a concentration in management information systems. Her interest eventually led to her to develop the world’s first corporate intranet for Human Resources. She created that innovation at Levi Strauss, and the idea sprang from both her interest in HR and her passion for using technology to improve service. Nobody else in the world has the same combination of experiences, talents and interests as you, so use them to your advantage! Two or three passions are better than one if you want to offer your company something unique.

2. Sign Your Name

It’s really important to give your product a personal imprint and show that you’re willing to take the responsibility to put your name on the door—at least metaphorically. Show that what you’ve created is so special, you’re willing to brand it. 

When Thompson was working in a cubicle amidst dozens of others as an HR job analyst for Bank of America, she found a way to brand her special way of analyzing more than 100,000 positions at that bank, and it really separated her from the pack. She created the brand name of “Market Analyzer,” which turned an average project into a special product that distinguished her work as strategic to the organization.

Sign off on all the unique work you do—make it clearwho you are and how important your projects are to your organization.

3. Know What Your Client Values

Understanding what your boss wants and values is not always as obvious as you might think. To figure it out, you must actually go and observe them—spend time figuring out your boss! Most supervisors appear to be clear about what they think you should be doing on your job, but have you found out how they are evaluated by their bosses? In other words, do you know what makes your boss successful?  What is she or he stressing out about, and how might you think of ways to help? 

When Thompson was an HR officer, she noticed that her manager was scrambling to keep up with current events in the international markets where the company had many offices and branches. She took it upon herself to scan the relevant publications and pull out articles for him that related to both his job and the company’s priorities. This ended up becoming a highly valued skill, and her boss became seen as an important source of strategic and up-to-date information by other senior executives.

If you want to be highly valued, don’t wait to be told how. Go find out what’s challenging your boss and clients, and seek to help them long before you’re asked.

4. Know Who They Value

It’s vital to not only know what your clients value, but who they value. Who matters to their success? A study recently emerged showing certain nonprofits that had actually grown during the economic crisis. Nonprofits were obviously amongst the hardest hit, but there was a group that mysteriously managed to grow anyway. They succeeded by making it their business to find out who was most important to their various donors or organizations and which individuals or outlets those donors would want recognition from. Save the Children, for example, put out press releases and sent notes to the communities and people who mattered to their donors, emphasizing how much of a difference that donor’s contribution had made. These non-profits effectively used both flattery and public promotion.

It’s definitely good to keep in mind that flattery can be over-used and end up appearing fake or off-putting, but if it’s given for specific behaviors that people take seriously, flattery has tremendous value.

In fact, Cliff Nass, a professor at Stanford, recently conducted a research study on flattery and criticism, and hesurprisingly discovered that people appreciate and respond to flattery far more than they’d like to believe. People actually love to be patronized! Nass also discovered that people criticize others much more than they flatter, meaning when we do praise someone or something, it’s more cherished.

If you want to be more valuable, give more status and recognition to the good work done by those around you—even your boss—and it will call your attention to the ways in which you should be valued, too.

5. The Three Ps: A Three-Part Secret to Success

We all too often separate our passion from the work we do for others, and this is such a big mistake! We conducted a World Success Survey of 110 nations for our bestselling book, Success Built to Last, which determined that people who had continued success and impact in their field or profession—for 20 years or more—had all three traits in common. Those who had success year after year, through both thick and thin, all defined success in three distinctive ways that were necessary for them to have success that was sustainable. We call them the three Ps as part of their value equation: passion, purpose and performance.

In our jobs, we’re all held accountable for goals, and that’s what we mean by performance—we have to deliver on our promises and deliver them well. But successful people don’t just work for a company or a job, per se. We have to also be serving a purpose that’s larger than ourselves.  Most people need to have a cause that motivates them—something that we feel makes a difference. 

At the same time, we also need to be doing something we care about and have passion and heart for—that’s what really recharges our batteries and gets us through setbacks. Otherwise, during tough times, how would you scrape yourself off the floor and get back out if you have no fundamental connection to what you’re struggling with? We have to pursue something that serves the market, which serves customers and serves something we care about and want to deliver year after year.

6. Get Known

In this day and age, there are so many techniques available to help get the word out about what you’re doing. Start by creating a connection to social media—blog about what you’ve been doing and whom you’ve been helping. Mention case histories, what problems have come up for you or your organization, and how you solved them.

Guy Kawasaki is a great example—he separated himself from the pack of early Apple folk by blogging his thoughts and experiences for many decades. He’d left the company early on, so he never became as wealthy as those who stayed. But he has had so many valuable things to say about his profession in general, and Apple in particular, that he’s became a sought-after speaker and consultant.

Joining professional associations is another really effective way to both get known and have impact: they’re always looking for more hands, hearts and brains. Offer to help out, and you’ll probably be asked to join the board. Reach out to reporters who write for your professional magazines and newsletters. Eventually, you will be quoted, and before you know it, you’ll be better known in your industry. Make sure that you can serve others and, in so doing, you’ll gain visibility for your valuable services.

7. Build Trust: Offer a Sense of Security

One of the best ways to be seen as valuable is to make people feel secure that you can get the job done right. In our research, we found three steps that are most valued in building trust. We call them the three Rs:

• Responsibility: People need to feel they can count on you, but there’s more to it than that—if you make a mistake, avoid playing the blame game. Don’t waste energy deciding who’s most at fault or finding excuses for why something went wrong. Instead, take responsibility for the mistake or problem, take accountability and offer solutions that can solve the problem.

Your credibility can actually increase more when you make a mistake if you show the ability to take responsibility and accountability even when things aren’t rosy.

• Reliability: Show up as the same person every time—people will not trust you if you’re someone different every time they meet you. Be consistent and behave consistently in all of your interactions.

• Responsiveness: Do it quickly! Deliver on whatever promises and commitments you’ve made in a timely way. The faster you follow through, the more credibility you will have.

8. Always Get Better

What can you do better next time? This step might also seem obvious, but it’s one of the most difficult to implement. When we fail, we again tend to play the blame game, focusing all of our energy on fault as opposed to what can be done to succeed in the future. Worse yet, we decide not to try again.

But success isn’t a good teacher, either. Most people find it hard to determine ways to improve when they are already succeeding. We don’t see the incentive to get better—we won, how could we possibly improve? The people who are valued most in an organization over the long term focus on how to do even better the next time around.

9. Create a Place for Your Goal

To make something happen and get something done, we need to be with the right people, at the right time, in the right space and with the right tools. Make sure your work environment has everything you need to succeed with your goal—including all of the tools, files and information that will help make progress the moment you turn your attention to it.

A successful entrepreneur that we know wanted to fulfill a lifelong dream of making a movie. But to make that happen, he realized he really needed to move to Hollywood or New York—he needed to start to access a community of people in that field and build a network. Another friend wanted to learn a new programming skill for her HR work, so she made sure that she always kept a special place on her desk with all of the books, software and tools together in that place—ready for her work every day. Studies show that people who lay out their gym clothes the night before a workout are many times more likely to actually work out the next day. 

Perhaps most importantly, if you want to be more valuable in your role or position in an organization, make it your business to meet other people who are valuable in the type of job that you’re striving for. When you dive into the nine steps we outlined in our research, start by observing and befriending people you value and respect—those you know are valued in their professional communities. What are they doing, and how are they interacting with others? Spend less time with toxic friends and invest more time with people who inspire you. How were they driven to do more and become more valuable? It’s a good bet they’re doing something they love with people who support their passions—and that is essential to success that is built to last. 

Author Bios:

Bonita Buell-Thompson

Bonita Buell-Thompson is a 30-year veteran of human resources and industrial psychology at Bank of America, Catellus, Pacific Telesis, Genentech and other leading organizations. She holds an MBA in information systems from U.C. Berkeley and was the Research Director for Success Built to Last—the sequel to Built to Last—and the Leaders of the New Century series.

Mark Thompson

Mark Thompson is CEO and cofounder of the Richard Branson Entrepreneurship Hub for Virgin Unite and cofounder of the John F. Kennedy University Institute for Entrepreneurial Studies. Forbes magazine called him the “venture investor with the Midas touch.” He is the former Senior Vice President of Charles Schwab, an executive coach, and venture investor in Best Buy, Facebook, and Apple’s leading iPhone and iPad applications companies. He is a founding advisor of the Stanford University Venture Design Lab and New York Times bestselling author of ADMIRED: Success Built to Last and Now, Build a Great Business! He also has a passion for Broadway, and his plays have earned 10 nominations and five Tony Awards. 

Share