marketers reconsider A/B testing

Time to Reconsider A/B Testing

A/B testing isn’t making marketers very happy. Once a promising solution to website optimization, A/B testing is now seen as faulty, time-consuming and difficult. While setting up A/B testing is easy with the right tools, getting the answers you need takes time, practice and expertise. In fact, most A/B tests are never published because they don’t increase conversions! Marketers are seeing much more benefit from focusing optimization efforts on more modern, user-centric testing methods. Here’s why you should reconsider A/B testing.

the latest on A/B testing

It takes time to get right

Each A/B test requires a hypothesis around how your variations will measure up to the control – and repeated rounds of testing to gain usable insights. It’s a time-consuming process best suited to helping you learn about your target audience with incremental steps. But remember, true lifts in conversions come from understanding the user and serving relevant and valuable offers (something that may be more easily done with data and personas).

It’s error-prone

Did you know that only 1 out of 8 A/B tests have driven significant change? A/B testing is so wonky because you’d need to run tests until you reach statistical confidence – the probability that a test result is accurate. (Most researchers look for a 95% confidence level in order to reach a conclusion.) Otherwise, results may be influenced by random factors and produce false data. Calculating statistical confidence can be too complex for those of us who are not statisticians.

It’s not for every business

Another reason A/B testing fails may be sample size. There are tools available for determining the correct number, but some estimates suggest that 25,000 visitors may be needed to reach statistical confidence, and smaller businesses are unlikely to have that kind of traffic. This is one of the reasons that Optimizely, one of the most popular A/B testing tools, is discontinuing its free platform in favor of one more focused on large enterprises.

Contact our team today to learn more about understanding your key audiences – and making sure your websites are living up to their promise.  

The smart 404 page and tips to improve the endgame stage of user journey

Website UX: Don’t Leave Users Hanging

Improving 404 error and post-conversion pages

Imagine someone cruising along and enjoying the journey, when a DEAD END sign suddenly appears. It can feel like a frustrating slap in the face! The same thing happens online with website dead ends that jolt users right out of the experience. The “endgame stage” of the user journey is an opportunity to create engagement, nurture leads and move traffic, so don’t miss it. Consider these tips to help improve the user experience with your website.

Do a detailed walkthrough

Look at your current content and navigation – including things like your information architecture, site map and user interface. Your goal is to root out spots where users are left hanging. If users have made a purchase, filled out a contact form, registered for an event or read a blog post, what happens after they complete the action? Ask yourself what you want users to do – and then funnel them to the next step with a cool link or clear call to action. Seems pretty simple, but these things are often overlooked during design.

Try these 2 ways to improve your UX

  1. The smart 404 page

Ease the frustration of a missing webpage or lack of search results with a smart 404 page. It’s a great place to play with design and copy – or even to incorporate gamification or interactivity. The Signal 404 page gives people multiple ways out, making the 404 but a minor delay. Here are some other inspiring examples:

  1. Thank you page with a twist

The thank you or “success” page is often a missed opportunity to set expectations – what’s happening now, after they’ve purchased your product or filled out your form? Keep people moving by including:

  • High-value product or project links
  • Helpful content, like white papers or videos
  • Special offers or promo codes
  • Social sharing and referral features
  • Feedback opportunities

Read our new white paper, User-Centered Design Playbook, to find expert tips on improving the UX / CX of your website.

Ecommerce Website Redesign

Eastern Skateboard Supply is the leading skater owned and operated B2B wholesale distributor in the industry. As skaters and surfers themselves, they pride themselves on knowledgeable customer service and a comprehensive selection of more than 11,000 products.

Our team redesigned the site to improve the user experience (UX) and shift to a more modern e-commerce framework, with helpful features that today’s customers expect. We polished the messaging and gave the site a sleek, modern design full of color and engaging imagery. From a technical perspective, Signal integrated Eastern Skateboard’s enterprise resource planning (ERP) system with a new e-commerce platform, Magento. The result was a seamless connection between technologies, syncing customers, orders and inventory.

Experience it here

user-centered design case study

User-Centered Design Playbook

Website case study

User-Centered Design

The world’s largest independent distributor of Toyotas called upon Signal to audit and optimize their finance website to provide customers with an improved user experience for managing their vehicle leases and loans.

Their goals were to improve navigation and usability issues, and to personalize content based on where customers are in the life cycle.

In this new case study, you’ll learn about the steps our team took to enhance the customer experience (CX).





Our UX optimization approach typically follows a 4 step process, tailored based on client needs. With this project, our first step was to understand current website performance and identify opportunities using existing data and information. Evaluating basic website metrics and internal feedback is an efficient way to gain insight into the present state of affairs.

Planning and Research

Put a measurement plan in place

Don’t forget to set your goals at the outset of the project. For a website, this may include things like fewer clicks, faster actions, reducing call center support or fixing reported usability issues. You’ll want to be able to measure improvements on these key metrics later.

We find that more companies are using an organic approach to UX optimization for their websites – meaning they replace full website redesign efforts (every few years) with continuous improvements to ensure the best possible customer experience.

Scrutinize website analytics

Your next step is to take a close look at your website analytics. It’s important to know which pages are getting views, so you can focus efforts on those that are most visited. As Forbes notes1, web analytics:

  • Allow you to personalize things to frequent visitors’ distinct tastes
  • Reveal what’s working – or not – from a UX / CX perspective
  • Enable you to proactively fix site issues before they escalate
  • Inform the customer journey – if you take time to use and listen to the data!

Analytics for our client revealed a couple of major action items. Their Home, Account Summary and Login pages carried 80% of the visitor traffic, and they needed UX attention. Analytics also revealed more mobile users than previously known – a critical point to keep in mind when redesigning web elements.

Investigate hot site search terms

We always recommend looking at search terms / keywords, because while some people use search as a primary means of navigation, most are using it because they are having trouble finding the information they need. The following top search words helped us prioritize certain information on the site:

Search image


  1. statement, statements
  2. payment history, pay
  3. interest rate, refinance
  4. payoff
  5. extension

Listen to the call center

Employees who work in the client’s call center are on the front lines fielding customer questions – especially those that could be answered by an effective site structure and relevant content. CSRs told us there were a number of areas which needed to be built out on the website. Based on their feedback, we improved instructions and UI to ensure that customers:

  • View the status of pending payments
  • See their outstanding balance, payoff amount, due date and interest rate prominently featured
  • Find an easy selection area for invoice preference (paper or paperless)

Look at customer survey data

Like website analytics, your customer survey data is another fertile source of information about pain points. Dig in and see if there’s anything relevant.

Cool tools for deeper dives

Heat Maps

Heatmaps and recordings

helped us see users’ movement within key pages, giving us a more accurate idea about engagement.



allowed us to be mindful of users’ key characteristics and needs.

Customer Journey Map

Customer journey maps

broke down the steps and phases, so that we could clearly see pain points and capture ideas for improvements.

GET THE DETAILS on these methods in our UX Demystified white paper





If you’ve ever gotten lost, you know about the importance of having a good map to steer you right! The concept extends to the sometimes twisting, turning paths of website navigation. As the Nielsen Norman Group (NNG) points out2:

Information Architecture

“Structure and navigation must support each other and integrate with search and across subsites. Complexity, inconsistency, hidden options, and clumsy UI mechanics prevent users from finding what they need.”

Audit website tasks and content

Our first step in thinking about information architecture – that is, the organization and labeling of website information – was to conduct a thorough audit of the client’s current site. This allowed us to create a comprehensive catalog of content and user tasks, segmented by stages of the customer lifecycle.

Action / content Onboarding 1 – 3 months 3 – 12 months 12+ months
Register for an account X
Learn about my vehicle’s features X
View my balance X X X
New lease rates X X
Monthly service coupons X X X
New vehicle offers X
View my payoff X

Use card sorting to optimize information architecture

Using the information gleaned in the audit, we conducted open and closed card sorting sessions with 25 users on a digital platform called Optimal Workshop. Card sorting is an easy UX research method that allows users to categorize content and tasks – as well as to develop intuitive navigation labels. The chart below answers3 some of the questions you may have.

When is card sorting helpful?
  • To find out how users expect to see information on a website
  • To organize a sitemap
  • To organize submenus
  • To classify products or services
How do you do card sorting?
  • Use a digital platform, such as Optimal Workshop, to categorize items and allow session participants sort them into groups that make sense to them.
What is the difference between “open” and “closed” card sort testing?
  • With open card sorting, users can create their own categories or add missing information. This helps you categorize and prioritize tasks/content and label groups.
  • Closed card sorting limits participants to using only those cards provided. The goal is to optimize and confirm your chosen navigation categories and labels.

Card sorting matrix

Develop a new site map

The Signal team combined all the feedback from user research along with IA best practices and testing to create the final site architecture, removing redundancy and making paths more relevant and intuitive to the tasks at hand.




USER INTERFACE defines user interface (UI) as a focus on “anticipating what users might need to do and ensuring that the interface has elements that are easy to access, understand, and use to facilitate those actions.”4

Based on our findings from research and testing, we worked with a number of methods to show our client the new user flow, functionality and layouts.

User Interface

Wireframes: a quick and cost-effective way to prove concepts

These low-fidelity iterations connected the information architecture we created to its visual design, helping us:

  • Prioritize content
  • Allocate space to given items
  • Decide where to locate items


Clickable prototypes: conveying intended functionality in the interface

Don’t build it – “fake it” first! Both wireframes and prototypes are mockups of the proposed site or application, but we recommend prototyping to experience functionality. The clickable site prototype gave us something to test, so we could work iterations faster than with fully functional code.

Higher fidelity mockup: the final step before the site build

Finally, we applied styles and branding to give users a preview of what the site would look like. The goal: improve design first before moving into (potentially) more time-consuming coding and development.

High fidelity layout

Agile development

At Signal, we operate from a place of “customer first.” And for us, that means thinking about what’s important to the end customer – our clients’ clients. This is a cornerstone of the Agile methodology, an iterative design process that aims to get the work into the hands of the customer as early and often as possible. It can be applied to any project, from websites and marketing communications to digital apps.

Learn how we do Agile at Signal!





Conduct user testing

The last step of the process is to test the new design with real website users to ensure optimal UX / CX. User testing doesn’t have to be as daunting – and expensive – as the traditional focus group in a lab environment. There are many digital platforms to help recruit volunteers and conduct tests. We chose, a site that uses crowdsourced testers who are ready and waiting. You can even define your target attributes such as age or gender.

Testing and Optimization

For our client, we assembled 10 tester volunteers for “think aloud” user testing, using’s easy online platform. (For the user experience design geeks out there, “think aloud” testing is one type of remote unmoderated usability study.5)

A remote unmoderated usability study is when participants complete pre-determined activities using a design or interface. The participant decides when and where they would like to complete the study, and uses an online tool to participate, provide feedback, and record the session.”

The male and female testers ranged in age from 18 – 65 and were average computer users. We tested the client’s current design against our new design, using simple tasks such as “Find your payment history” or “Change your user/password.” Listening to real users narrate their way through web pages was incredibly valuable.

  • “Hey, it’s not popping out at me…” [OLD DESIGN]
  • “I’m confused about vehicle vs. account maintenance…” [OLD DESIGN]
  • “Yes, this is where I want to go…” [NEW DESIGN]

Listen to a few clips from the live user testing sessions.

The UX experts at NNG call think aloud testing the #1 usability tool, and for good reason. The benefits include6 being:

  • Cheap, because you don’t need special equipment to gather insights
  • Robust, because it’s easy to get good findings (without a lot of statistical expertise)
  • Flexible, because you can use it any stage of development
  • Convincing, because clients love direct insight into how their customers think and act


  • Improved “time to task”
  • Validated label names
  • Prioritized better navigation and page layout
  • Tone and attitude changes from frustration to confidence from old to new design

Improved key metrics from the measurement plan put in place at the beginning of the process

Final thoughts

User-centered design principles can improve the overall customer journey, which helps you:

  • Better compete in the marketplace
  • Turn happy customers into repeat customers
  • Shorten the customer lifecycle

The UX / CX process we outlined in this case study can be scaled to fit your needs. The deeper you go, the more you can uncover and fix.


Signal’s multi-disciplinary team of writers, designers, strategists and technologists won’t lean towards a one-size-fits-all solution – we’ll craft one just for you.


ux optimization and card sorting

Create a Better User Experience with Card Sorting

card sorting is an easy ux research method

The structural design of your website – AKA informational architecture – is critical to UX optimization. Visitors who can’t find what they need while navigating will become frustrated – or even worse, abandon your site altogether! Card sorting is an easy UX research method that helps you design a user-centered site that increases usability and delivers a positive customer experience.

What is card sorting?

Card sorting allows users to categorize content and tasks – as well as to develop intuitive navigation labels. This time-saving step helps you create clarity before moving to site design.

What data do I need?

The first step is to conduct an information audit of your website to understand how things are organized and labeled. Look at things like website analytics, hot search terms and customer survey data. Consider creating a spreadsheet of content and user tasks – what do users need to know and do?

How do you actually do card sorting?

Simple – just write down the elements you want to categorize on cards or sticky notes, and have session participants sort them into groups that make sense to them.

Another option is to use a digital platform such as Optimal Workshop, which gives you the flexibility to run quick research remotely. 

digital card sorting

When is it helpful?

  • To find out how users expect to see information on a website
  • To organize a sitemap
  • To organize submenus
  • To classify products or services

What are “open” and “closed” card sorting? 

  • With open card sorting, users can create their own categories or add missing information. This helps you categorize and prioritize tasks/content and label groups.
  • Closed card sorting limits participants to a set batch of cards, and defines the available groups or categories to put the cards into. Volunteers can’t create their own groups and tasks – they’ve got to stick to what you’ve given them. The goal is to optimize and confirm your chosen navigation categories and labels.

What do I do with the card sorting data?

Use it to create your final information architecture, one that’s more intuitive – and relevant to your audience. Now you’ll be set to proceed to creating the user interface – the flow, functionality and actual website layout.

Contact Signal to help you plan and execute UX research to ensure the best possible customer experience.

Website Redesign

W.M. Jordan Company is a construction management and general contracting company that serves clients in Virginia and the Carolinas. They called on Signal to redesign their website in order to improve lead generation, engage with customers and subcontractors – and showcase the company’s culture to prospective employees.

Our team collaborated with W.M. Jordan on the website redesign, making the content more digestible and memorable – as well as giving the site a bright, clean look and feel. Steps included:

  • Incorporating more rich media such as drone and construction time lapse footage of projects, and video customer testimonials
  • Replacing static content with engaging, interactive tools to better showcase the company’s work
  • Creating a more intuitive site navigation structure to improve the user experience (UX)
  • Giving company contacts the flexibility to upload and categorize different content
  • Building out an enhanced news and events page that allows visitors to filter posts by tags such as Awards, Community Outreach and Industry Insights
  • Refreshing copy to give it a friendly, approachable tone and to foster thought leadership

Given the company’s focus on marketing through tradeshows and industry events, we also enhanced site functionality to maximize lead generation. Our team provided an email sign up function connected to Constant Contact, and gave WMJ the ability to create custom landing pages for different markets and events.

Experience it here

Onsite Deployment Unit Campaign


In response to the needs of their pharmaceutical clients, IQVIA established an Onsite Deployment Unit (ODU), a large pool of analytics professionals ready to provide commercial operations support at a client’s premises anywhere in Europe. Because the new ODU team represented an innovative new resource previously unavailable in the region, IQVIA wanted to get the word out to their sales personnel in a vivid and memorable way.


IQVIA asked Signal to develop a creative promotional kit to be mailed to the salesforce in three installments. Signal presented several concepts, from which the client chose a secret agent/mission briefing theme. With a colorful retro style that evokes James Bond and Mission: Impossible, the mailers introduced the sales representatives to the ODU in the form of a mission dossier on a new “secret weapon” at their disposal to provide a higher level of service to their customers. We kept the focus on the salesforce audience, explaining how the ODU can help them look like a hero and detailing the rewards to be earned by successfully “completing the mission.”

Rather than sending simple postcards that might be disregarded, IQVIA wished to send intriguing packages to get the sales representatives’ attention. Each mailer contained a secret “mission communiqué” message that could be read with the use of an enclosed color filter spyglass decoder. Also included with each mailing was a special spy-themed trinket, such as a USB flash drive pen, a cocktail kit or a toy Aston Martin car.


The client was pleased with the response to this creatively engaging campaign. Numerous ODU placement contracts were signed and the salesforce gained awareness of the new resource going forward.