Chapter 2
Research

One never knows, do one?
—Thomas “Fats” Waller

In designing web pages we often base initial plans on widely used page layout and navigation patterns, and use best practices such as consistency, modularity, and simplicity in creating our web interfaces. However, the best method for making design decisions is to expose these design best practices and familiar web layout conventions to a regimen of close consultation with users at each stage of the design process. Involving users in the development process helps us understand user requirements, which allows us to make informed design decisions and produce more effective designs.

In this chapter we cover research methods that support a research-directed approach to web site development. Every aspect of user experience design benefits from engaging with people, through activities such as brainstorming, observation, inquiry, and usability testing. We also discuss the use of web analytics as a way to gain insights into how users work with digital products. Analytics can provide a useful perspective but are insufficient on their own in providing a window into understanding user needs and preferences.

Research goes beyond asking questions and delivering solutions based on what people ask for. As a quotation attributed to Henry Ford goes, “If I had asked people what they wanted, they would have said faster horses.” Research is about identifying opportunities to meet even unexpressed needs, and to devise innovative solutions that provide improved ease of use for everyone.

Brainstorming

One of the most important parts of the process of developing an innovative and highly functional site is ideation and brainstorming, and the research and exploration of user needs that should always precede ideas about how to create or improve the user experience of your online content. Although open-format free-for-all brainstorming sessions work well in some instances, it is a good idea to set ground rules to guide expectations about how the results of ideation will influence prototyping and later site development. It’s also good to have a process framework for developing new ideas that is based solidly on user-centered design principles and research.

Using “design thinking” for brainstorming

Charles Eames, asked in a 1972 interview, “Does the creation of design admit constraint?” answered, “Design depends largely on constraints.” This tension between creativity and constraints is at the core of “design thinking,” a creative approach to problem solving popularized by the design firm IDEO. A design thinking approach encourages the use of creative, human-centered methods to identify real needs and to build solutions that can be successful within environmental constraints.

Inspiration: Defining the problem that needs solving.
Ideation: Engaging in a process of generating and trying out solutions.
Implementation: Taking a solution from concept to product.

IDEO’s human-centered design process is not specific to the web, and was developed primarily to help nongovernmental organizations (NGOs) in their support and charitable work with poor communities. Nonetheless, the design thinking approach is applicable to any ideation, design, and prototyping process, helping to provide structure at each step along the way and, perhaps just as important, establishing a filtering process by which to judge which ideas are truly worthy of development into prototypes and working products.

H-C-D framework

IDEO’s H-C-D framework (Hear, Create, Deliver) is particularly appropriate for fast-paced and iterative development of user observations, potential solutions, and design deliverables. The process starts with a substantial period of exploration, research, and simple old-fashioned listening to what real users would find helpful or interesting. These exploration sessions are by far the most trustworthy way to generate ideas that not only are interesting and helpful to users but also help the development team to understand the audience’s perspective on different aspects of the project, such as desktop versus mobile uses of your site. Web developers who sit at desks all day in front of double-monitor rigs often severely underestimate the needs and ambitions of mobile users.

Hear: In this phase the design team collects user stories, looks at both general and specific use cases for any existing web sites (or similar or competing sites), collects feedback and suggestions from users, and gets to know representative members of the audience that will use the final designs. The goal in this phase is to collect as much information as possible from real users and project stakeholders, and as many ideas as possible about how to serve their needs. This is the time to engage in user research activities, to put people first and learn about real end-user needs and preferences rather than make assumptions.
Create: The design team works together in informal workshop sessions to examine and prioritize user stories, look for general themes that might underlie or unite some user stories into broader ideas and solutions, and brainstorm about how this better understanding of user needs might translate into concrete prototypes and finished designs.
Deliver: In this phase, working detailed plans and working prototypes are first developed and then translated into detailed production plans for specific content, features, and code to support the highest-priority user stories and develop finished site elements.

Brainstorming Rules

The activity of brainstorming can be difficult and uncomfortable, for individuals and groups—even its name implies intensity, turbulence, and potential damage. But the benefits of brainstorming are also personally and collectively significant, as the activity encourages expansive thinking. A well-executed brainstorming session can strengthen the bonds and shared vision of a project team. Facilitation and some structure can help, including setting expectation among the group for how to engage in discussion. Here we present brainstorming rules from the excellent Design Thinking Toolkit for Educators, created by the Riverdale Country School and IDEO.

Defer judgment. There are no bad ideas at this point. There will be plenty of time to narrow them down later.
Encourage wild ideas. Even if an idea doesn’t seem realistic, it may spark a great idea from someone else.
Build on the ideas of others. Think “and” rather than “but.”
Stay focused on topic. To get more out of your session, keep your brainstorm question in sight.
Present one conversation at a time. All ideas need to be heard, so that they may be built upon.
Be visual. Draw your ideas, as opposed to just writing them down. Stick figures and simple sketches can say more than many words.
Go for quantity. Set an outrageous goal—then surpass it. The best way to find one good idea is to come up with lots of ideas.

Generating ideas and making choices

Good ideation processes will generate far more ideas than will be practical or wise to act upon. The final stages of a properly run ideation process winnow the mass of ideas into the small percentage that are worth the time and resources to develop into working prototypes of final project deliverables. How do you decide which stories, problems, and solutions are most practical, given the real-world constraints of time, resources, and budgets? In the design thinking process each potential solution must pass through three filters:

Desirability: How much do people want this?
Feasibility: Can we actually create this successfully?
Viability: Do we have the financial resources and stakeholder support to make this happen?

This filtering can be crucial to project success: as we’ll show in the next chapter, the vast majority of software project “features” are seldom or never used; every mediocre idea or unneeded feature that survives the filtering process wastes time and money, and will clutter and complicate your final product (see “filtering failure” in fig. 2.2). This rigorous prioritization is especially important in projects run with agile methods, as the core idea of an agile process is always to focus sprints on the most important features or goals.

This business of filtering might be the most important—and socially awkward—phase of the whole design process. We tend to concentrate on the fun and positive aspects of ideation, where “all ideas are welcome, and there are no bad ideas.” This wide-open “divergent thinking” start to ideation is crucial for real innovations to emerge. However, also important is “convergent thinking,” where ideas are eliminated and choices are made. As each idea is held up against the filters of desirability, feasibility, and viability, many good ideas are set aside in favor of great ideas. But all ideas contribute to the strength and depth of the ideas that remain. It’s crucial that all participants understand that good group ideation is a team process, and that the final great ideas are a team product. The whole team loses every time a mediocre idea survives the filtering process.

Group versus individual ideation

The standing presumption about ideation and brainstorming is that these are group activities. While group activities are useful at later filtering stages of ideation, studies show that:

A large percentage of the population is uncomfortable with group brainstorming techniques, and individual ideation produces more unique ideas.
Group ideation sessions tend to home in quickly on a few ideas, to the exclusion of others. Some of this is social dynamics, where the loudest or most passionate voices can dampen discussion of competing ideas. This “cognitive fixation” on a few ideas tends to shut down discussion of other ideas, and research shows that the fixation actually hampers many individuals from further creative thinking.

Research on ideations shows that the ideal process is a balance of initial individual ideation, followed by group sessions in which all ideas are presented and actively debated. This is essentially similar to the Delphi method in ideation and group consensus building, where each member of a group separately and privately records his or her best ideas or answers to particular problems. Each idea is given to a group facilitator, who anonymizes each contribution, so that no one knows the source of the idea. The group then meets and considers each idea in turn, ranking each according to usefulness. Delphi groups often go through several ideation and ranking sessions to achieve group consensus on the best or most useful ideas.

The H-C-D process also supports individual and group ideation phases. The early “hearing” and research period is well suited to individual ideation. The later “deliver” phases are best for group sessions and vigorous discussions about which ideas are best to pursue, and the Delphi method is ideal for this. Appoint a neutral moderator for brainstorming groups to act as timekeeper, run Delphi ranking sessions, and monitor against the cognitive fixation phenomenon that can too quickly shut down discussion of competing ideas.

Profile of a design thinker

One shortcoming of “design thinking” is in its name—the word “design.” Most people do not consider themselves designers, and this can be particularly true on technology projects. As Tim Brown notes in his Harvard Business Review article “Design Thinking,” to be a design thinker you don’t need to wear “weird shoes or a black turtleneck.” He presents the following characteristics, suggesting that the right development and experiences can bring these out in many people, who can then use design thinking to build innovative solutions.

Empathy: Taking a “people first” approach and getting into the shoes of potential users means you can arrive at solutions that are based on real needs and address real concerns. Without empathy, solutions are often based on factors that are not relevant to users, such as organizational priorities or assumptions of user needs made by project team members.
Integrative thinking: Being able to see issues across multiple data points and even contradictory research findings means you can analyze complex needs and constraints and devise solutions that are better suited to the reality of the problem area.
Optimism: Having the conviction that there is a solution, and holding fast to that optimism, gives you the persistence and endurance to stay the course, seeking out the best alternative.
Experimentalism: Creativity requires trial and error. You must be willing to try new ideas, and ready to discard ideas that don’t measure up and aren’t worth pursuing.
Collaboration: Dialog and exchange of ideas and insights with people who have other areas of expertise build synergy and yield creative approaches.

User Research

The first step in any web site design process is to gather information about users—who they are, what their goals are—and identify their requirements for working with the site. The research phase is normally the most time-consuming phase of any design project, but the time spent on research makes the design and evaluation phase move more rapidly. With good user research, the decisions that drive the design are based on a solid understanding of users’ goals and requirements and therefore are far more likely to hit the mark without many cycles of iteration.

Defining scope

Engaging with users is critical in the design process, particularly at the beginning phases, where the team is engaged in defining the need and devising the solution. As much as members of a web development team may feel they understand the problem domain, other points of view are essential. The problem may seem obvious and the solution straightforward—until you learn the constraints from stakeholder interviews and the unexpected user behaviors from field studies.

Several techniques exist for collecting feedback directly from users about their goals and behaviors. The information gathered from these collection techniques is subjective; it represents what users say they do, which is not necessarily the same as what they actually do and care about.

Surveys

Surveys are helpful for collecting a broad range of responses about demographics and goals. Web site surveys typically ask initial questions to help define the user: age, gender, audience type (customer, potential customer, buyer, seller). Then there might be questions related to frequency of use: first-time visitor, sometimes visitor, frequent visitor. The meat of the survey is likely to be determining which elements of the site are most used, along with some assessment of the effectiveness and enjoyment derived from using them. This question might be represented as a list of site sections, each with a sliding scale measuring the success of use. Finally, an open-ended question inviting general feedback is always a good idea. Although the information is difficult to analyze, a simple read of the responses will yield common themes that may be useful for planning.

Interviews

In-person techniques open the door to more accurate and detailed information gathering because of the opportunity for interaction and exchange. In an interview, you inquire into user goals, interests, needs, and behaviors. One effective method for understanding user behaviors is to ask users to give a verbal walkthrough that describes their typical interaction and task flow when accomplishing a goal. For example, in designing a new dating web site, you might ask users from your target audience to describe their current process when interacting with a similar web site and how they make connections without the help of the web. The key is to get the user talking and keep him or her talking. Ask clarifying questions, pursue important details, and periodically sum up the important elements of the conversation. Don’t be too quick to speak when the conversation lags; pause, and give the user the opportunity to continue the discussion thread. Also, don’t be too rigid in sticking to the script; digressions may lead to valuable insights.

Focus groups

Focus groups have a very different dynamic. Whereas the interview is all about the individual, the focus group is about the collective—a group of users who share common concerns. Focus groups have a number of benefits, not the least of which is economy; they offer an opportunity to hear many different perspectives at once. Also, the collaborative nature of focus groups helps people contribute insights that they may not have been aware of on their own. One user raises an issue that resonates with another, and the second user picks up the issue and takes it farther, and the next farther still.

Field studies

Observing users in their natural setting yields valuable insights and objective information about user behaviors and the efficacy of designs. With observational methods, we move beyond self-reported goals and behaviors to observed goals and behaviors.

In a field study, you observe users working with a system, typically in their own context. For example, in designing a library catalog system, observation of library patrons as they navigate the library is an excellent way to understand how they work with online catalog systems and how such systems help or hinder them in achieving their goals. Field studies are particularly helpful at the beginning of a project. For a redesign, observe users working with your current system to identify points of conflict between the design and user goals. For a new web site project, observe users working with a similar web site.

Guiding design decisions

Design is a process of making decisions. When user research is part of the design process, those decisions are guided by user perspectives and insights gleaned from observation. The following methods ensure that those insights and perspectives remain part of the decision-making process.

Personas

Personas are fictional representative users placed in narratives to stand in for real users during the design process. Personas arise from the research phase and are informed by surveys, interviews, focus groups, and other research methods. A persona typically includes a name, demographic information, level of expertise, and such details about the fictional user’s platform as connection speed, operating system, and browser software. Personas also include detailed information about user goals and motivations.

For universal usability, personas should push the boundaries of the fictional “average user” to include a range of use contexts and user motivations. Personas should represent the full range of age groups, computer expertise, and access technologies, including mobile devices, the wide but short screens of typical laptop computers, and screen reader software for nonvisual access.

Sample persona: Emily Johnson

Photo of older woman standing outdoors holding a tray of flowers and smiling at the camera.

Veteran investor, novice web user

Age: 72
Residence: Cape Cod, Massachusetts
Internet experience: Novice
Primary uses: Banking, email
Favorite sites: Fidelity, AOL
Average time online: 3 hours/day

“I love the independence of the web,  but I wish it wasn’t so hard.”

Emily has been retired for eight years. She taught grade school in the Watertown school system for forty years. She learned to budget her finances by raising a large family on a small public school teacher’s salary. When she retired, she sold her house of forty-three years and moved to an inexpensive condo. With the income from her house sale, she began to explore investing. Initially she worked through an agent, but then she began managing her own investments through the web. Emily has had great success, more than doubling her initial investment over the course of ten years.

Emily is often frustrated when using web sites. She has an intuitive sense about investing and thoroughly understands the mechanics of the process. However, she sometimes makes mistakes when managing her accounts online. When she encounters difficulties, she blames her physical shortcomings—her arthritis, which makes it difficult to manipulate interface elements, and her failing vision, which makes it difficult to read.

Goal analysis

A person uses a system in response to specific goals he or she is trying to achieve. When we roll out of bed and into the shower in the morning, our goal is not turning on the water; our goal is to get clean and wake up. It’s the tasks we undertake to accomplish this goal that involve water, faucets, and soap. In design, it’s important to focus on goals rather than tasks in making design decisions. A focus on goals helps us think outside the box and create better designs. Instead of designing a shower from the perspective of the tasks it needs to support, we can focus on the goal of getting clean and waking up, which opens the door to new approaches.

For example, let’s say our goal when interacting with a travel web site is to get from Boston to Barbados without spending too much time or money. To accomplish our goal, our tasks are to compare flight schedules and prices. The designer who designs for our goal will understand that time spent in transit and cost are the primary factors and will design an interface that shows information in a way that permits easy comparisons. Other information related to the trip, such as meals served or type of aircraft, is secondary and available on request.

Screenshot of the Travelocity search results page showing a list of hotels. — Figure 2.3: Complex data displays can easily overwhelm the reader. Travelocity’s generous use of white space and the carefully honed interface of its search results pages help make a complex list manageable and unintimidating.

Goals come in many shapes and sizes and emerge from different perspectives. Users have both personal and professional goals: being more productive, having fun, saving time and money. Organizational goals also vary: increasing revenue, recruiting more and better employees, offering more services. With any web site project, user goals should be the driving factor behind design decisions and the compass to steer by throughout the design process.

Scenarios

Scenarios, or use cases, are brief narratives that tell the story of a particular user’s path in accomplishing his or her goals. To construct a scenario, we use personas representing our users and walk them through the various tasks necessary with the web site in order to accomplish their goals. In composing a scenario, we can play out different approaches to design and functionality and in the process identify possible problems with each approach.

Evaluating designs

The design phase is an iterative one in which we conceptualize designs to create mock-ups, which we then evaluate and refine, often repeatedly, before arriving at a final design and finished web site. Consulting with users during the design phase allows us to home in on a design that presents content and functionality in a shape that maps to user goals and behaviors, and leads to successful and enjoyable user experiences.

Wireframes

Wireframes provide low-cost and highly effective support for iterative design. Wireframes are easy to create and, most important, easy to change—far easier than a coded web site or web application. The wireframe phase should provide a conceptual screen design and task flow and, through testing, evolve into a solid interface and task flow to hand off to programmers.

Spend ample time creating and testing wireframes. By virtue of their malleability, they are your best defense against poorly conceived, ineffective designs. Once you move your web site or application into development, changes to the user interface and task flows become more costly and are therefore less likely to be implemented.

Your initial wireframes might be simple sketches on paper or a whiteboard, to help conceptualize the user interface and task flow. Paper prototypes provide an inexpensive method for collecting user feedback on an evolving design. A paper prototype typically illustrates the location of page elements, such as site navigation, search, and content, and includes the labeling that will be used for these elements. These low-resolution mock-ups can be used in user research to determine whether the organization of the page and the navigation labels are easy to use and understand. For small projects, these may be sufficient to move a project from concept to development. Most projects, however, will move from sketches to diagrams created using software such as Adobe Illustrator, Visio, or OmniGraffle. These wireframes are best for complex projects with many design cycles because they are easier to modify and easier to share and distribute among the design team (see fig. 4.23).

Prototypes

A prototype is a set of wireframes used to simulate a functioning application. The prototype models the purpose of the application, its flow, and its patterns for interaction. With this model, you create an environment for walking a user through a task and identifying points of confusion or difficulty. You can then produce new wireframes to address any problems and test the prototype again (and again and again).

Paper prototypes consist of a set of wireframes, each on its own page (or index card), and can be used to conceptualize the flow of an application. Paper prototypes can be used for usability testing early in the design process, before any screens are built or code is written. To test a paper prototype, place the first screen of an application—say, the login screen—in front of the user and ask for feedback—what action would you take? Then replace the first screen with the one that would result from the user’s action, and continue working through the various stages of the interaction. Ask for feedback all along the way, and spend time after the session collecting additional feedback and suggestions.

A “functional” prototype provides an intermediate step between a rough sketch and a fully designed web application and can furnish a framework for moving from the conceptual to the design phase of a project. html wireframes form the basis of this high-fidelity prototype, with the essential elements of the application represented on separate web pages. The “functional” features of the application are simulated using basic links among and between pages, allowing users to experience and respond to the flow of the application. The html prototype can be refined in response to user feedback and then retested and further refined. Once the functionality and back-end systems are fully developed and tested, the visual design and web site interface can be “poured” into the wireframe to produce a finished design.

Functional wireframes use the same minimalist design as paper prototypes, but on a functional web site. They take more time to develop than paper prototypes but are well worth the effort, particularly for sites that are built on complex information architectures and for web applications that contain high levels of interaction. For a functional wireframe, several layers of the web site are established to model the site’s architecture, navigation, and functionality. For interactive sites, such as web applications, the wireframe should include a basic user interface in order to play out and test the flow of the application with users.

Include content in your user research activities. For example, use real content on your wireframes and prototypes rather than “lorem ipsum” dummy text. When you are seeking feedback on wireframes or prototypes, ask users questions about content, such as “What are the writer’s goals for this content? Who do you think is the target audience for this content? What would you expect a user to do after reading this content?”

Usability testing

Usability testing is a controlled and directed observation of user behaviors when working with a design. Usability testing is used throughout the design process to evaluate different design approaches by observing how well, or how poorly, they work in helping users accomplish tasks.

A typical usability testing session has a tester and a participant who represents the target audience. The user is assigned a set of tasks intended to put specific elements of a design to the test and reveal any shortcomings of the approach. During the session, the user is asked to think aloud so that the tester can understand the rationale behind his or her choices. The session normally ends with an open-ended interview in which the user is given the opportunity to discuss his or her experience with the system more broadly and provide insights and suggestions on how the system might be improved.

Usability testing is valuable throughout the design process for identifying usability problems. The lessons learned from usability testing can be used in refining the design, improving usability, and ensuring that people can find and make use of the content.

Like accessibility, many development teams leave usability testing to the end of the project or, worse, do not budget in time and resources for usability testing, much less usability testing that involves people with disabilities. The inertia around usability testing appears to be due to a misconception that “real” usability testing must be complex and time-consuming, and must involve a usability lab, expensive equipment, lots of participants, and a bulletproof test design. While a lab and resources and trained researchers certainly are advantageous, the lack of these is no reason not to do usability testing. The gains from informal usability testing with one participant are exponentially greater than doing no usability testing at all.

In his book Don’t Make Me Think, Steve Krug promotes do-it-yourself usability testing, in which you “do your own testing when you have no time and no money,” and his book Rocket Surgery Made Easy describes in detail how to do it. His approach encourages short but frequent sessions with few people—one morning a month with three participants, followed by a debriefing session to decide what to fix.

Involving people with disabilities in research activities

In Change by Design, Tim Brown suggests that innovative products are the outcome of observation, and suggests that we can learn a great deal by moving away from the mainstream and observing people who have what might appear to be singular needs and preferences. He explains, “By concentrating solely on the bulge at the center of the bell curve…we are more likely to confirm what we already know than learn something new and surprising.”

Including people with disabilities in user research and usability testing is a great way to gain insights into issues that may seem unique but can have wide-ranging influence on the quality of user experience. Many examples of design that have arisen out of accommodating the needs of people with disabilities have improved quality of life for everyone: entry ramps, automatic door openers, lever handles, high-contrast colors. By including people with disabilities in your research activities, you will learn how to make your web site more accessible and usable for people with disabilities. You may also identify opportunities to provide accessibility features that will improve the user experience for everyone in your audience.

Beyond opportunities for innovation, there are many practical benefits to involving people with disabilities in user research. In some cases, there are standards that require or encourage consultation with people with disabilities, including, in the United States, the 21st Century Communications and Video Accessibility Act (CVAA), and, in the United Kingdom, the British Standards Institute Standard bs 8878. User research activities that involve people with disabilities will help meet obligations as well as highlight accessibility issues. The development team can work to resolve these issues. Any unresolved issues should be recorded according to the process standards, and be the focus of a defined plan for remediation and resolution.

With accessibility, adhering to defined standards and guidelines, such as the Web Content Accessibility Guidelines (WCAG) 2.0, can help meet legal obligations and improve usability for people with disabilities. However, user research can uncover additional issues that are not addressed by standard compliance, and that can be improved through design. Observation and inquiry can help identify solutions that go beyond standards compliance to create accessible and enjoyable user experience.

For many designers and developers, accessibility is a set of guidelines and checklist rather than something direct and experiential that provokes feelings of connection and empathy. Most people on web development teams don’t know how a person who is blind uses, for example, a touchscreen, or how someone who doesn’t use a mouse works the controls on a web page. This reflects not a lack of concern but simply a lack of exposure. By involving people with disabilities in user research activities, designers, developers, strategists, and writers can see firsthand how people use digital products and services, the issues they encounter, what works well, and what doesn’t. Most designers and developers love solving problems, and will be eager to resolve problem features once they know their impact.

Analytics

Web technology collects metrics about users: what operating system and browser they use, their screen resolution, what page they visited just before their arrival at your site. And although it’s certainly useful to know these attributes, they are not necessarily helpful in defining the audience for your web site. Web metrics will not tell you precisely why users visit your site, what they hope to find from your site, or whether they are visually impaired, expert or novice, young or old.

In the end, even with the best web analytics, many things about your audience’s hopes, motivations, and expectations will remain a mystery if you rely solely on web metrics to understand your users.

In contrast, a target audience is a group of users that you have identified as critical to the success of your site. For instance, you may be designing for a certain age group, such as grade school children, teens, or retirees. Or you may be designing for a specific technology, such as mobile devices. Bear in mind that members of your target audience may share common interests, but they are not likely to share access requirements. Some may be experts and others first-time users. Some may have impaired or no vision, and others may have mobility or dexterity issues. The same person may access your site on a laptop, table, or smartphone. And although you may target a certain audience, others will come. For example, an investment service for retirees will also draw visits from investors, competitors, family members, and those lucky enough to enjoy early retirement. It would be a mistake to design such a site to meet only the needs of older users.

And there is much at stake when you exclude users. Even if your web logs show that only 2 percent of your users use a specific brand of browser, don’t make the mistake of using technology that excludes those users. It’s bad business to exclude anyone from access to your information and services, and there is no way to place a value on those users who you have excluded. Who knows? Your next major donor might be one of the 2 percent you turned away at the door!

Using analytics as a tool for decision making

With each page served, web servers collect basic information about the user and save the information in a server log file. Web analytics involves using the data in the server logs to study user behaviors. More advanced web analytics make use of additional tracking techniques that usually involve embedding something on the client—a “bug” on the page or a session cookie—that enables collection of additional information. With web analytics, it’s possible to reconstruct elements of a web site session, such as:

Where the user was before arriving at your site
Whether the user used a search engine, and if so, which keywords were used
The sequence of pages the user visited while on your site
Attributes of the user’s configuration, including browser, operating system, and screen dimensions and color settings
Whether the user was a new or repeat visitor
The last page in the session before the user left your site

Screenshot of Google Analytics dashboard page showing session data using a line graph and data table. — Figure 2.4: Web analytics tools like Google Analytics are crucial for making informed decisions about the current usage patterns of your site. It is extremely important to collect good analytics data before planning a major site redesign, or you’ll never have the baseline data you need to judge whether your new design is an improvement.

These and other site-specific metrics (the number of page requests helps determine which pages are most used; the number of distinct hosts helps determine how many site visitors there are) offer important insights into how users are working with your site, which is essential to any redesign process. But bear in mind that although web analytics appear to measure what is happening on your site, the reality is far more complicated. Many factors skew the data, such as browser caching, in which pages are stored locally on the client machine. When a user visits a cached page, that visit will not be logged in the server log file, since the server was not asked for the page. Use web analytics, but don’t treat them as the last word on user behaviors. Use them to complement the other user-centered techniques described in this chapter.

A/B Testing

There is no “silver bullet” in web site design—no single solution that works for all users. Often design teams are faced with several approaches and must make a decision on where to focus energy and resources. In these cases it can be helpful to use A/B testing, in which different versions of content and features are launched and evaluated for effectiveness.

The key to successful A/B testing is a clear set of measures that the team and stakeholders agree represents success in achieving project goals. These are typically derived from an increase in engagement, such as when more people sign up for a newsletter, click an advertisement, or purchase a product. They can also be based on user journeys, where each design affords a different path through the site. Through analytics, the team can assess which path is more direct in getting users to their destination.

Chapter 2 Research