Читать книгу Remote Research - Tony Tulathimutte - Страница 2

Dedications

Nate: To my family, friends, colleagues, teachers, and loved ones

Thanks for listening to me ramble on about this field for solong.

Tony: To Mike Hardnett and Renee Zalles

How to Use This Book

Who Should Read This Book?

This book is about remote user research, which is a method of using Internet tools and services to conduct user research with participants who are in another location. (User research, in turn, is the field of studying how people interact with technology.)

Are you a user experience/human-computer interaction practitioner? If so, you’re totally gonna love this book, especially if you’ve ever been frustrated with current in-person or lab methods of user research for any of the several reasons we describe in Chapter 1. If you’re a software or Web developer looking for insights into your own (or your competitors’) designs, or an interaction designer or consultant, you’ll probably dig this book too.

Is there anyone this book isn’t for? You don’t need to be a veteran user-experience researcher to understand what we talk about in this book, although we do focus mostly on the “remote” aspects of remote research. You won’t find much advice on how to conduct user research in general—for that, a great place to start is Mike Kuniavsky’s Observing the User Experience.

What’s in This Book?

Remote Research is a how-to book about remote research methods: using a phone and the Internet to conduct user experience research from a distance.

In the Introduction and Chapter 1, you’ll get an overview of what remote research is all about, when you should and shouldn’t use remote methods, and the two main kinds of remote research studies: moderated and automated.

In Chapters 2 through 5, you’ll learn how to set up, recruit, and conduct a basic remote moderated study. We describe a method called “live recruiting,” which involves intercepting visitors to your own Web site to participate in your studies immediately. We also discuss the privacy and consent issues around recruiting, session recording, and remote participation.

In Chapter 6 we describe various automated research methods, illustrating them with case studies.

Chapter 7 provides advice about how to analyze and report on the findings of remote research studies.

Chapter 8 is a short guide to tools and services you can use to fill the many technological needs of remote research, including screen sharing applications, recording software, and several online automated research Web apps.

Chapter 9 illustrates how many of the basic principles described earlier in the book can be adapted and applied to special testing circumstances, where normal remote testing methods aren’t possible or desirable.

Chapter 10 is a review of the major challenges you’ll face when planning, conducting, and presenting a remote research study.

What Comes with This Book?

This book comes with two companion Web sites. The first is the Remote Research page

( www.rosenfeldmedia.com/books/remote-research) on the Web site of our publisher, Rosenfeld Media. The other site is our companion Web site for the book, Remote Usability (http://remoteusability.com). On both sites you’ll be able to find detailed resources, research document templates, supplementary articles, and up-to-date lists of remote research tools and services.

We’ve also made the book’s diagrams, screenshots, and other illustrations available under a Creative Common license for you to download and include in your own presentations. You’ll find the original illustrations and diagrams from this book at http://www.flickr.com/photos/rosenfeldmedia/sets/, or you can just double-click the pushpin next to the image to see them in high resolution.

Frequently Asked Questions

What is remote user research, anyway? Is it anything like focus groups or surveys?

Remote user research is simply a type of user experience (UX) research that’s conducted over the phone and Internet, instead of in person. In general, UX research seeks to understand how people interact with technology. Unlike focus groups and surveys, market research techniques that are used to learn people’s opinions and preferences, UX research focuses on studying people’s behavior. In that sense, remote user research isn’t really like market research; however, both remote user research and market research can be applied toward improving the design of existing technologies and inspiring new ones. See Chapter 1, “The Appeal of Lab Research”.

What kinds of remote research are there?

That’s a huge question, and we spend a good chunk of this book introducing and describing the many varieties and specialties of remote research out there. In general, there are two branches of remote research: moderated and automated. In a moderated study, a researcher talks directly to the participants as they use the interface in question, and it’s good for obtaining rich, qualitative feedback. In an automated study, you use online tools and services to gather behavioral or written feedback and information automatically, without the researcher’s direct involvement. For more about moderated testing, see Chapter 2, “Preparing for a Real Study”, and all of Chapter 5. For more about automated testing, see Chapter 6.

I’m skeptical about remote research. If it’s so great, why haven’t I heard of it?

A lot of the misgivings that people have about remote research come from its novelty. The method is still cutting edge, and the technique requires a certain degree of know-how. Until now, there hasn’t been a book you could learn the method from—which is, of course, the reason we wrote it.

Still, lots of companies have done it, with great success. We’ve done remote studies with Sony, Autodesk, Greenpeace, AAA, HP, Genentech, Wikipedia, UCSF Medical Center, the Washington Post, Esurance, Princess Cruises, Hallmark, Oracle, and Blue Shield of California, among many others.

I’m still skeptical. Can you really get valid behavioral feedback without seeing your participants in person?

Since remote research is conducted over the phone and Internet, many people worry about missing “rich details” like facial expressions and body language. First, we believe that for most user research studies, the way that users interact with the interface and their think-aloud comments are the only really necessary things to focus on. And on top of that, much of the tone does come through the user’s voice and language. We weigh the pros and cons of in-person research and remote research in Chapter 1, “Is Lab Research Dead?”, and discuss moderating over the phone in Chapter 5, “Ain’t Nothing Wrong with Using the Phone”.

Still skeptical? Then you should check out our exhaustively documented study for Wikipedia, complete with full-session videos and highlight clips at http://usability.wikimedia.org/wiki/Usability_and_Experience_Study. It includes both lab and remote sessions with identical goals, so it’s a good comparative case study.

The best way to see if a remote study is for you, however, is by getting your feet wet with a quick, painless pilot study, which we’ll walk you through in Chapter 2, “Doing a Pilot Test Right Now”.

I want to cut costs for my user research study. Is using remote methods a good way to do that?

Not really. You might save on costs related to travel expenses, renting a lab facility, or hiring a recruiting agency, but then there are the expenses of the specialized remote research tools and services you’ll need, and where the researcher’s time, participant incentives, and project timeline are concerned, nothing is much different. We cover the equipment requirements of a basic moderated study in Chapter 2, “Gearing Up: Physical Equipment”, and the costs of many remote tools and services in Chapter 8.

Where can I get people to participate in my remote study?

You can technically use any methods to recruit for a remote study that you’d use for an in-person study: email contacts, recruiting agencies, and even craigslist ads (blegh!) are still an option. However, in this book, we introduce a method called live recruiting, with which you intercept visitors to your Web site by using a pop-up form to get them to participate in your study right away. We strongly believe this approach is ideal for remote research because it allows you to do what we call “Time-Aware Research,” which we introduce in Chapter 1 and discuss in depth in Chapter 3.

How on earth can you call this a book about research without way more academic references and doctoral degrees?

This book is aimed at people who want to hear practical information about how remote research is done in the real world.

We don’t claim absolute peer-reviewed scientific rigor, but we are sharing what’s given us the best results after almost a decade of remote research experience: 2,615 moderated users, 2,676 automated users, 234 projects, 19,120 project hours, and 89 clients to date. Simply put, this is just what works for us, and we think it will work for you, too. (And for what it’s worth, Nate and Tony both have cognitive sciences/human-computer interaction degrees: yes, we roll that deep.)

Foreword

Peter Merholz is a founding partner and president of Adaptive Path, an experience strategy and design firm

(www.adaptivepath.com). He coauthored Subject To Change: Creating Great Products and Services for an Uncertain World, published by O’Reilly.

You are holding in your hand a portal into the future. The approaches discussed in this book are the first steps toward a user research methodology suited to our technological context. In comparison, lab research will be increasingly seen as an archaic approach to understanding people, akin to phrenology or trepanning.

You see, lab usability engineering was born of a simpler time. Files were stored on floppy disks, and the computer wasn’t connected to a network. People used computers for Calculation (basic math, spreadsheets, etc.), Creation (word processors, graphics programs), and Capture (data entry). Computers would have only one “program” running at a time. And someone could be expected to focus on a single task at hand for many minutes, if not hours on end.

These days, our customers’ technological world is much more complex. The bulk of their time online is spent engaged in Consumption (browsing the Web, listening to music) and Communication (email, instant messaging, Twitter), though they still Calculate, Create, and Capture. They have multiple applications open and multiple windows within those applications. Thousands of files and emails fill their hard drives, and they’re managing multiple devices, including computers, mobile phones, iPods, and digital cameras. Technologies have driven users to a point of extreme distraction—recent research has shown that workers are interrupted an average of every 11 minutes.

Although the world has changed, the methods of standard usability practice are essentially the same as were practiced in the early 1990s. We recruit participants who satisfy demographic requirements, invite them into a fluorescent-lit lab with an imposing mirror along one wall, ask them to use a computer they’ve never seen before, have them engage in a set of scripted tasks, and “think aloud” while doing so. These participants won’t have an IM window pop open unannounced, nor be able to click on the bookmark bar to see their Facebook page, nor be able to pull up the notes they saved in a draft in their email. When everything about the observation environment is so unnatural, how can we expect our findings to provide legitimate insights?

This is where the genius of Nate Bolt and his crew come in. Since I’ve known them, they’ve experimented with technologies in an effort to capture the most authentic user research data. Whether it was the pioneering screen sharing and recording tool Ethnio (which fundamentally changed how I considered usability), or real-time uploading of video captured in a drive-along study for Volkswagen, their mission is to deliver the unvarnished truth.

Now, it’s easy to get caught up in the tools. While remote research is enabled by nifty technologies, which should be of secondary interest at best, I know that for Nate and Tony, this research approach is about mindset. When you simply desire to understand people in the most direct ways, you come up with clever means of doing so. And sometimes, you can’t go “remote.” When I first met Nate, he crowed about how they kitted out a usability lab with hidden microphones and cameras so that the participant would be as comfortable as possible. Or when researching the use of multiplayer video games, he recreated a cozy living room set up in his office, with snacks and lounge chairs, to make gamers feel at home.

And that’s how this book is a time machine. Only the Flying Spaghetti Monster knows what the future will bring. The only thing that’s certain is that it’s going to keep changing. To prepare for that future, you need the mindset represented in this book, in order to figure out which approaches will provide the best user research data given your circumstances. Like Bolt | Peters, don’t settle for standard practices; instead, play with new ideas that will deliver fresh insights.

—Peter Merholz