Читать книгу Engaged - Amy Bucher - Страница 39

Having a measurement plan is crucial for the success of your product. You want to tell a compelling story about why your product is great, and the data you gather with your measurement plan will help you tell that story with conviction. Metrics allow you to determine whether your product works, how much people like it, and what the most effective ways to improve it would be. If you have a B2B model for your product, metrics help you sell yourself to companies who want to get the positive outcomes you can provide for their people. And whether you’re distributing through a B2B or B2C model, success stories interest people in becoming your users.

Perhaps counterintuitively, the most effective metrics are planned at the very start of product development. Doing this ensures that you can build the right hooks into your product to collect the data you’ll need, and that you include the right content and features to achieve the results you want. A tool like an outcomes logic map can help you plot out all of the steps that will need to happen to make your product effective. It will guide you in effectiveness research to determine how your product works, as well as research to investigate what your next iteration should look like. And the upfront planning will help you work effectively with IRBs or other reviewing bodies to ensure that all of your research is done in a way that respects users and maintains their trust.

PERSPECTIVE Cynthia Castro Sweet and Pragmatic Scientific Rigor

In the digital health world, one company stands apart for the strength of its outcomes story: Omada Health. As of this writing, they’ve published 11 studies in peer-reviewed journals (that’s a lot), and have launched a large randomized controlled trial of their diabetes prevention product. Their work on creating an outcomes story has translated to business success, with Omada boasting one of the highest venture capital fundraising totals in digital health and a top-notch list of clients. A driving force behind Omada’s ongoing research program is Dr. Cynthia Castro Sweet. I was interested in talking to Cynthia to learn how Omada has become a leader in telling their story with data. What can other teams learn from Cynthia’s experience at Omada?

How can you leverage existing evidence?

There’s a big body of literature out there around diabetes prevention programs, but we need to show what our specific product does. You can draw a line from the original Diabetes Prevention Program (DPP) format to the way Omada has implemented it and show apples to apples. By producing our own evidence, we are instilling reassurance that we’ve held up the integrity and are faithful to the essential elements that made that product or service work in an older, traditional format.

Well-designed, well-conducted scientific study of the product is necessary. You may be able to break into the market with a really cool product, but you’re not going to make bigger transformations unless you follow some of the traditional rules of health care. If you’re taking something that came from a more traditional format and you’re bringing a technological revolution to that, you still need to prove yourself and show what your product can do.

How can you balance research with product development?

I use the phrase “pragmatic scientific rigor” pretty frequently. What I aim for is the best science possible, understanding all the other conditions that are in place for whatever stage the product is in. Sometimes, it’s necessary to get your product out there while there’s interest in it, even if you haven’t done years of highly crafted validation studies on it. A lot of behavior scientists struggle with dialing back their desire to be pristine with their science and trying to marry good science with the other pressing contingencies of product development.

It’s part of my job to understand our product vision, what we’re aiming for, what the benchmark is in traditional medicine that we are targeting with our program, and when those elements are scheduled to launch. Then I know what I can make happen and when, in terms of evidence generation.

How do you use user data for research?

Our standard setup allows us to use participant data for our product improvements. So we’re watching to see how they’re responding to different features and what’s being used and what isn’t to help us improve the product. When we want to use their data for public-facing research or evidence, then we need to layer in a separate level of consent, permissions, data use, and sharing. I need to show that I’ve done my due diligence and gotten permissions, and am acting responsibly and ethically in the use of that data. I use an IRB for using my own company’s data quite often. We need to be very mindful about when we’ve collected permission to use data and for what.

How does the science message change for different audiences?

We actually write the same story for different audiences. It’s more of a process of distilling and tailoring the right message for the audience. We tend to come out with what we call our power statements, undisputed facts. Then our communications, marketing, brand, and creative teams will craft them into messages for different audiences. Then we have another team of people who review from all angles to make sure what we’re saying is true, accurate, appropriate, and can stand up under scrutiny.

And it’s not always just about the evidence. There’s the user experience, the implementation process, the marketing—there’s a bunch of other pieces that you have to weave together to tell the right story. We’ll shape the message toward what is most salient and what information we think is most accessible and important for the audience and their decision-making.

Cynthia Castro Sweet, Ph.D., is a health psychologist and behavior scientist. After earning her Ph.D. in clinical psychology from the University of California, San Diego School of Medicine, she took a series of research roles focusing on improving health habits in diverse populations. Cynthia worked for the Stanford Prevention Research Center before joining Omada in 2015. She is now Omada’s Senior Director of Clinical Research and Policy. Cynthia’s research at Omada focuses on externally validating the efficacy of their programs, particularly their flagship CDC-recognized digital Diabetes Prevention Program.