Читать книгу Research in the Wild - Paul Marshall - Страница 10

CHAPTER 2

Moving Into The Wild: From Situated Cognition to Embodied Interaction

2.1 INTRODUCTION

The phrase “in the wild” first came to the forefront, in the late 1980s and early 1990s when anthropologists Lucy Suchman (1987), Jean Lave (1988), and Ed Hutchins (1995) began writing about cognition in the wild. Collectively, they critiqued the fledgling field of cognitive science, which was concerned with how the mind worked. The accepted theorizing at the time focused on information processing in the head, and the construction of rational models of behavior as the execution of plans. In sharp contrast to this classical view, they explain cognition—as observed in everyday practice—being distributed and situated in the moment. Moreover, in their respective books (see Figure 2.1), they cogently argue that cognition can only be studied in the wild. Their approach was to present a social anthropology of cognition and cognition in practice, respectively.

Figure 2.1: Suchman’s, Lave’s, and Hutchins’ classic “In the Wild” books.

2.2 PLANS AND SITUATED ACTION

The first in the wild classic was Lucy Suchman’s (1987) Plans and Situated Actions book. It took the fields of HCI and computer science by storm—and its insights were quickly adopted by a new generation of researchers and students. For many, it resonated with their discontent and worries about the limitations of traditional cognitive models. For others, it opened their eyes to new ways of thinking about human-machine interactions. The accepted view at the time was that scientific models were needed to explain how the mind works and that these should form the basis of user models used in machine-human dialogues. Folk theories or common sense explanations were dismissed as inadequate. Suchman, however, argued the opposite: common sense notions of planning should not be viewed as inadequate versions of scientific models of action, but taken as resources people use in their practical deliberations. To support her contrarian view of how to conceptualize and understand human behavior, she described how people use these resources along with various constraints in the environment in their everyday planning and action. Instead of developing so called scientific models to develop human-computer interfaces, developers should draw from accounts of how people act and react in their everyday lives.

Much of Suchman’s early research was to provide detailed situated accounts of the relations among people and between people and technology. One of her most cited examples is of a study she conducted of pairs of users trying to fathom out how to use a Xerox photocopier. While not an in the wild study (since it was conducted in a Xerox Research lab), she noted how its complicated help system did not match the way the pairs understood how it worked or what to do when it did not work in the way they thought it should. The outcome of her detailed analyses of the mismatched photocopier-user interactions led many programmers and developers to rethink how they should structure and what to include in their human-computer models, replacing the simplified process models, that followed sets of rules, such as “if x then y” with alternative kinds of situated models of action (Dourish, 2001).

An analogy that she used in her book to illustrate what she meant by situated action is a description of what it is like to ride the rapids in a canoe. She notes how a great deal of deliberation and reconstruction goes into a canoeist’s plan both before they begin and in their account of what happened after the event, but from then how they actually navigate the rapids, depends on embodied skills in responding to whatever comes their way. This powerful image resonated with many as to why models of plans as a control structure that specify behavior were inadequate when designing user interfaces. Despite its impact on a generation of researchers, however, this example, itself, has been somewhat parodied and often misunderstood. Many took it to mean plans are irrelevant to how we act. Suchman never claimed this (and goes to great length to explain what she meant in her later revised version of the book), arguing that what happens in practice is the interaction of both the contingencies and the projected course of action. A legacy from her pioneering work is the commonly accepted view that users don’t follow instructions and plans as simply as had been assumed.

2.3 COGNITION IN PRACTICE

Jean Lave’s (1988) book Cognition in Practice, published a year later, was primarily concerned with debunking the academic snobbery associated with “common sense explanations and real-world contexts.” Similar to Suchman’s critique of cognitive science models of everyday planning, she went to great lengths to explain how experimental lab research wasn’t superior to everyday people’s accounts of what they do in their lives. Moreover, her program of research showed how it was more valuable and legitimate to study people’s cognitive behavior in everyday contexts, which she described as “cognition in the wild.” To demonstrate how her approach could provide new understandings, she studied adults practicing math in a variety of real-world contexts. Some of the examples she described in her book, which are most illuminating, are of people working out the best price for groceries when shopping in the supermarket and for how dieters measure unusual quantities of ingredients when making a dish at home while following a recipe. Similar to Suchman’s book, she compellingly demonstrates, through her detailed case studies, how people often use opportunistic structures in the real world in their everyday cognition.

The legacy of Lave’s work was to show how it was possible and necessary to move one particular form of cognitive activity—arithmetic problem-solving—out of the laboratory back into the realm of everyday life. In so doing, she showed how mathematics in the real world is the same for all kinds of thinking, shaped by the reflexive encounter between human minds and the context people find themselves in. A salient example that has been much cited—as illustrative of doing math in practice—is the “cottage cheese” problem; a male dieter, preparing a meal, was faced with having to measure out 3/4 of 2/3 of a cup of cottage cheese that was stipulated in the recipe he was using. How did he work it out? Not by multiplying 3 × 2 and dividing that by 4 × 3, resulting in ½, as would be expected if using algebra in school, but instead by using the available structures in the environment in a situated way. He first measured out 2/3 of a cup, and then spread it on a chopping board in the shape of a circle. Next, he divided the circle into 4 quarters, removed one of them and returned it to the container, leaving on the board the desired 3/4 of 2/3 a cup.

Similar to Suchman’s canoe example, the dieter example of using external resources to solve a math problem paved the way for rethinking cognition in practice rather than in abstraction, and the insight that mathematics is for something; the mathematical abstractions taught in schools don’t necessarily transfer well to use outside the classroom. A number of other examples in Lave’s book are used to emphasize how people use the resources from the context they are in to solve problems. Together, the examples convincingly demonstrate how activities in settings are complex improvisations that have much variability. Doing math when out there takes a different form in different situations. One of the outcomes of this early form of in the wild research was to make studying everyday and common sense reasoning acceptable, by giving it credibility and respectability (cf. Rogers, 1993).

2.4 COGNITION IN THE WILD

Ed Hutchins published Cognition in the Wild a few years later in 1995. His seminal book was also very much a reaction against the status quo; but more broadly than either Lucy Suchman’s or Jean Lave’s efforts. His beef was very much a rally against “cognition in captivity” and “disembodied cognition.” He argued that much of mainstream thinking about cognitive science for the past 30 years had resulted in systematic distortions of our understanding of the nature of cognition. Instead, he proposed that cognition should be studied in its natural habitat and that, in doing so, it would change our ideas about its nature. He argued that what was problematic with the classical cognitive science approach, was not its conceptual framework per se, but its exclusive focus on modeling the cognitive processes that occurred within one individual. As an alternative, he argued that what was needed was for the same conceptual framework to be applied to a range of cognitive systems, including socio-technical systems at large (i.e., groups of individual agents interacting with each other in particular environments). To do this, he proposed studying cognition beyond the skin of the individual, encompassing the distributed nature of cognitive phenomena across individuals, artifacts, and internal and external representations.

Hutchins also argued that in order to reveal the properties and processes of such a cognitive system required conducting an ethnographic study of a setting. Paying close attention to the activities of people and their interactions with material media was considered fundamental to understanding how such a cognitive system works. Hutchins’ intricate analyses of what happens inside a cognitive system at both the micro and macro levels were at the time groundbreaking. One of his most well known examples is an account of how the cockpit plus air traffic control tower system work together as a cognitive system. He illustrates this eloquently by describing the joint activity and accomplishment of a situation when a pilot and co-pilot fly their plane to a higher altitude in conjunction with listening to and talking with air traffic controllers. It demonstrates just how much coordinated activity depends on the orchestration of mechanisms through which co-located and distributed people make small signals to each other in order to progress a sequenced activity, and the levels of inter-subjectivity involved for different states of the system. A missed cue can easily result in a misunderstanding, especially if someone is not expecting it. These can happen even for the most routine of activities which then requires the cognitive system to engage in various forms of repair work and sometimes the adoption of workarounds to get the activity back on track.

The legacy of the distributed cognition approach is its demonstration of how insightful it can be to analyze the complex interdependencies between people and artifacts in their work activities—which is often overlooked in other kinds of cognitive analyses. Hutchins approach to “cognition in the wild” clearly showed how important micro-analyses can be to reveal the multi-layered work that a cognitive system has to do—and where subtle actions, such as a glance, a gesture, or a flick of a switch at a particular time, are often integral to the coordination and mediation of teamwork.

2.5 EMBODIED INTERACTION APPROACHES

Since these early pioneering “in the wild” theory books, there have been further writings in HCI and cognitive science that have stressed the importance of understanding the ways people are closely coupled with their environment. These include seminal works that view interaction as embodied (Dourish, 2001), cognition as external (Scaife and Rogers, 1996), and perception as enactive (Noë, 2004). In the 2000s, and to this day, conceptualizing human-computer interactions as embodied in real-world contexts has become an influential approach (Marshall et al., 2013). In its broadest sense, embodied interaction refers to the “everyday, mundane experience” (Dourish, 2001, p. 125) and the ways that people understand the world through their accomplishment of practical activities. However, many questions remain as to exactly what it means in practice and whether it can be considered a coherent program of research in HCI.

Many of the ideas about embodied interaction, which have been developed in HCI, built upon the phenomenological ideas of Husserl, Merleau-Ponty, and Heidegger. These earlier philosophical writings were largely concerned with the essence of what it means to be, to see, to have a conscience and be aware of the world. The “lived” experiences were accounted for in terms of many aspects, but primarily in terms of space, time, and what it means to live in the world. Likewise, McCarthy and Wright’s (2004) influential Technology as Experience framework, concerned with the “felt” experience of being in the world, was based on phenomenology, and in particular, the writings of John Dewey and Mikhail Bakhtin. Instead of describing HCI in terms of how usable a device or interface is, they argued for explaining user’s interaction with technology more in terms of their felt experience, i.e., how something is felt by them. In so doing, they make the case for understanding users in terms of their whole experience of a technology, especially how they makes sense of it in the context of use, by considering the emotional, intellectual, and sensual aspects of their interactions with technology. This stance emphasizes the importance of understanding how people do not just use technology, but that they also live with it.

Contemporary philosophers have also become interested in understanding cognition in the world. Particularly notable, is Andy Clark and David Chalmers’ (1998) “Extended Mind” hypothesis—the underlying idea being that the mind does not have to be contained within the brain or physical body, but can be extended to elements of the environment. The hypothesis suggests how technology continues to extend and increase what humans are capable of doing, enabling them to make more rapid decisions, understand complex situations, and find solutions to difficult problems. Since, Clark and Chalmers ideas have been taken up and elaborated by other philosophers, including Ward and Stapleton’s (2012) provocative paper, “Es are good. Cognition as enacted, embodied, embedded, affective and extended.” A central idea is that tools and artifacts are absorbed into the body schema, extending and changing it. An example that is often used to illustrate this idea of extended cognition is the blind man with his stick—where the stick becomes an extension of his arm/hand, extending the boundary of the space surrounding his body and perception of the world.

Alva Noë (2009) also wrote cogently about how perception is not like looking at pictures in the mind; instead we perceive the world by gradual active inquiry and exploration of it. A central tenet of his position is that given that we spend all our lives embodied and situated in the world around us, it follows that our perceptual experiences are acquired through our bodily experiences with the world. Hence, it does not make sense to understand the mind, consciousness, or problem-solving as something that occurs in the brain. We are in the moment in a shared context—whether it is a football match, having dinner together, or engaged in banter on social media. A challenge is how to take into account this context when designing new technologies to aid, augment, or provide new opportunities for cognition, social interactions or cultural experiences.

Technologies, such as augmented reality, virtual reality, and haptic feedback, have been able to provide philosophers with new interactive tools with which to investigate and validate embodied theories of body/mind. For example, Bird et al. (2009) explored how they could design tactile interfaces to mediate novel sensory information to enable people to experience the technology as an extension of themselves. Conversely, theories about embodied and the extended mind can inform the design of technologies that extend what humans can perceive and do in the world. For example, technologies have been designed to provide users with extended ways of perceiving the world, such as “Sixthsense” (Misty and Maes, 2009), which was a demonstration of a wearable gestural interface intended to augment the physical world with digital information that could be interacted with using natural hand gestures.

Both approaches can inform new theories about augmented cognition while also providing empirical evidence for embodied theories (cf. Rogers, 2011). For example, David Kirsh (2013) notes how, “The theory of embodied cognition offers us new ways to think about bodies, mind, and technology. Designing interactivity will never be the same.” He illustrates this bold claim with his research on how dancers use their bodies when rehearsing: where he demonstrated how they are able to learn and consolidate mastery of a complex dance phrase better by physically practicing a simplified but distorted model than by mentally simulating it undistorted (Kirsh, 2014). The idea that we think with our bodies, not just our brains, that in turn shapes how we think and solve problems is profound and has important implications for how we think about designing cognitive tools to think with and augment human behavior.

2.6 CURRENT THEORIZING WITHIN RESEARCH IN THE WILD

Whereas the anthropologists and philosophers’ alternative theories of cognition in the wild were largely pitted against cognitive psychology and cognitive science theories, that were mainstream at the time, today’s HCI researchers are largely concerned with technology in the wild, with no particular discipline to put to right. While a few sociologists, who had ventured into the field of HCI in the 1990s, railed against having any kind of theory about cognition, including a damning critique of Hutchins Cognition in the Wild (Button, 2008), many others embraced the ideas of explaining cognition as situated or distributed across technology, people, and artifacts—leading them to develop new conceptual frameworks from which to account for, analyze, and inform the design of situated technologies (e.g., Rogers and Ellis, 1994; Rogers, 1992; Halverson, 2002; Hollan et al., 2000; Furniss and Blandford, 2006; Liu et al., 2008).

At the same time, HCI research in the wild continues to discover how established theories of human cognition, largely derived from research conducted in the lab, are not adequate accounts of real-world behavior. A number of HCI researchers have found that old school cognitive and social theories do not describe or adequately account for how people interact with technology in their everyday lives, especially when considering how digital technologies and physical artifacts have now become so entwined in what people say, do, think, or remember (Rogers, 2012). For example, Bergman and Whittaker’s (2016) research on personal information management shows that classical theories of information management do not match up with how people actually manage their “digital stuff.” In contrast, based on their body of empirical work of what happens in the real world, they propose an alternative three-stage model of personal information management, where curation is viewed as being at the core of how people store, retrieve, manage, and exploit their data—be it via their phone, computer, laptop, or other device. They suggest this alternative theorizing can provide new insights and principles for how to design new digital management and navigation tools—that differ from existing approaches, such as tagging, searching, and grouping. Furthermore, they point out how people’s curation behaviors persist over time—despite changes in the technological devices they use, together with the exponential growth of digital content they create, keep, and want access to. Many of the problems people have organizing, storing and re-accessing their email are the same ones they have with their photos, personal data, or files. We are creatures of habit and they argue we need to design our technologies accordingly—rather than take existing theories of how to optimize information management/retrieval.

While the situated, distributed, and embodied theories have provided new understandings and framings of human activity in the real world, they only go so far. What is also needed, besides new theories of cognition to replace the old classical ones (cf. to Bergman and Whittaker’s approach, 2016), is to rethink theory more broadly, both at macro and micro level of analyses, to account for how people are using, relying on, and appropriating the diversity of technologies that have become suffused in their lives.

One way to achieve this is to explore the interdependences between design, technology, and behavior. While this approach is not new—for example, socio-technical systems theories has been around for years—the subject of interest is, i.e., theorizing about people’s everyday use and interactions with technologies and their environment. Another way is to begin theorizing about how digitalization, in its various manifestations, is affecting society. For example, consider the growing concern in society about whether children’s reading skills are declining. In particular, a question has been raised as to whether the practice of bedtime reading (which is considered instrumental to helping children learn to read independently) is changing through the widespread take up of interactive ebooks and tablets. A study conducted by Nicola Yuill and Alex Martin (2016) investigated how to operationalize the wider context of understanding children’s reading skills. They wanted to know if it matters whether a traditional paper book or a tablet screen is used for bedtime reading. Are there differences in their affordances and properties that affects the age old practice of parents and their children reading a bedtime story together? To answer this, they carried out a controlled experiment in a naturalistic setting. They came up with a number of indices to describe when children are reading and being read to, in order to see if there were any differences between shared reading of digital and paper texts. The measures they used were for: cognitive aspects (e.g., do they differ in their attentional engagement), interactive and affective aspects (e.g., are there differences in the warmth of mother-child interactions when reading screen and paper media?), and postural aspects (are there differences in the physical positioning of mother and child when reading from screens vs. paper?).

The experimental design drew heavily from developmental theory and experimental design. The theory of joint attention was used to frame the design of the study to explore these aspects. An in situ study was then conducted to answer the questions—by observing and recording the joint attention between parents and children when sitting on a sofa together in their own homes reading a book at bedtime. Much thought went into the selection of the participants, the materials used, and the length of reading with the use of a repeated-measures design, using four conditions (Mother-Paper, Child-Paper, Mother-Digital, Child-Digital).

Reading errors and recall of material were collected and then coded, providing specific measures of richness of description and narrative coherence. The findings from the study revealed a number of differences, for example, they found that reading interactions involving a screen showed slightly lower warmth than those with a paper book. However, tellingly, they found no differences in the narrative and descriptive aspects of story recall for stories shared on paper or screen, whether the mother or child was reading. Hence, in contrast to the lab experimental paradigm, where hypotheses that are found to be statistically insignificant are considered to be a failure and often not published, Yuill and Martin’s (2016) non-significant findings were very revealing in the naturalistic context—showing how the practice of bedtime reading was not any inferior when reading together from a tablet compared with a paper-based book.

More generally, the study shows how it is possible to conduct a theory-driven experiment in the wild, based on a growing digitalization concern in society, without compromising the experimental design or the control in order to compare conditions. It shows the value of taking into account a wider set of concerns and using a broader set of measures than is usually done in lab experiments. Namely, the in situ study provides more ecological validity while demonstrating a wider appreciation of the factors that can influence children’s experience of naturalistic shared reading in everyday settings.

Thought Box: A Challenge for HCI Research in the Wild