Читать книгу Handbook of Web Surveys - Jelke Bethlehem - Страница 9

The last 10 years have witnessed a significant increase in Internet penetration. What is particular about this growth is that a number of generations are currently experiencing the contemporary and highly technological environment. Social media, constant connectivity, and on‐demand entertainments are innovations that Millennials (aged between 23 and 38 in 2019) adapted to as they grew up. For those born after 1996, the so‐called Generation Z (aged between 7 and 22 in 2019), these innovations are mostly taken for granted, having been part of their lives from the beginning. The iPhone was launched in 2007, when the oldest members of Generation Z were 10. By the time they are in their teens, young Americans access the Internet mainly via mobile devices, Wi‐Fi, and high‐bandwidth cellular services. Pre‐Millennial generations play an important role in the general population, but for them, this environment based on technological communication is a new experience.

The implications of some population subgroups having adapted to the technological environment (Millennials and pre‐Millennials) while others have lived in this “always on” technological environment all their lives are of relevance for survey‐based research, particularly in the case of web surveys. The way that questionnaires are administered undoubtedly has an impact which differs according to population group. Furthermore, the behavior of the respondents while participating depends on their digital experience, their generational characteristics, and their attitude toward technology in their lives. Therefore, surveys—and in particular web and mobile web surveys—have to adopt a number of changes in their methodology to take into account any differences in the cultural backgrounds of potential survey participants and the characteristics of the eventual devices used.

Due to high Internet penetration and the relatively low cost of conducting web surveys compared with other methods, the number of surveys being conducted via the Internet has increased dramatically over recent years. The panorama of survey‐based research has changed drastically over the last few decades.

First there was a change from traditional paper‐and‐pencil interviewing (PAPI) to computer‐assisted interviewing (CAI). Since the 1990s, there has been a gradual replacing of face‐to‐face surveys (CAPI), telephone surveys (CATI), and mail surveys (CASI, CSAQ) with web‐based surveys. With the relatively recent diffusion of smartphones and other mobile devices, it has become possible to run mobile web surveys, i.e., questionnaires sent to interviewees may be submitted and also completed via mobile devices. A web survey is a simple way to access a large group of potential respondents. Questionnaires can be distributed at very low cost. They require no interviewers, and there are no mailing or printing costs involved. Surveys can be launched rapidly, and little time is lost between the moment the questionnaire is ready and the moment that fieldwork begins. Web surveys also offer interesting new possibilities, such as the use of multimedia (images, sound, animation, and video). Panel surveys are also moving toward data collection via the web.

The recent trend toward the use of big data and the integration of data sources will not render the role of web surveys obsolete, although they may in the future have a different role.

At first sight, web surveys appear to have much in common with other types of survey, seeming to be just another way to collect data, with questions asked over the Internet instead of face‐to‐face, by telephone or via e‐mail. There are however a number of factors that may render the results of web surveys unreliable. Some examples are under‐coverage, self‐selection, and measurement errors. These can cause estimates of population characteristics to be biased, thus leading to incorrect conclusions being drawn from the data collected.

Under‐coverage occurs when the target population is wider than the number of people with Internet access. This leads to bias in estimates in the case of relevant differences between those with Internet access and those without.

Self‐selection is when a questionnaire is simply made available via Internet to all, with individuals nominating themselves.

A respondent is therefore anyone who happens to have Internet access, visits the website, and decides to take part in the survey. These participants generally differ significantly from nonparticipants.

General‐population surveys that aim to provide reliable and accurate statistics are traditionally carried out face‐to‐face or by telephone. Interviewers are used to persuade people to take part and to help respondents to provide the right answers. In web surveys, there is no interviewer assistance, a fact that can have serious impact on the quality of the data collected.

The diffusion of smartphones increases the possibility for interviewees to be reached via their mobile device and to have the questionnaire completed via the same device, resulting in the current trend in running mobile web surveys. Consequently, there are new risks for error in the survey due to device characteristics and the behavior of the user.

The researcher should have in mind that when a web survey is run a mobile web survey takes place, if questionnaire is not blocked against mobile devices. Here, for simplicity the term web surveys is used, meaninig the mobile web survey is included. Summing up, web surveys afford several challanges and need reseacher be conscious of the methodological issues for a good survey. At the time beeing, collecting data through web surveys is going to become a common practice both in market research, academic research and official statistics. Knowledge about how to manage a web survey, risks, errors and advantages is important.

This book provides an insight into the possible use of web surveys and mobile web surveys for data collection. Web surveys allow for lower data collection costs. It is also expected that web surveys lead to increased response rates. Is this the case? What about the quality of the data collected? This book examines many theoretical and practical aspects of mobile web surveys and can therefore be considered as a handbook for those involved in practical survey research, including survey researchers working with official statistics (e.g., in national statistical institutes), academics, and commercial market research.

The book's two authors have widespread expertise in survey methodology. They come from two different countries (the Netherlands and Italy) and different research organizations (a national statistical institute and a university). They therefore provide a broad view on the various theoretical and practical aspects of mobile web surveying.

The second edition of the book involves a revision of each chapter with the following criteria:

(1) to maintain the existing text and content as much as possible, (2) to update the existing text and content with comments based on new literature and results, and (3) to add new paragraphs (if necessary) to cover new relevant topics (see the contents and chapter description below). A number of new examples have been provided, some of the existing examples have been updated or substituted, and some applications have been replaced. Updates have also been included to highlight new trends in web surveys and emerging solutions. There are two new chapters on topics concerning mobile web surveys: one presenting a flowchart to illustrate the steps involved in running a survey via web and the other examining adaptive design. It was therefore necessary to renumber the chapters in respect to the first edition.

The first two chapters of the book provide an introduction into web surveys. Chapter 1 provides a historic account of developments in survey research and shows how web surveys have become a tool for data collection. Section examines the Blaise system, which has been around for more than 30 years. New developments have taken place over the last 10 years, but no papers have been written on this subject. The section looks at the history and recent developments regarding Blaise; it was written by Lon Hofman and Mark Pierzchala and is published for the first time here.

Chapter 2 is an overview of basic aspects of web surveys. It describes how and where they can be used. Official statistics departments, research institutions, market research companies, and private forums are all interested in web surveys studying both households/individuals and businesses.

Chapter 3 presents a flowchart illustrating the steps (and sub‐steps) of web surveys, each accompanied by a short description. The flowchart is of potential use to both practitioners as a guideline for how the survey process should be carried out and to researchers in highlighting and explaining the positioning of their studies at the different steps of the survey process. It is also useful in discussing the errors that can occur in different steps. The chapter provides an introduction to the framework and its structure and discusses the relevance of bearing in mind the framework and the survey steps when considering web survey errors. It then goes on to describe the concept of the step and the structure of the flowchart, breaking down the web survey process into six main steps. These are analyzed in detail, and an overview of survey errors is provided.

Chapter 4 examines the aspects of sampling. It is stressed that valid population inference is possible only if some form of probability sampling is used and that a proper sampling frame is required for this. A number of sampling designs and estimation procedures useful for web surveys are discussed.

A researcher conducting a survey may encounter a number of practical problems, and Chapter 5 provides an overview of possible errors, with two types of error examined in further detail. The first concerns errors in measurement. These can be caused by specific issues in questionnaire design, as well as a number of other aspects such as technology, incorrect unit definition, and so on. The second type of error regards nonresponse. This is a phenomenon that can affect all surveys, but the specific aspects of nonresponse in web surveys require particular attention. The chapter provides advice on relationships between errors and information on the various types.

A web survey is just one form of data collection. There are others, such as face‐to‐face, telephone, mail, and mobile surveys. Chapter 6 compares these various methods with online data collection, discussing the advantages and disadvantages of each one.

As web surveys do not involve interviewers, the respondents complete the questionnaire on their own. Furthermore, when questionnaires are sent out, they may very well be received and even completed on a mobile device (such as smartphones, which are very widespread). This means that questionnaire design is of crucial importance. Questionnaires must be adapted in order to be suitable for mobile devices; otherwise they cannot be used for this purpose. Small imperfections in the questionnaire may have serious consequences in terms of data quality. Questionnaire design issues are addressed in Chapter 7.

Chapter 8 examines strategies for data collection with adaptive/responsive survey design. In this case, strategies are not defined in advance, but instead are adapted, if necessary, during fieldwork. These designs may contribute to countering growing problems of nonresponse. This chapter was written by Annamaria Bianchi and Barry Schouten, who applied their particular expertise in this field to the subject examined.

A web survey may not always be the best solution for providing reliable and accurate statistics, with quality being affected by problems of under‐coverage and low response rates. An interesting alternative is to set up a mixed‐mode survey, in which several data collection methods are combined either sequentially or concurrently. This approach is less expensive than a single‐mode interviewer‐assisted survey (conducted either face to face or by telephone) and solves under‐coverage problems, but at the same time it poses other difficulties, known as mode effects, with one of the most significant of these being measurement error. Mixing modes is also of critical importance, as is the fact that in practice, a web survey is always mobile, unless questionnaire access via mobile device is restricted. All these aspects, as well as others concerning mixed‐mode surveys, are discussed in Chapter 9.

Chapter 10 is devoted to the problem of under‐coverage. This remains an important problem in many countries due to poor Internet coverage and the fact that Internet access is often unevenly distributed throughout the population. The chapter demonstrates how this can lead to survey estimates being biased. A number of techniques that may reduce under‐coverage bias are discussed.

Chapter 11 examines self‐selection. The correct and scientifically well‐founded principle is to use probability sampling in order to select survey subjects and therefore allow reliable estimates regarding population characteristics to be calculated. Nowadays, it is easy to set up a web survey. Even those without any survey knowledge or experience can create one through dedicated websites. Many of the resulting web surveys do not apply probability sampling, but instead rely on self‐selection of respondents. This causes serious problems with estimation. Self‐selection and its consequences in terms of survey results are discussed in this chapter, demonstrating that correction techniques are not always effective, and there are many reasons why web‐survey‐based estimates are biased.

Nonresponse, under‐coverage, and self‐selection are typical examples, and adjustment weighting is often applied in surveys in order to reduce such biases. Chapter 12 describes various weighting techniques, such as post‐stratification, generalized regression estimation and raking ratio estimation. The effectiveness of these techniques in reducing bias caused by under‐coverage or self‐selection is examined.

Chapter 13 introduces the concept of response probabilities, describing how they can be estimated through response propensities. If estimated accurately, response probabilities can be used to correct biased estimates. Here, two general approaches are described: response propensity weighting and response propensity stratification. The first attempts to adjust the original selection probabilities, while the second is a form of post‐stratification.

Chapter 14 is devoted to web panels. There are many such panels, particularly in the field of commercial market research. One crucial aspect is how the panel members (households, individuals, companies, and shops) are recruited. This can be carried out via a proper probability sample, or through self‐selection. There are consequences for the validity of the results of the specific surveys conducted with the panel members. The chapter discusses several quality indicators.

The accompanying website, www.web‐survey‐handbook.com, provides the survey data set for the general population survey (GPS), which has been used for many examples and applications in the book. The data set is available in SPSS (SPSS Corporation, Chicago, IL) format.

Silvia Biffignandi

Jelke Bethlehem

The editors acknowledge the contributions of:

Lon Hofman (Manager Blaise, Statistics Netherlands) and Mark Pierzchala (owner of MMP Survey Services, Rockville, USA) who wrote Section 1.3.1.

Annamaria Bianchi (University of Bergamo) and Barry Schouten (Statistics Netherlands) who wrote Chapter 8.