Читать книгу Handbook of Web Surveys - Jelke Bethlehem - Страница 52

The mobile web survey or the web‐only survey process differs in some respects from a survey based on traditional modes (paper and pencil, telephone, and fax). Errors arising in mobile web surveys and only mobile surveys have some specific characteristics due mainly to the coverage aspects of the target population and to various other possibilities (the availability of auxiliary variables, the time required to deliver the questionnaire and to receive the completed questionnaire, technical skills, the equipment needed, and many other situations). Therefore, a well‐defined outline of the survey research steps and of the errors that might occur at each step is vital for the surveyor. Furthermore, in many cases, the errors at different steps have some relationships, even if the relationship is not clearly defined and formalized. Thus, decisions pertaining to the reduction of an error at a specific step could increase the errors occurring at other steps of the survey process. For instance, increasing response rate by stressing the interviewee with a high number of solicitations might decrease the quality of the data (more item nonresponse, less accuracy, and so forth). Therefore, the overall quality of the survey process could be either improved or deteriorated depending on the decision made.

The flowchart mentioned in the previous section is in Figure 3.1. It shows the main steps and sub‐steps for a web‐only or mobile web survey based on probability sample.

Figure 3.1 Flowchart of a probability‐based web‐only or mobile web survey

The six main steps (gray highlighted) and their related sub‐steps (hell gray and white highlighted) are listed in the previous section. Hereunder, detailed comments of the flowchart steps are reported.

The first step is Determine the survey objectives, and it is a preliminary point of every survey, independent from the mode. Objectives need to be clearly stated and the questions to be identified. Since this flowchart considers web mode (either web only or mobile web), it should be critically evaluated if the objectives and the structure of the survey are adequate for that mode. Web mode implies a self‐completed questionnaire; thus, for example, self‐completed surveys are better in capturing sensitive topics. If the questionnaire is long self‐completion, especially using web mode, it is not in favor of high participation rates and smaller measurement error. This becomes especially critical in mobile web surveys, where the participation situation has a high probability to be disturbed from other activities or tasks the interviewee is doing.

The second step is Metadata definition. Recent literature and empirical research brought significant attention to metadata, showing a set of information describing the survey (that is, elements for evaluating the data quality, like the list and description of the variables, the target population, the sample units, and so forth). The metadata refer to variables and activities occurring in every step of the process. The International Organization for Standardization (ISO) defines metadata as “data describing and defining other data in a certain context.” In such a case, the contest is the condition under which the data collection and processing takes place. The definition of metadata is an important step, recommended for all surveys, independently from the mode. Details to consider in mobile web surveys (and in web‐only surveys) differ from traditional surveys due to the characteristics of various steps and sub‐steps. Metadata are mainly semantic based; a bad or imprecise metadata description affects the quality of the questionnaire and could cause errors in the answers (questions not correctly understood, questions that each interviewed could interpret in a different way). Metadata are useful for the selection of appropriate sampling techniques and for guiding and evaluating the procedures of the survey process. The so‐called metadata database contains metadata, which complement the data database. See Example 3.1. The data user should look at the metadata to be conscious of the quality and meaning of the data he or she is going to use. On the other hand, a surveyor should take care to write down the metadata precisely because accuracy is crucial for undertaking the survey process in a correct and valuable way. For example, if a surveyor is drawing his probability‐based sample from a largely incomplete sampling list, it will not be possible to obtain highly accurate results from the inference process. Thus, information about the sampling frames characteristics (coverage, reference time of the frame, i.e. is it updated or an old one?) is an important quality. Another example is a bad or insufficient description of one variable; in this case, each interviewed could assign to the variable a different meaning, sorting out in scarcely meaningful results. From the user point of view, this implies both the use of low‐quality data and the impossibility to understand what is the effective meaning of the variable and how to interpret the data.