Читать книгу Researching in the Age of COVID-19 Vol 1 - Группа авторов - Страница 12

A total of 26,182 phone calls were made to 8,356 individuals and 3,536 surveys were completed across the six studies. Table 1.2 presents the scale of survey operations across the various studies. The six studies had a combined response rate² of 42.32 per cent, while 30.33 per cent of the respondents were not reachable and 11.50 per cent refused to participate. The remaining respondents either did not complete the survey or their number was incorrect.

Table 1.2 Summary of calls status

Final status	Study
CDR	CERR	CERRBC	CHG	GBV	HBB	Total
Completed	501	2,022	652	155	29	177	3,536
Percentage	27.08	39.78	87.52	62.50	22.31	59.00	42.32
Incomplete	81	99	0	0	2	19	201
Percentage	4.38	1.95	0.00	0.00	1.54	6.33	2.41
No Response	666	1,592	61	69	65	81	2,534
Percentage	36.00	31.32	8.19	27.82	50.00	27.00	30.33
Refused	256	653	31	1	8	12	961
Percentage	13.84	12.85	4.16	0.40	6.15	4.00	11.50
Wrong number	346	717	1	23	26	11	1,124
Percentage	18.70	14.11	0.13	9.27	20.00	3.67	13.45
Total	1,850	5,083	745	248	130	300	8,356
Percentage	100.00	100.00	100.00	100.00	100.00	100.00	100.00

Note: First row shows frequencies and second row shows column percentages.

A key consideration in developing CATI survey protocols is the number of call attempts that need to be made before categorizing a respondent as ‘not reachable’. For the purposes of this study, a total of nine call attempts were made for each potential respondent. This is based on a study (Özler and Cuevas, 2019), conducted in Turkey in which the response rate at the seventh call was close to 70 per cent, with marginal gains for each additional call.

Furthermore, a recent webinar organized by the Abdul Latif Jameel Poverty Action Lab (J-​PAL) (Suri, 2020) recommended that six to nine calls be made to each respondent before categorizing them as ‘No response’. However, the research finds that it is both cost-​ and time-​efficient to categorize respondents as ‘No response’ after the third call and survey new respondents from the population pool, instead of complying with the nine calls protocol.

Figure 1.1 presents the survey status and response rates for five of the six studies that adopted a comparable strategy in this regard. One study (CDR) capped the maximum number of attempts at four before categorizing them as ‘Not answered’, compared to the five other studies, which categorized a respondent as ‘Not answered’ after nine unsuccessful attempts (this excludes those categorized as a ‘Wrong number’, as only one call was necessary to determine their status).

Panel A represents the distribution of the final survey status for each study and Panel B illustrates the number of call attempts required to complete surveys. Panel B shows that 77 per cent of successful surveys were completed by the third attempt, after which the gain from an additional attempt to complete the survey was marginal. Across these studies, 22,728 calls were made to complete 3,035 surveys. Among all these calls, only 7,944 were made to respondents who completed the survey; 47.3 per cent of the calls were made to 1,868 respondents who were not reachable (that is, out of service coverage, did not answer or had an invalid number).

Figure 1.1 Survey status and response rate

Table 1.3 presents the response rates and number of calls required to complete surveys before and after the third call. A total of 18,706 calls were made to respondents (not including those that were ‘Wrong number’ and ‘Refused’), but only 21.1 per cent of calls were needed to complete 77 per cent of the 3,035 completed surveys by the third attempt. More than 57 per cent of the calls were made to respondents who were not reachable (that is, never picked up their phone or whose number was switched off or outside the coverage area).

Table 1.3 Completion rate and field operations time investment

	Completed
	< = 3 Calls	>3 Calls	Total	Non-​response
Number of surveys	2326	709	3035	1868
Percentage of completed surveys	77	23
Total number of calls	3954	3990	7944	10762
Percentage of calls made	21.1	21.3	42.5	57.5

Assuming the response rate remains the same, by making this one change the studies could have achieved the sample size of 3,035 respondents with 14 per cent fewer calls. However, this would have also required increasing the pool of respondents by 30 per cent. The decision of where to cap the calls per respondent will depend on the study’s goals of either ensuring a higher response rate or a faster data collection turnaround. The former is a common objective for follow-​up surveys.