Читать книгу Evaluation in Today’s World - Veronica G. Thomas - Страница 95

Read the following case example, keeping in mind the AEA’s (2018b) Evaluators’ Ethical Guiding Principles. Then, organize into small groups and discuss the case. Complete the worksheet and question at the end of the case.

Evaluation Context. The Health Care Collaborative program grew out of a multiyear effort funded in many sites by a national foundation. That initiative promoted local collaboration among health care providers and residents in poorly served or underserved neighborhoods. The Health Care Collaborative office uses trained residents as outreach health workers to raise health-issues awareness among residents and to give them options for accessing health care. Health care providers who are collaboration partners deliver a range of services to neighborhood residents. A local funding source supports the Health Care Collaborative, which has a program director, administrative staff, and a small network of outreach workers. The Health Care Collaborative Board of Directors consists of a small group of health care providers.

The Health Care Collaborative serves an economically challenged neighborhood in a small metropolitan area: Average income is one-third to one-half of its metro and national counterparts. The neighborhood is quite diverse along many dimensions, including age, household composition, sexual identity, education, religious preference, race, and ethnicity. The neighborhood has a large African American population, an increasing population of refugees from African and Eastern European nations during the past 20 years, and a rapidly growing Hispanic population in recent years.

Entry, Contracting, and Design. The Health Care Collaborative Board and local funders found that they needed more information than the program’s reporting system alone could provide about how program participants viewed the Health Care Collaborative, how the staff viewed the program and the neighborhood, and how the program met or did not meet identified service needs. The funder provided $20,000 for this purpose, and the Board established a one-year schedule for completing an evaluation. The funder and the program director approached a local faculty member, an evaluator who also teaches evaluation, to ask for a proposal. The faculty member has previously served on the Health Care Collaborative Board. Discussions with the funder, the program director, and some members of the Board identified key expectations and constraints.

The faculty evaluator proposed a multimethod approach for a formative evaluation. The design included surveys of participants (brief), program staff, and other health care provider partners. The surveys would include questions about racial and ethnic identity. Selected program participants would be asked to keep journals and to participate either in a focus group or in an observed service delivery for a small group. Three focus groups were proposed: one for senior citizens; another for adult, nonsenior males; and a third for adult, nonsenior females. The Health Care Collaborative focus group participants would be offered a $25 gift card for their time. The institutional review board’s approvals would be obtained for informed consent to voluntarily participate in the evaluation.

A graduate student would do most of the data collection, under the evaluator’s supervision. The student was fluent in Spanish and English, and this project would be the subject of the student’s master’s thesis. The evaluation’s final product would be a presentation of results, in PowerPoint format, with the slides and notes delivered to the program director and funder.

Data Collection. The student administered the staff surveys in person. These surveys asked for how long the staff members worked with the Health Care Collaborative, what they did in the program, how they viewed the participants, and what difference the program made in the neighborhood. Surveys of other providers involved with the Health Care Collaborative were web-based. The questions concerned what kinds of interaction the providers had at the Health Care Collaborative, with whom, and how often; how that relationship affected both organizations; and what services the responder brought to resident-participants in the Health Care Collaborative.

The Health Care Collaborative staff administered surveys to program participants during ongoing program contact. The student also conducted a small number of interviews of people identified for their longevity in working with this particular neighborhood, and added open-ended historical questions.

The student observed both staff and participants in health care awareness sessions for small groups to better enrich the evaluator’s and student’s understanding of the program, its staff, and the participants. Participants’ journals provided inspirational stories of their experiences in navigating the health care maze.

Data Analysis and Interpretation. From the surveys, some data were aggregated and reported descriptively (e.g., comparisons of the racial and ethnic composition of the Health Care Collaborative participants for the neighborhood). Scaling and cluster analyses were used to structure and analyze the results of the focus groups, and some journal entries and responses to open-ended questions from interviews also were analyzed.

All in all, the program served a disproportionate number of Hispanic adults (compared to the neighborhood’s composition) and disproportionate numbers of people without health insurance and without other known ways to access health care. Participants and staff were very positive about the program and its value in their neighborhood and lives. The Health Care Collaborative program participants overwhelmingly credited the use of racially and ethnically diverse staff, from the neighborhood itself, as the main reason for the Health Care Collaborative’s success.

Younger adults placed more concern on financial issues related to health care, compared with older adults. Hispanic participants in focus groups were all female, and most were unemployed. From all three focus groups, whether participants were treated fairly and had access to insurance and to health care was more important than waiting times or actually getting to appointments.

When the evaluator and student felt comfortable with their work, they shared draft findings informally with the program director, funder, and Board members—through in-person as well as telephone conversations and through email. Some feedback was given and considered in reviewing those findings and in developing the final product.

Dissemination and Utilization of Results. The final evaluation briefing was delivered at a meeting of the Health Care Collaborative Board, to which the funder and some residents were invited. The funder could not make this meeting, accepted the electronic PowerPoint file, and asked no further questions. Only one resident—a regular attendee of Board meetings—was present for the briefing. Two or three questions were asked, more of apparent curiosity than any other cause or purpose. No future plans for the findings were discussed at this meeting.

The student completed the thesis based on this project, and it was very well received by the faculty committee. The evaluator adapted the evaluation for use in an advanced evaluation course for graduate students.

The student and evaluator also proposed a poster session focusing on the evaluation findings to an annual, national professional conference in their discipline. The proposal was accepted and a large poster developed, which covered the basics of the evaluation. Those who stopped to read and talk about the evaluation expressed admiration for its scope and methods.

As the evaluator, what are some things that you would do differently to better ensure that your actions are ethically defensible?

Source: This case is republished with permission of the American Evaluation Association (with minor edits).