Читать книгу Evaluation in Today’s World - Veronica G. Thomas - Страница 65

The Health Services Center (HSC) at North Central Southeastern State University, a small public institution (popularly known as Where Are U?), has recently engaged you, an external consultant, to conduct an evaluation of its programs and services. Last year, the state legislature mandated that the various administrative units within public colleges and universities be systematically evaluated on a periodic basis, and actually went so far as to allocate funds for this purpose.

The HSC has been included in the “first wave” of offices to be reviewed at Where Are U?, and the HSC director is less than thrilled. He has occupied the director’s position for the past 10 years, and he is convinced that (1) several highly placed administrators at the university would like to replace him and (2) these administrators view the evaluation as an opportunity to build a case to support such a move. This afternoon you are meeting with the director to discuss a draft version of a survey that you have prepared, focusing on students’ experiences with and opinions of the HSC. One section of the survey contains an extensive list of items that describe both positive and negative encounters that the diverse student body might have had with the HSC, and asks respondents to indicate which ones they have personally experienced.

In previous evaluations, you have found the survey to be an extremely useful, efficient, and comprehensive method for examining stakeholders’ perceptions. The survey will also give students an opportunity to rate how important these various issues and experiences are to them in terms of influencing their overall opinion of the HSC.

It is noteworthy to mention that the survey represents just one of the data-gathering strategies that you intend to use in this project. The director is, to put it mildly, unhappy with certain aspects of the survey. He believes that the items focusing on negative experiences will turn the HSC into a “punching bag” (the director’s words) for disgruntled students with an “axe to grind,” and that it would only be fair to use such a list if comparable ones were employed in the evaluation of other departments on campus. You, of course, are not involved in the evaluation of these other departments.

The director also maintains that many of the negative survey items pertain to matters that are not fully under the control of the HSC (e.g., the hours when certain medical specialists are available at the HSC). As the discussion continues, you become increasingly convinced that the key issue here is not the methodology you have proposed for gathering data. Rather, it seems to be the director’s intense desire to declare certain domains of the HSC’s functioning as “off-limits” in this evaluation. Although you can understand why he might be motivated to take such a position, you also strongly believe that an adequate, professionally respectable evaluation of the HSC cannot omit consideration of the areas that the director wants you to avoid. Interestingly, during the entry/contracting stage of this project, the director had not voiced the “domain concerns” that appear to worry him so greatly now. Your conversation with the director is cut short when he is called out of the office due to an emergency.

 What do you see as the major ethical issues that the evaluator is facing in this case?

 What hidden agendas could become ethically problematic for the evaluator?

 What might be some professional consequences for this evaluator if the ethical issues remain unresolved?

Source: American Journal of Evaluation (AJE) Morris, M. (2000). The Case of the Sensitive Survey. American Journal of Evaluation, 21(2), 263 -263. https://doi.org/10.1177/109821400002100216

A number of evaluation-related circumstances could pose ethical dilemmas for the evaluator. Some examples include (a) being contracted to conduct an evaluation on a program not yet ready to be evaluated; (b) being given insufficient time to complete the evaluation, (c) treatment of people associated with the program around issues such as confidentiality, informed consent, and assignment to program groups; and (d) role conflicts facing evaluators (Posavac & Carey, 2016). Ethical issues can also be the result of inappropriate behaviors such as exerting inappropriate influence on program participants, usurping the role of others (e.g., service providers), or having conflicts of interests (e.g., promoting personal interests over those of the client).

Evaluation ethics concerns the responsibility of evaluators to be competent, honest, and respectful to all individuals and groups that are affected by the evaluation. This includes not only evaluators’ responsibilities to participants, program staff, and other beneficiaries, whom they must treat with respect, but also their responsibilities to evaluation sponsors to yield a quality and credible evaluation. Because of the different roles (e.g., consultant/administrator, data collector/researcher, reporter, member of profession, or member of society) evaluators fulfill at any given time while in the field, an evaluator may face a variety of circumstances in which ethical dilemmas arise (Newman & Brown, 1996). As a result, evaluators must strike the right balance between what is in the best interest of their client and society (Wolf et al., 2009). Ethics in evaluation does not exist in a vacuum. No set of guidelines or standards can cover every possible situation that evaluators will encounter in the field; thus, evaluators’ ethical decision making and behavior must be nuanced to the context with consideration of things such as the values held by the stakeholders, cultural issues, and potential conflicts of interest within the particular setting.

How evaluators make decisions and exercise their ethical and professional judgment in practice will determine the evaluation ethics in context. While professional ethics are designed to protect against the reoccurrence of major atrocities such as the Tuskegee experiments, they also are needed to guard against less obvious, yet still potentially harmful, effects of evaluation when there might be issues such as inaccurate representation of stakeholders’ perceptions and viewpoints, invasion of privacy, deception, and insensitivity to those being studied, especially when they represent minority and other vulnerable populations. Evaluators, like other individuals, are likely to overestimate how ethical they are, which, in turn, can be a barrier against behaving ethically, especially in ambiguous situations. Ethical behavior is not just making the easy ethical choices between right and wrong, but also working through the more complex decisions that involve hierarchies of values, prioritized according to circumstances, and deciding what is, and is not, valid and credible information (Mabry, 2004).

Ethics in evaluation can be considered at both the individual and professional levels—both affecting evaluation quality. At the individual level, ethics is concerned with the behavior of an individual evaluator. The evaluator is expected to act with integrity throughout the entire evaluation process from conceptualization and design through dissemination and use of findings. Ethical issues can exist in the way the evaluator presents (or misrepresents) him- or herself during planning and recruitment, data collection, and dissemination activities; ethical issues can arise due to how an individual evaluator interacts and shares responsibility with stakeholders.

Evaluators are presented with an array of messy, complicated issues in the field requiring them to think quickly and ethically in a given situation. They may face what Guillemin and Gillam (2004) refer to as a number of “ethically important moments.” These are the difficult, often subtle, and usually unpredictable situations that arise in the practice of doing all kinds of research, including evaluations. During the course of the evaluation, important moments arise without warning, and the evaluator must be prepared to respond in an ethically defensible manner at the time the situation occurs. Sometimes, ethically important moments arise and must be addressed immediately. In such cases, the evaluator must decide what action to take in real time. For example, in a situation where a project administrator offers a new evaluator a gift, the evaluator has to make an immediate decision whether to accept or refuse the gift. On one hand, and in some cultures, a gift conveys a great deal of respect and is a sign that the relationship is valued by the giver. If there is a problem, it may rest with the evaluator who may not trust the program director’s motives. On the other hand, when gift giving is intended to favorably influence the evaluator’s judgment, acceptance of the gift is ethically problematic. In this case, the evaluator must immediately decide, in real time, his or her course of action—accept or decline the project director’s gift.

Ethics in evaluation at the individual level is not only restricted to the behavior of the evaluator. It is also related to the rights, responsibilities, and behaviors of various evaluation stakeholders. For example, program staff, clients, sponsors, and other relevant stakeholders (e.g., users, community members) have an ethical responsibility for acknowledging unique entry and access issues, as well as particular community habits, customs, and values that determine suitable conduct in the evaluation process (Yarbrough, Shulha, Hopson, & Caruthers, 2011).

Over the past three decades, the evaluation profession has undergone major developments that have served to highlight the importance of ethics in the discipline. Probably most notable among these include the development and multiple subsequent revisions of both the AEA’s Evaluators’ Ethical Guiding Principles and the Joint Committee on Standards for Educational Evaluation’s Program Evaluation Standards (discussed in the next section). Additionally, there have been published books on program evaluation ethics (e.g., Morris, 2008; Newman & Brown, 1996), research on evaluation ethics, and numerous journal articles and book chapters on the topic. Through publication of the Ethical Challenges section of the American Journal of Evaluation, the AEA has sought to promote evaluators’ understanding of ethical challenges, enhance evaluators’ sensitivity to the ethical dimensions of their work, and translate the Evaluators’ Ethical Guiding Principles and the Program Evaluation Standards into everyday practice.