Читать книгу Evaluation in Today’s World - Veronica G. Thomas - Страница 96

In addition to the Evaluators’ Ethical Guiding Principles, the Program Evaluation Standards is another document that provides guidance and direction for those in the evaluation field. It includes much more specificity regarding what to do and not do in program evaluation than the Evaluators’ Ethical Guiding Principles. Whereas the Evaluators’ Ethical Guiding Principles are concerned specifically with the ethical conduct of the evaluator, the Program Evaluation Standards pertain to the quality of the evaluation. Initially established in 1981 by the Joint Committee on Standards for Educational Evaluation¹ with multiple editions since then, the Program Evaluation Standards provide guidance for improving evaluation quality and accountability. The Program Evaluation Standards contain 30 standards organized around five central attributes of evaluation quality. These quality attributes include (a) utility (N = 8 standards), (b) feasibility (N = 4 standards), (c) propriety (N = 7 standards), (d) accuracy (N = 8 standards), and (e) evaluation accountability (N = 3 standards). A full description of the 30 Program Evaluation Standards is provided in Appendix B. An overview of the five central attributes discussed in the Program Evaluation Standards, as adapted from Yarbrough et al. (2011), include the following:

¹ The Joint Committee on Standards for Educational Evaluation (JCSEE) is supported by 17 sponsoring organizations and has been a member of the American National Standards Institute (ANSI) since 1989. During its history, the mission of the JCSEE has remained to develop and implement inclusive processes producing widely used evaluation standards that serve educational and social improvement. To learn more about the history and organizational support of the JCSEE, visit www.jcsee.org.

 Utility standards are concerned with evaluation use, usefulness, influence, and misuse. Utility is supported by standards that provide guidance to increase the likelihood that the evaluation will have positive consequences and substantial influences such as contributing to stakeholders’ learning, informing decisions, leading to improvements, or providing information for accountability judgments.

 Feasibility standards are intended to increase evaluation effectiveness and efficiency by ensuring that an evaluation is practical, efficient, and contextually viable. These standards highlight the logistical and administrative requirements of evaluations that must be managed, bring the world of possible evaluation procedures into the world of practical procedures for a specific evaluation, and serve as a precondition for other attributes of quality.

 Propriety standards support what is proper, fair, legal, right, and just in evaluations. These standards cover three overlapping domains: (a) the evaluators’ and participants’ ethical rights, responsibilities, and duties; (b) systems of laws, regulations, and rules that regulate the conduct of people and organizations, such as federal, state, local, and tribal regulations and requirements, institutional review boards, and local/tribal constituencies that authorize consent to work in and with respective communities; and (c) the roles and duties inherent in evaluation professional practice.

 Accuracy standards seek to increase quality in data collection and analyses and to increase the truthfulness and dependability of evaluation representations, propositions, and findings by urging that evaluations strive for as much accuracy (i.e., validity, reliability, reduction in error and bias) as is feasible, proper, and useful to support sound conclusions and decisions in specific situations. Ignoring nondominant cultural perspectives and assuming that certain methodologies (e.g., experimental designs) are the only factor necessary for justified conclusions and decisions is a barrier to adherence to the accuracy standards.

 Evaluation accountability standards encourage adequate documentation of evaluations and a metaevaluation (evaluation of the evaluation) focuses on improvement and accountability for evaluation processes and products. Attention to accountability guides improvement during all phases of the evaluation, and it encourages reflection and a metaevaluative perspective in evaluators and evaluation users.

In his Voices From the Field interview, Michael Morris stresses that evaluators must uphold the Evaluators’ Ethical Guiding Principles and the Program Evaluation Standards and resist pressure to act unethically. This sometimes takes, as he points out, consideration of potential ethical challenges during evaluation planning, moral courage, and just the willingness to do the right thing.