Читать книгу Collaborative Approaches to Evaluation - Группа авторов - Страница 37

Al Hudib, H. (2018). The role of evaluation policy in organizational capacity to do and use evaluation (Unpublished Ph.D. dissertation). University of Ottawa, Ottawa, Canada. https://ruor.uottawa.ca/handle/10393/38117

Alkin, M. C. (1991). Evaluation theory. In M. W. McLaughlin & D. C. Phillips (Eds.), Evaluation and education: At quarter century (pp. 91–112). Chicago, IL: University of Chicago Press.

Alkin, M. C. (2011). Evaluation essentials: From A to Z. New York, NY: Guilford.

Alkin, M. C., & Vo, A. (2018). Evaluation essentials: From A to Z (2nd ed.). New York, NY: Guilford.

Alkin, M. C., Vo, A. T., & Hansen, M. (2013). Using logic models to facilitate comparisons of evaluation theory. Evaluation and Program Planning, 38, 33.

Bryk, A. (1983). Stakeholder-based evaluation. New directions in program evaluation, no. 17. San Francisco, CA: Jossey-Bass.

Carden, F. (2010). Introduction to the forum on evaluation field building in South Asia. American Journal of Evaluation, 31(2), 219–221.

Chambers, R. (1981). Rapid rural appraisal: Rationale and repertoire. Public Administration and Development, 1, 95–106.

Chouinard, J. A. (2013). The case for participatory evaluation in an era of accountability. American Journal of Evaluation, 34(2), 237–253.

Chouinard, J. A., & Cousins, J. B. (2007). Culturally competent evaluation for Aboriginal communities: A review of the empirical literature. Journal of Multidisciplinary Evaluation, 4(8), 40–57.

Chouinard, J., & Cousins, J. B. (2015). The journey from rhetoric to reality: Participatory evaluation in a development context. Educational Assessment, Evaluation and Accountability, 27, 5–39.

Cousins, J. B. (2001). Do evaluator and program practitioner perspectives converge in collaborative evaluation? Canadian Journal of Program Evaluation, 16(2), 113–133.

Cousins, J. B. (2005). Will the real empowerment evaluation please stand up? A critical friend perspective. In D. M. Fetterman & A. Wandersman (Eds.), Empowerment evaluation principles in practice (pp. 183–208). New York, NY: Guilford.

Cousins, J. B. (2007). Process use in theory, research and practice. New Directions for Evaluation, No. 116. San Francisco, CA: Jossey-Bass.

Cousins, J. B. (2013). When does a conceptual framework become a theory? Reflections from an accidental theorist. Evaluation and Program Planning, 38, 67–70.

Cousins, J. B., & Chouinard, J. A. (2012). Participatory evaluation up close: A review and integration of the research base. Charlotte, NC: Information Age Press.

Cousins, J. B., & Chouinard, J. A. (2018, Oct.). Ethical justification for collaborative approaches to evaluation. Paper presented at the meeting of the European Evaluation Society, Thessaloniki, Greece.

Cousins, J. B., Donohue, J. J., & Bloom, G. (1996). Collaborative evaluation in North America: Evaluators’ self-reported opinions, practices, and consequences. Evaluation Practice, 17, 207–225.

Cousins, J. B., & Earl, L. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14(3), 397–418.

Cousins, J. B., & Earl, L. (1995). Participatory evaluation in education: Studies in evaluation use and organizational learning. London, UK: Falmer.

Cousins, J. B., Hay, K., & Chouinard, J. A. (2015). The third perspective: Uniting accountability and learning within an evaluation framework that takes a moral-political stance. In C. Christie & A. Vo (Eds.), Evaluation use revisited. Charlotte, NC: Information Age.

Cousins, J. B., & Whitmore, E. (1998). Framing participatory evaluation. In E. Whitmore (Ed.), Understanding and practicing participatory evaluation. New Directions in Evaluation, No. 80 (pp. 3–23). San Francisco: Jossey Bass.

Cousins, J. B., Whitmore, E., & Shulha, L. M. (2013). Arguments for a common set of principles for collaborative inquiry in evaluation. American Journal of Evaluation, 34(1), 7–22.

Cousins, J. B., Whitmore, E., & Shulha, L. M. (2014). Let there be light: Response to Fetterman et al., 2014. American Journal of Evaluation, 35(1), 149–153.

Dahler-Larsen, P. (2009). Learning-oriented educational evaluation in contemporary society. In K. E. Ryan & J. B. Cousins (Eds.), Sage international handbook for educational evaluation (pp. 307–322). Thousand Oaks, CA: Sage.

Daigneault, P.-M., & Jacob, S. (2009). Toward accurate measurement of participation rethinking the conceptualization and operationalization of participatory evaluation. American Journal of Evaluation, 30(3), 330–348.

Davies, R., & Dart, J. (2005). The ‘most significant change’ (MSC) Technique: A guide to its use. Cambridge, UK: Authors. Retrieved from http://www.mande.co.uk/docs/MSCGuide.pdf

Fals-Borda, O., & Anisur-Rahman, M. (1991). Action and knowledge: Breaking the monopoly with participatory action research. New York, NY: Apex Press.

Fetterman, D. (1994). Empowerment evaluation. Evaluation Practice, 15, 1–15.

Fetterman, D., Rodriguez-Campos, L., Zukoski, A. P., & Contributors (Eds.). (2018). Collaborative, participatory, and empowerment evaluation: Stakeholder involvement approaches. New York, NY: Guilford.

Fetterman, D., & Wandersman, A. (Eds.). (2005). Empowerment evaluation principles in practice. New York, NY: Guilford.

Freire, P. (1970). Pedagogy of the oppressed. New York, NY: Seabury Press.

Gilbert, N. (2018). Understanding the process of patient engagement in planning and evaluation of health services: A case study of the Psychosocial Oncology Program at the Ottawa Hospital (Unpublished Ph.D. dissertation). University of Ottawa, Ottawa, Canada.

Gilbert, N., & Cousins, J. B. (2017). Advancing patient engagement in health service improvement: What can the evaluation community offer? Canadian Journal of Program Evaluation, 32(2), 1–20.

Harnar, M. A. (2012). Theory building through praxis discourse: A theory- and practice-informed model of transformative participatory evaluation (Unpublished doctoral dissertation). Claremont Graduate University, Claremont, CA.

Hay, K. (2010). Evaluation field building in South Asia: Reflections, anecdotes, and questions. American Journal of Evaluation, 31(2), 222–231.

Hoare, T., Levy, C., & Robinson, M. P. (1993). Participatory action research in Native communities: Cultural opportunities and legal implications. Canadian Journal of Native Studies, 13(1), 43–68.

Hood, S., Hopson, R., & Kirkhart, K. (2015). Culturally responsive evaluation: Theory, practice and future implications. In K. Newcomer, H. P. Hatry, & J. S. Wholey (Eds.), Handbook of Practical Program Evaluation (4th ed.). (pp. 281–317). Hoboken, NJ: Wiley.

King, J. A. (2005). Participatory evaluation. In S. Mathison (Ed.), Encyclopaedia of evaluation (pp. 291–294). Thousand Oaks, CA: Sage.

LaFrance, J., & Nichols, R. (2008). Reframing evaluation: Defining an indigenous evaluation framework. Canadian Journal of Program Evaluation, 23(2), 13–31.

Mark, M. M. (2009). Evaluation, method choices and pathways to consequences: Trying to make sense of how evaluation can contribute to sensemaking. In K. E. Ryan & J. B. Cousins (Eds.), Sage international handbook on educational evaluation (pp. 55–75). Thousand Oaks, CA: Sage.

Mark, M. M., & Shotland, R. L. (1985). Stakeholder-based evaluation and value judgments. Evaluation Review, 9, 605–626.

Mayne, J. (2001). Addressing attribution through contribution analysis: Using performance measures sensibly. Canadian Journal of Program Evaluation, 16, 1–21.

Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270–280.

Mertens, D. (2009). Transformative research and evaluation. New York, NY: Guilford.

Miller, R. L., & Campbell, R. (2006). Taking stock of empowerment evaluation. American Journal of Evaluation, 27(3), 296–319.

Milley, P., Szijarto, B., Svensson, K., & Cousins, J. B. (2018). The Evaluation of social innovation: A review and integration of the current empirical knowledge base. Evaluation, 24(2), 237–258.

O’Sullivan, R. G. (2004). Practicing evaluation: A collaborative approach. Thousand Oaks, CA: Sage.

O’Sullivan, R. G. (2012). Collaborative Evaluation within a framework of stakeholder-oriented evaluation approaches. Evaluation and Program Planning, 35, 518–522.

Patton, M. Q. (1978). Utilization-focused evaluation. Beverly Hills, CA: Sage.

Patton, M. Q. (1994). Developmental evaluation. Evaluation Practice, 15(3), 311–320.

Patton, M. Q. (1997). Utilization-focused evaluation: A new century text (3rd ed.). Thousand Oaks, CA: Sage.

Patton, M. Q. (2005). Toward distinguishing empowerment evaluation and placing it in a larger context: Take two. American Journal of Evaluation, 26, 408–414.

Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage.

Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York, NY: Guilford.

Patton, M. Q. (2017). Principles-focused evaluation: The GUIDE. New York, NY: Guilford.

Preskill, H. (2008). Evaluation’s second act: Spotlight on Learning. American Journal of Evaluation, 29(2), 127–138.

Preskill, H., & Torres, R. T. (2000). A learning dimension of evaluation use. In V. Carcelli & H. Preskill (Eds.), The expanding scope of evaluation use. New directions in evaluation, no 88 (pp. 25–37). San Francisco, CA: Jossey-Bass.

Rodriguez-Campos, L. (2005). Collaborative evaluation: A step by step model for the evaluator. Tamerac, FL: Llumina Press.

Rodriguez-Campos, L. (Ed.). (2015). Collaborative evaluation in practice: Insights from business, nonprofit, and education. Scottsdale, AZ: Information Age.

Rugh, J., Steinke, M., Cousins, J. B., & Bamberger, M. (2009). Summary of discussion of the ‘Alternative to the Statistical Counterfactual Think Tank,’ presented at the American Evaluation Association Conference, San Antonio, TX, 2010.

Ryan, K. E., & DeStefano, L. (Eds.). (2000). Evaluation as a democratic process: Promoting inclusion, dialogue, and deliberation, no. 85 (pp. 27–38). San Francisco, CA: Jossey-Bass.

Serrat, O. (2009, January). The most significant change technique. Knowledge Solutions, 25, 1–4. Retrieved from http://www.adb.org/knowledgesolutions

Shulha, L. M. (2000). Evaluative inquiry in university-school professional learning partnerships. In H. Preskill & V. Caracelli (Eds.), The learning dimension of evaluation use, new directions for evaluation (Vol. 88, pp. 39–53). San Francisco, CA: Jossey-Bass.

Shulha, L. M., & Cousins, J. B. (1997). Evaluation use: Theory, research and practice since 1986. Evaluation Practice, 18, 195–208.

Shulha, L. M., Whitmore, E., Cousins, J. B., Gilbert, N., & AlHudib, H. (2016). Introducing evidence-based principles to guide collaborative approaches to evaluation: Results of an empirical process. American Journal of Evaluation, 37(2), 193–215.

Shulha, L. M., & Wilson, R. J. (1995). Inviting collaboration: Insights into researcher-school community partnership. In J. B. Cousins & L. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning. Bristol, England: Falmer Press.

Snowden, D. J., & Boone, M. E. (2007). A leader’s framework for decision making. Harvard Business Review, 85, 69–76.

Vo, A., & Christie, C. (2015). Advancing research on evaluation through the study of context. In P. R. Brandon (Ed.), Research on evaluation: New directions for evaluation (Vol. 148, pp. 43–55). San Francisco, CA: Jossey-Bass.

Wadsworth, Y. (1998). What is participatory action research? Action research international paper 2. Retrieved from http://www.scu.edu.au/schools/gcm/ar/ari/p-ywadsworth98.html

Weaver, L., & Cousins, J. B. (2004). Unpacking the participatory process. Journal of Multidisciplinary Evaluation, 1, 19–40.

Wehipeihana, N., McKegg, K., Thompson, V., & Pipi, K. (2016). Cultural responsiveness through developmental evaluation. In M. Q. Patton, K. McKegg, & N. Wehipeihana (Eds.), Development evaluation exemplars: Principles in practice (pp. 25–44). New York, NY: Guilford.

Whitmore, E. (1988). Participatory approaches to evaluation: Side effects and empowerment (Unpublished Ph.D. dissertation). Cornell University, Ithaca, NY.

Whitmore, E. (1991). Evaluation and empowerment: It’s the process that counts. Empowerment and Family Support Networking Bulletin, 2(2), 1–7.

Whitmore, E. (1998a). Understanding and practicing participatory evaluation. New directions in evaluation, no. 80. San Francisco, CA: Jossey-Bass.

Whitmore, E. (1998b). “We need to rebuild this house.” The role of empowerment evaluation of a Mexican farmers’ cooperative. In E. T. Jackson & Y. Kassam (Eds.), Knowledge shared: Participatory evaluation in development cooperation (pp. 217–230). West Hartford, CT: Kumarian Press.

Whitmore, E. (2001). Emancipatory qualitative research. In I. Shaw & N. Gould (Eds.), Qualitative research in social work (pp. 83–99). London, UK: Sage.

Whitmore, E. (2014). Researcher/evaluator roles and social justice. In S. Brisolara, D. Seigart, & S. SenGupta (Eds.), Feminist evaluation and research: Theory and practice (pp. 59–94). New York, NY: Guilford.

Whitmore, E., & Wilson, M. G. (2005). Popular resistance to global corporate rule: The role of social with (with a little help from Gramsci and Freire). In I. Ferguson, M. Lavalette, & E. Whitmore (Eds.), Globalisation, global justice and social work (pp. 189–206). London, UK: Routledge.

Whitmore, E., & McKee, C. (2001). Six street youth who could… In P. Reason & H. Bradbury (Eds.), Handbook of action research: Participative inquiry and practice (pp. 396–402). London, UK: Sage.

About the Authors

J. Bradley Cousins, Ph.D. is Professor Emeritus and Senior Researcher, Centre for Research on Educational and Community Services, University of Ottawa. His interests include collaborative approaches to evaluation, evaluation use, and evaluation capacity building. Contact: bcousins@uottawa.ca

Lyn M. Shulha, Ph.D. is Professor Emerita, Faculty of Education, Queen’s University. Her interests include evaluation use, collaborative approaches to evaluation, and student assessment.

Elizabeth Whitmore, Ph.D. is Professor Emerita, School of Social Work, Carleton University. Her interests include collaborative approaches to evaluation, gender issues in evaluation, and social justice.

Hind Al Hudib, Ph.D. is an independent consultant in Ottawa and recent Ph.D. graduate, Faculty of Education, University of Ottawa. Her interests include evaluation policy and organizational capacity for evaluation and collaborative approaches to evaluation.

Nathalie Gilbert, Ph.D. is a Director at the Champlain Local Health Integration Network and Ph.D. graduate, Faculty of Education, University of Ottawa. Her interests include collaborative approaches to evaluation, evaluation in health systems, and patient engagement.

Appendix 1: COVE Promotional Activities for the CAE Principles

2014, October: Single paper session, American Evaluation Association, Denver, USA

2015, January: Half-day workshop, Swiss Evaluation Society, Geneva, Switzerland

2015, February: Half-day workshop, Autonomous University of Barcelona, Barcelona, Spain

2015, March: Half-day workshop, Universidad Complutense de Madrid, Madrid, Spain

2015, May: Expert lecture, Canadian Evaluation Society, Montréal, Canada

2015, December: Video: CRECS Ten Minute Window, 3(4). www.crecs.uottawa.ca/publications/ten-minute-window/evidence-based-principles

2016, September: Two-hour seminar, Israeli Professional Evaluation Association, Jerusalem, Israel

2016, October: Two-hour seminar, International Congress on Palliative Care, Montréal, Canada

2016, October: Full-day workshop, European Evaluation Society, Maastricht, The Netherlands

2016, November: Full-day workshop, American Evaluation Society, Atlanta, USA

2017, February: Virtual coffee break session, American Evaluation Association, hosted from Clearwater FL, USA

2017, May: Keynote Address, L’avaluacio participative, Conference on Youth Empowerment, Barcelona, Spain

2017, June: Full-day workshop, Canadian Evaluation Society, Vancouver, Canada

2017, October: Two-hour seminar, Fed Gov, Buenos Aires, Argentina

2017, October: Half-day workshop, Universidad National de Jan Juan, San Juan, Argentina

2017, October: Half-day workshop, Universidad Federal de Espirito Santos, Vittoria, Brazil

2017, November: Full-day workshop, American Evaluation Association, Washington, D.C., USA

2017, December: Half-day workshop, Turkish Monitoring and Evaluation Society, Ankara, Turkey

2017, December: Half-day workshop, Jordanian Development Evaluation Association (EvalJordan), Amman, Jordan

Appendix 2: Launch of CAE Principles (E-mail/Listserv Text)

Greetings. On behalf of the COVE research team (Collaborative Opportunities to Value Evaluation: Brad Cousins, Lyn Shula, Elizabeth Whitmore, Hind Al Hudib, Nathalie Gilbert) it is my very great pleasure to introduce Principles to Guide Collaborative Approaches to Evaluation (CAE). We would like to take this opportunity to promote their application and use.

The eight evidence-based Principles were developed on the basis of data from 320 practising evaluators in North America and beyond and were recently published in the American Journal of Evaluation (Shulha at al., 2016, vol. 37, no. 2).

Please find attached for your convenience and use two resource documents:

1 A brochure-style document that describes the Principles and potential applications in i) guiding CAE practice, ii) reflecting on the implementation and consequences of recent CAE projects, iii) evaluation policy review, iv) structuring professional development, and v) framing research on CAE practice (as well as other possible applications).

2 An indicator document that provides items for consideration with respect to the application of each of the eight CAE principles.

The eight Principles are to be: used as a set, although individual principles may be differentially weighted depending on context; considered to be interdependent and overlapping; and recognized as being nonlinear, albeit with some adherence to a loose temporal order.

For further information about the principles please follow this link to a short video presentation: http://crecs.uottawa.ca/publications/ten-minute-window/evidence-based-principles

We sincerely hope that the use and application of the principles will benefit in significant ways your practice in collaborative and participatory approaches to evaluation. At this juncture, despite our systematic efforts to validate the Principles, we recognize the current version to be preliminary and subject to ongoing development and refinement. To that end …

You are invited!

The principles were developed on the basis of thoughtful input by practising evaluators such as you. We invite you to seriously consider an opportunity to contribute to their ongoing development through empirical inquiry. We plan to publish peer-reviewed collections of practical field studies of the principles conducted by English-, French-, and Spanish-speaking evaluators. Please consider contributing. For more information follow this link: [link to field test invitation]

Best wishes in your evaluation pursuits and may all of your evaluations be learning experiences.

Appendix 3: Call for Field Studies of CAE Principles (Intro Text for Online Fillable Form)

The COVE research team (Collaborative Opportunities to Value Evaluation: Brad Cousins, Lyn Shulha, Elizabeth Whitmore, Nathalie Gilbert) invites you as a practising evaluator to contribute to a body of empirical research concerning the use and application of Principles to Guide Collaborative Approaches to Evaluation (CAE). This invitation is being circulated to English-, French-, and Spanish-speaking members of the global evaluation community.

A full description of the development and validation of the principles is available in an article published in the American Journal of Evaluation (Shulha et al., 2016, vol. 37, no. 2) and further information is available in the aforementioned brochure-style document and list of indicators of the Principles.