Читать книгу Choosing 360: A Guide to Evaluating Multi-rater Feedback Instruments for Management Development - John Fleenor W. - Страница 13

STEP 5:FAMILIARIZE YOURSELF WITH THE INSTRUMENT-DEVELOPMENT PROCESS

In order to know how to identify quality instruments, you must understand the basics of sound instrument development.

The development process can be seen as occurring in four stages:

• developing instrument items and feedback scales,

• assessing reliability and validity,

• designing the feedback display, and

• creating supporting materials.

At each stage different issues are being addressed.

When items and scales (fully defined in step 6 below) are being developed, the author must identify, as much as possible, the full range of behaviors or skills that he or she believes represents management or leadership competency. Another question at this stage is whether items of behavior or competency cluster in groups that are internally consistent, distinct from each other, and useful for feedback purposes.

To assess reliability, the author of an instrument must consider whether the measurement of these skills or competencies is stable in a variety of ways. To assess validity, the author must determine whether the scales really measure the dimensions they were intended to measure and whether they are related to effectiveness as a manager or leader. Because 360-degree-feedback instruments are primarily intended for individual development, the question of whether the areas assessed can be developed also must be considered.

When designing feedback, the author should try to maximize the manager’s understanding of the data to enhance its impact. In creating supporting materials, the aim of the author is to help the feedback recipient gain deeper understanding of the theory or research behind the instrument and thereby enhance the ability to interpret and work with the data. Your task as an evaluator is to assess the work completed in each of these four stages and balance what you find against the needs of your target group.