How Data can Support Reflective Practice
Reflective practice is a teaching strategy for improving teaching and learning which emphasizes learning as a personal, internal process of making meaning. Through reflective practice, learners adopt the acquired knowledge, then use that knowledge to modify their thinking, revise, integrate, and accommodate new knowledge. Developing an evaluation tool which includes a pretest and posttest design can address whether educational objectives were met based on knowledge acquisition and attitude changes after instructions are provided. This can be accomplished through well-written educational and learning objectives to guide individualized learning plans, selection of appropriate learning activities, and evaluation strategies. For evaluations to be useful to learners a timely feedback loop must be established. Further, establishing a timely feedback loop encourages the student to learn from the process, to reflect on their work, and to assimilate the knowledge for future practice. The quality of the learning experience and identification of future learning needs is vital for the evaluation process.
One strategy to provide a structural way to evaluate learning using reflective practice is to incorporate Kirkpatrick’s four steps of evaluation which include reaction, learning, behavior, and results:
- Step 1 is the reaction and planned actions phase. This stage evaluates how students respond to the training. Measurements are from verbal feedback, questionnaires, and observation from nurses applying the training in the classroom and work environment.
- Step 2 evaluates what the student learned. In other words, to what extent did the student change awareness, skills, or motivation. Measurements for this step are a pre and posttest change score of skill or knowledge.
- Step 3 evaluates behavior, specifically what changes in clinical performance resulted from the learning process. Measurements are from feedback, interview, or a survey.
- Step 4 consists of a summative evaluation which evaluates the tangible results of the learning process in terms of cost, improved quality, increased production, and efficiency. Additionally, this step evaluates at what level of change in student performance was a result of the training. Measurement is to compare changes against a baseline in terms of what is, and what is desired, in students learning.
Implementation of new knowledge often requires a comprehensive change of behavior by individuals who interact with each other in a complex organization. It should be understood that evaluation will not all be seamless and, in some instances, it will take time to allow students to try out new skills; or, additional plans may emerge for long-term follow-up to measure the retention of learning and sustainment of the training. Sustainability is a more vital component of the success of introducing innovative teaching strategies, and often times requires supportive leadership to enhance the implementation of these evidence-based teaching strategies in clinical practice. Currently, there is limited knowledge about factors which promote sustainability at the individual level. Choosing to support a long-term evaluation process; including formative, summative, and confirmative phases, indicates a strong commitment to measuring and owning the entire process and outcome while refining the reflective practice strategy.
Kirkpatrick’s Model for Evaluation
|Action Steps||Frequency of Evaluation||Timeline||Process to
Modify and Update Plan
|What Will Be Done?||By When? (Day/Month)|
|Reactions and Planned Actions (formative)||Immediately after completion of educational activity.||Ongoing throughout program development.||Document immediate feedback from participants and utilize information to make changes.||Verbal feedback, mentoring, questionnaire and observation.|
|Learning (formative)||Sometime after educational activity after learner returns to work setting.||Weekly for the 1st month then every other week until the end of the evaluation period.||Feedback or communication is clearly evident to the learner on their behavior.||Pre and posttest, check list.|
|Behavior (formative)||Feedback as soon as possible after the learner’s behavior.||Multiple follow up intervals at agreed upon time with learner. Involve learners in setting objectives Offer meaningful incentives when possible.||Behavior, Impact, Silence and Alternatives model for feedback. Document and modify plan as needed.||Feedback, interviews or a survey, self, and peer assessment.|
|Results (summative)||Strategies and actions identified that can help learner to remedy problems.||Outcomes will be tracked month to month and will be measured before and after the training.||Ongoing evaluation, early and often.||Compare changes against baseline, compare cost versus benefits in terms of amount of money save.|
|Strategies and action identified to determining the adequacy of competence of learners.||Outcomes will be tracked a year after student graduate to verify continuing effectiveness of instructional materials.||Yearly evaluation||Multiple data-gathering instruments: questionnaires, interviews, performance assessments, self-reports and knowledge tests.|
Kirkpatrick, J.M. & DeWitt, D.A. (2012). Strategies for assessing and evaluating learning outcomes. In D. Billings & J. Halstead (Eds), Teaching in Nursing: A Guide for Faculty, 4th Edition (pp. 441-463). St. Louis, MO: W.B. Saunders Company. Retrieved from <vbk:978-1-4557-0551-1#outline(126.96.36.199)>.