Evaluating Programs and Human Performance

1)      For this week’s module, I researched two models of evaluations used in instructional design. One of those models (although in his article he does not refer to it as a model but rather as an approach) is Patton’s utilization-focused approach. In his article A utilization-focused approach to contribution analysis (2012), Patton asserts that “utilization-focused evaluation begins with the premise that evaluations should be judged by their utility and actual use; therefore, evaluators should facilitate the evaluation process and design any evaluation with careful consideration of how everything that is done, from beginning to end, will affect use” (2012, pp. 365- 366). Patton examines “the process of working with primary intended users in rendering judgments about the extent to which the preponderance of evidence supports a meaningful and useful conclusion about degree to which an intervention has affected observed outcomes and impacts” (pp. 365). To Patton, the primary intended users are the stake-holders in the program. The evaluator actively involving the intended users in the process is extremely important in order to meet their needs. Instead of specifically using a ready-made evaluation tool, the evaluator and the primary users work together to create a method of evaluation that works best for them. I felt that this was a constructivist approach, because the evaluator acts as more of a facilitator guiding intended users to make decisions that will best promote use of the training or program.

The second model that I researched was the ADDIE model, which is a model of instructional design with evaluation as one of its components. I feel that this model, too, is rooted in constructivism. I chose to include this model after reading an article entitled Using the ADDIE Model in Designing Library Instruction (Reinbold, 2013). There are five stages of the ADDIE model: analysis, design, development, implementation, and evaluation. Reinbold informs us that “The purpose of ADDIE is not to restrain but rather act as a guide or roadmap for a designer to reach the best possible training solution” (2013, p. 246). The five stages of the model are cyclical in nature, and each phase impacts the next phase (Reinbold, 2013, p. 246). The analysis phase involves identifying the problem or learning objective. The design phase is the planning phase, determining which activities and tools are best suited to achieve the learning goal. The development stage, which usually overlaps with the design phase, “includes materials, instructor aids, scripts, student assignments, and any other product needed to deliver the course” (Reinbold, 2013, p. 250). In the implementation phase, the course, training, or lesson is delivered. Finally Reinbold tells us that “the evaluation phase takes place during every step of the process” as the entire training, course, or lesson is constantly being examined and revised as needed (2013, p. 252). The evaluation phase usually consists of both formative and summative assessments.

2)      One can measure the amount of learning and learner satisfaction was achieved in order to reflect on whether particular instructional design was successful, but other questions should also be taken into consideration as well. Some of those questions include, but are not limited to:

a)      Was this program/ training/ course cost efficient? For example, our district has put a lot of money into a program called Reasoning Minds. When it was first introduced, the district put money into training the teachers who would be using it, which included a program facilitator. After the first year, only one teacher was to be trained and receive assistance from a facilitator. In my personal opinion, I think it was a waste of money to only train one teacher. The other teachers who were not trained do not know how to most effectively teach with the program. As a result (according to my informal observations), student achievement is not where it should be and morale about the program is low. Technically, this program was intended to be the main math curriculum, not supplemental. Which brings me to my next question:

b)      Is the program best suited as the main curriculum/ project or supplemental?

c)       And finally, is this the most efficient way to effectively reach intended learning goals?

3)      As our professor, Dr. Lee said, Many times a non-instructional approach is a more appropriate solution to solving learning problems. I can think of a couple of instances and solutions.

I have taught some very eager and curious children who absolutely hate to read, and therefore struggle with literacy. It’s “boring” or intimidating. Or perhaps these students have attached a “dork” stigma to the idea of reading. In any instance, this breaks my heart. I have heard of a program in which people bring their dogs to the library. Special students get to read with their furry buddy, which can encourage a non-threatening and encouraging environment for reading. I hope to be able to implement this solution as a librarian.

One year I had a couple of students who struggled academically in almost every subject. I honestly feel that their home environment, while their mothers were extremely kind, didn’t really place much value on education. These girls, however, tended to enjoy science instruction. After visiting the Museum of Nature and Science in Fair Park, they learned a plethora of things they could do as a scientist. Becoming a scientist was something they had never thought of before going on this field trip. I believe this experience and my encouragement pushed them to work harder academically so that they could strive to one day work as an astronaut or robotics engineer.

 

 

 

References

 

Reinbold, S. (2013). Using the ADDIE Model in Designing Library Instruction. Medical Reference Services Quarterly, 32(3), 244-256.