809 Assignment 2 Program Models

Assignment 2: “Description and evaluation of a prenatal exercise program for urban Aboriginal women”

For the second assignment in EdCur 809, we were given an article on a program implemented to prevent and manage Gestational Diabetes (diabetes during pregnancy) in aboriginal women through exercise. Deciding which program evaluation model to use was not necessarily a straightforward task.  Many aspects of the program seemed to fit various models, in varying degrees. Ultimately, it was looking at the intended outcome of the program that helped the decision.  When the authors stated the purpose of the program was “both a means and an end to address health-related issues” I considered a combination of models might be required. In the end, I decided that Stufflebeam’s CIPP model was best suited to look at the effectiveness and quality of the program, but it did help to look at the purposes of the other models to eliminate them as choices.

 The CIPP model looks at the areas of context, input, process, and product to evaluate a program.

Looking at the context helps to determine what the program is actually for: its objectives, purposes, evaluation criteria.  In this case, the primary target was pregnant aboriginal women, a high-risk group for gestational diabetes.  The program had specific data on socio-economic factors, such as wealth, education, and employment.  The context of a large urban center was also a factor.  The program coordinators also considered indigenous supports such as an Elder, for the aboriginal-only participants.  Analyzing the context would be a strength of the CIPP model.

The input stage of CIPP considers the design and details of the program.  In this case, the program designers looked at feasibility in several areas. Economics seems to have played a big part in their decision-making as it was mentioned frequently. Some examples included joining scheduled water aerobics so there was no pool rental, free information, and library materials for use. Employment of a nurse, physiotherapist, exercise instructor, and aboriginal facilitators were an economic factor, as well as transportation and child care costs covered by the program. They used survey results to determine location, and used client feedback throughout to make changes (popularity of water aerobics led to increased pool time.) Socially, the design provided for a lot of interaction amongst clients through fun activities, games, door prizes, and more. They provided many supports in terms of an “extensive, ongoing campaign to advertise,” advance schedules, and phone call reminders. Scientifically, they program designers must have done tests on the women before, during, and after pregnancy to determine if they developed gestational diabetes (9% did) but no data was presented. Considering the main objective was to increase the physical activity of the pregnant women, many design factors were important.

The process stage of CIPP is to gauge the program’s overall effectiveness. Because the article we viewed was a sample, perhaps this area was omitted, or possibly the program did not have a formal summative process to gather feedback. No data or anecdotal notes were included at the end of the program, although it might be inferred that because surveys were completed before the program, that perhaps they were also completed at the end. As mentioned above, it seems that some informal data was gathered throughout as there were some changes made to programming, but a specific mechanism wasn’t mentioned. It is at this process stage of evaluation that the lack of substantive data might be discovered.

In the final product stage of CIPP, information is gathered for the purpose of evaluation: did the program actually work? Because this program was completed, summative evaluation is most appropriate to judge its merit in meeting the original objectives. Short term effects of the program could be done for the women who had previously participated in a two-year pilot study.  For the women who joined as part of the “parallel” group, anecdotal and other evidence could also be collected. No mention was made whether the clients were satisfied with the program, or whether any follow-up was done after they left the program. At one point, it mentions that women who had given birth were still invited to participate, so post-partum results were possible.  Again, there seems to be a lack of scientific data listed here, but if it was collected, it would be useful at this final stage to make decisions on its effectiveness.

 

After working through the CIPP stages with the Gestational Diabetes program, it reaffirmed to me that it would be an effective model to evaluate the program.  Having said that, looking at the other models, they were eliminated for various reasons. The Stake-Countenance model and the Provus-Discrepancy model both consider ‘standards’ heavily. In this program, no set standards were listed at the beginning, aside from a general fitness goal, and so it is difficult to hit targets that might not exist. Because they seemed to be taking a holistic approach, the Naturalistic model was something I considered as it “focuses on social interactions” and “doesn’t look at documentation.” (Wilson 19) Both of those are applicable here, but the gestational diabetes program did have a goal it was trying to achieve. The Transactional Evaluation model by Rippey could potentially work. It follows a similar program to CIPP in that it determines an issue, figures out a response to the issue, creates a program to help, monitors the program, and then provides feedback on effectiveness.  My understanding of this model, however, is that it goes beyond looking at program effectiveness particularly in the area of interpersonal relationships. (Howard 14)  I don’t think that the program coordinators here gathered enough data from the primary clients, let alone other stakeholders, in order to facilitate this model.

So through a process of elimination, and by putting the program details through the CIPP model, I feel it is still the most appropriate model to evaluate the effectiveness of the gestational diabetes program.

 

Klomp, H., Dyck, R., and Sheppard, S. (2003). Description and evaluation of a prenatal exercise program for urban Aboriginal women. Canadian Journal of Diabetes, 27: 231–238.

Wilson, J. (Professor) (2014, February 1). Background to PE. EdCur 809. Lecture conducted from University of Saskatchewan, Saskatoon, SK.

Howard, L., & VanCollins, J. (n.d.). The ABCs of Program Evaluation. Program Evaluation Workshop. Retrieved February 3, 2014, from http://www.ncpublicschools.org/docs/21cclc/.../abc-program-evaluation.pdf

Stufflebeam, D. (2003, October 3). The CIPP Model for Evaluation. 2003 Annual Conference of the Oregon Program Evaluators Network. Lecture conducted from OPEN, Oregon.