in addition...

Perhaps there was grant money left over or a position that needed to be filled within a specific time frame, as I cannot understand why the program did not continue.

It seemed the program was flexible enough, as participants could work at their own pace, invite a friend and as instructors learned about what the participants enjoyed, modifications were made (eg. swimming).

Perhaps, the evaluation used was specifically Goal-based, where the program met and fulfilled its goals: "yes the program worked," being the only purpose.  Perhaps the sustainability of the program and feasibility of it continuing is another type of evaluation...

Assignment #2

Assignment #2   - Evaluating  Program Evaluation

After reading the program explaining the implementation of a prenatal exercise program to aid Aboriginal women living in Saskatoon and their children, I found Stufflebeam’s model most appropriate.  I believe a formative approach to the evaluation would be most beneficial to direct decision making and future recommendations.

The goal or according to Stufflebeam’s model, context evaluation, indicated the problem: What can be or needs to be done to promote regular exercise in Aboriginal women in order to diminish or decrease the risk of GDM and promote healthy pregnancies? 

One question I had was what other programs exist or once existed which had similar goals and how did they fare?

When researching the input or plans of the evaluation, it was very clear that creating an exercise program and inviting pregnant women to participate would be the best approach. 

The process or actions and how the program should be implemented were well thought-out.  The appropriate location of the YMCA being relatively central to the city, along with food, maternity bathing suits, healthy snacks, providing free childcare and the program being free itself would attract, not hinder women being involved in the program.

The product or outcomes of the project in the initial article were not given, however the program was designed to improve the participants’ involvement in regular exercise.  It would be interesting to see out of the 69 women involved, how regular their participation was? 

According to the article, “From a health promotion perspective, this program functioned as both a means and an end to address health-related issues in the participants,” (Klomp, Dyck, and Sheppard, 2013), so it is curious to me why the program ended in 1997 and only ran for two years?

Questioning this, I wonder if the evaluation completed was a summative evaluation to sum up an effort and examine the worth of the program, rather than make suggestions or aid in decision-making? 

Assignment One

Assignment One

 An evaluation was completed on the program: Culturally Responsive Youth Work: The Journey Matters . http://www.joe.org/joe/2011october/a9.php

 The program recognizes the diversity which exists among students and cites Geneva Gay, an expert in cultural responsiveness, “…the academic achievement of ethnically diverse students will improve when they are taught through their own cultural and experiential lenses,”( p. 2).  The pilot program, Culturally Responsive Youth Work: The Journey Matters training program believe that individuals working with diverse youth:

1.     Must develop an “intercultural mindset and skillset,” (p. 3)

2.     need to recognize and self reflect on their cultural and how it has affected their lives

3.     must develop skills to recognize areas of misunderstanding and learn to make adaptations during intercultural interactions

The Program

 The program was designed as a six day, 18 hour cultural training designed to provide the participants “…with the knowledge and skills to be culturally responsive to all the young people living in their communities,” (p. 2).  Specifically the program was designed to educate and train the participants on the following six criteria:

1.     Understand the meaning of subjective culture and becoming aware of one’s own cultural self.

2.     Raise awareness of the impact of privilege and oppression in their own lives and others.

3.     Recognize the value of learning about the context and cultural differences of the young people youth professional engage.

4.     Identify and develop skills and resources for building intercultural relationships.

5.     Assess current programming and create next steps for making youth/student programming more inclusive and empowering.

6.     Make a personal commitment to ongoing development of intercultural sensitivity in order to create mutual adaptation strategies (p. 4&5)

The Evaluation Process

The program was evaluated by the participants in the following ways:

1.     At the end of each of the six training sessions

2.     An evaluation was given at the over all completion of the training

3.      A pre-then-post “retrospective”, (p. 6) test was administered, using a 5-point Likert scale (p. 6). 

4.     A “retrospective pre-test was used based on recommendation in the literature,” (p. 6) to gain participants understanding and changes of their cultural competence.

5.     Participants were surveyed on their beliefs and attitudes.

6.     Qualitative feedback on their experiences was also collected from participants.

7.     A follow-up interview with sixteen out of the twenty four participating members eight months after the training, lasting less than 15 minutes per interview.  The questions posed were: * Have you utilized the material from the Cultural Responsive Youth Work training in your classroom/site?  If so, how? If not, what has prohibited you?  * What impact (if any) has this training had in your practice?  * As part of the training, you developed a plan to implement at your site.  Did you follow through with the plan?  Explain.  * During the training, you identified a partner to support you and keep you accountable for implementing your plan.  Has this been an effective strategy or what other methods could be better forms of support? (p. 6 & 7).

Strengths of the Evaluation

            The evaluation asked for feedback from participants at the end of every session, which would provide the evaluators with accurate information, as everything was fresh in the participants’ minds.  Participants would be reflecting on what they had learned each day, rather than try to remember at the end of the entire training.  The evaluators also asked participants for qualitative feedback on their experiences, which provided them with examples and explanations of their perceptions and thoughts.  The evaluators noted that the post-training evaluations provided them with accurate short-term learning outcomes, which would be important to record for assessment day to day, but also gave the evaluators information for comparison long term as well.

            The interviews conducted eight months after the training would be valuable in assessing whether or not the skills and concepts learned in the training were actually implemented and helpful.  The participants were given two reminders of the upcoming phone interview, which would a good idea as it would allow participants some to think about what they might share.  The phone interviews were conducted by an anonymous evaluator, which would allow the information to be accurate and impartial.  I also thought the four questions asked during the telephone interview, would provide the evaluators with important information and give them a good idea of the training information and skills were being utilized or not. 

Weaknesses of the Evaluation

            It would have been interesting to see the questions asked each day of the training.  I wondered if they were all the same series of questions each day (just a standard rating) or were they tailored to allow feedback on the specific skills or ideas presented that day. 

            The phone interviews seem short, fifteen minutes was the longest conversation and I wonder how explicit feedback can be when four questions were asked in that time frame. 

            Out of the 24 participants, only 16 completed the phone interview, which is only a little more than half the participants.  I would be interested in knowing why, as perhaps they did not feel the training made a significant impact on their work with youth. 

Hey Jay!

Just finished setting  up my BLOG!  Hope you are able to access this...
krista