JA Zambia prides itself as an organization with integrity and transparency. We have global and regional key performance indicators by which we measure the success and impact of our programs. These include student enrollments, volunteer engagement and program completion, to name a few.
JA programs are generally delivered within a school year and students surveys are compared before and after the program. Student surveys can show that students moved toward a favorable attitude on dimensions such as 1) the importance of gaining skills for their future careers now or 2) The importance of being attentive to the industry they aspire to work in as examples of practical impact. Attention is given to avoid an inflated number of schools or students reached (i.e. double counting schools and universities). Data is also captured and disaggregated by gender and age in order to be able to produce useful analytics.
The delivery of in-school programs takes place inside of the traditional school day (i.e. is carried out in schools) and is provided directly for the students. Care is taken to ensure that this data is backed up with paper-based survey forms and audited for accuracy, consistency and quality. Multiple points of reference are used to ensure accurate reporting, such as the number of students per class.JA’s approach for evaluating its programs is consistent with established best practices within the evaluation industry/community.
During the pilot or initial release of a new program, we conduct a formative assessment (or baseline survey) which is either paper-based or technology-based depending on the target group). The formative assessment explores the general nature of the program, establishes baselines for future evaluation, and provides an important feedback mechanism for our curriculum developers to suggest improvements to the program. At the end of the programs, JA conducts summative evaluations to evaluate program impact. These studies explore and assess students’ knowledge gain, skills acquisition, attitude change, and indication to adopt or engage in new behaviors (“behavioral intentionality”) relative to the program’s content. The methodology employed for the summative evaluation is rigorous, and allows JA to calculate meaningful measurements that are statistically sound and can be compared to other student development programs.Examples of questions we seek to answer through our evaluations are:1) What are the impacts of the programs on participating students?2) What is the impact of participation in program on students’ knowledge of financial literacy and other knowledge components embedded in the curriculum?3) What is the impact of participation in a program on students’ acquisition of skills related to the curriculum?