Somalia: TERMS OF REFERENCES FOR THE FINAL EVALUATION OF WAXBARASHADU WAA IFTIIN II(EDUCATION IS LIGHT II) PROJECT

TERMS OF REFERENCES FOR THE FINAL EVALUATION OF WAXBARASHADU WAA IFTIIN II(EDUCATION IS LIGHT II) PROJECT

1. INTRODUCTION

CARE is an International NGO working in Somalia. CARE and its partners work with vulnerable communities to address the underlying causes of poverty, promoting peace and development, by supporting sustainable livelihood and economic development initiative programmes, strengthening civil societies, responding to emergencies and advocating for policy change.

The Puntland State of Somalia is in the midst of momentous political, social and economic transformation since its leaders declared the territory an autonomous region in 1998. The region has managed to maintain relative peace and security in recent years, which has enabled it to establish political and administrative institutions, basic social services, an active civil society, and a growing private enterprise community. In addition, much progress has been made in the education sector in the last few years. Despite these improvements, however, educational provision, participation and completion in Puntland are among the lowest in the world. The GER[i] for primary and secondary education as per the Ministry of Education and Higher Education (MoEHE) 2015/2016 EMIS stands at 56.9% (Girls: 51.8%; Boys: 63.8%) and 14.7% (Girls: 11.10%; Boys: 18.10%) respectively.

The major challenges experienced in the education sector are multifaceted factors and include weak technical and institutional capacity of the MOEHE, low access to quality, relevant and equitable education system at all levels (Early childhood. primary, non formal, secondary, Higher Education and TVET) as well as limited pedagogical skills and the availability of teachers, especially in rural and nomadic areas. All these constraints lead to poor service delivery.

It is against these backdrops that a consortium comprising of CARE, Save the Children, ADRA and VU Amsterdam University are implementing the two and half years EU-funded Waxbarashadu Waa IftiinII(Education is Light II) project to address the challenges in close coordination, support and guidance of the MoEHE in Puntland. The project aims to contribute to Education and training efficiently and effectively delivered’, contributing to poverty alleviation within a peaceful, secure and democratic Somalia and specifically strengthens Education and training services, responsive to the priority requirements of the Somali population, efficiently and equitably delivered

Through this ToR, CARE (lead agency) is planning for a final evaluation of the two and half years project (scheduled to end on 9 March 2018) to ascertain the achievements and implementation levels against the set indicators and also inform the subsequent phase of the project on the needs and priorities of the MOEHEon continuity and consolidation of the existing gains of the EU funded consortium project.

The project interventions were expected to achieve the following results, among others:

Overall objective: *Education and training efficiently and effectively delivered’, contributing to poverty alleviation within a peaceful, secure and democratic Somalia***

Specific Objective: Education and training services, responsive to the priority requirements of the Somali population, efficiently and equitably delivered

Results:

· Result 1: Increased access to equitable and quality education for learners

· Result 2: Increased participation of youth and adults, including vulnerable groups, in technical and vocational education and training

· Result 3: Capacity of education institutions, administrations and systems strengthened

1. OBJECTIVES OF THE EVALUATION

The results of this evaluation are expected to provide a detailed measure of the degree to which stated objectives and results were achieved (or not) in the project. It should generate relevant findings, lessons learned and recommendations which will be shared with key stakeholders of the project and used by the implementing agencies to guide and inform on the best practices and strategies in for improving other future education programs. The final evaluation of will have full participation of the project staff and stakeholders.

The specific objective of the evaluation is to:

Objective 1: Provide information on the performance of the project against all indicators and the project’s overall relevance, effectiveness, efficiency, validity of design, sustainability, factors affecting performance, alternative strategies and its strengths and weaknesses.

1. THE EVALUATION SCOPE

The Consultant should take into consideration the evaluation criteria of Relevance, Effectiveness, Efficiency and Project Performance Rating. The criteria should include assessment of all the different components of the project including Impact and Sustainability, activities implemented or not done in reference to the indicators stated in the project log frame.

The consultant will use the rating scale below for grading the level of achievement of the main activities projected to be implemented by the project and give a summative mean grade for all the activities evaluated and graded. The mean grade will provide the overall performance of the project.

Table 1: The Rating Scale

IMPLEMENTATION

The activity was implemented in:

SCORE

IMPACT/Likely Achievement

A completely appropriate, efficient and timely manner

1

Completely achieved

A largely appropriate, efficient and timely manner

2

Largely achieved

A moderately appropriate, efficient and timely manner

3

Partially achieved

an appropriate, efficient and timely manner to a very limited extent

4

Achieved to a very limited extent

neither an appropriate, nor an effective or timely manner

5

Not achieved to any discernible extent

Unverifiable

X

Unverifiable

Differences among areas of intervention, related to different constraints and problems, should be analysed and reported for the different level of analysis. The evaluator should assess the following:

Relevance

• Investigate and make conclusions on the appropriateness of project concept to the problems it was supposed to address taking into account the prevailing political and social economic situation in Puntland.
• Review the complementarity of the project with other CARE interventions in Puntland and particularly how this project has contributed to the reduction of vulnerability through the provision of quality primary education and non- formal education.
• Review the project design including the risks and assumptions and their impact on the project.

Efficiency

• Assess how well the project activities transferred the available resources into the intended results in terms of quantity, quality and time.
• Assess how the project strategy used induced sustainable change, particularly in regard to establishment of Consortium relationships with key actors and participatory processes.
• Assess how inputs and means have been converted into activities and the quality of the results achieved.
• Analyse the efficiency of the project implementation and monitoring at result level using the indicators of the log frame and the proposed timetable.
• Assess the level of collaboration with other agencies and the various project stakeholders.
• Verify the appropriateness of the consortium approach and the innovativeness and response by project management to bring changes to the social environment in which the project operates.

Effectiveness

• Measure the extent to which the project achieved its outcomes and purpose using the OVIs column of the Logical Framework
• Assess whether planned benefits have been delivered and received as perceived by collaborating partners and target groups and will continue towards the intended purpose.
• Analyse the effectiveness of project strategies and capacity building efforts to determine if they are sustainable.
• Determine the progress made in achieving results of the project at all levels.
• Assess the quality of operational work planning, budgeting and how the project managed the originally identified risks and others that may not have been foreseen.
• Assess the quality of reporting.
• Check if the assumptions were correct and if not how this has affected the project achievements.
• Assess the quality of the developed training curricula.

Impact

a) Analyse whether the project has made milestones that will contribute to the WWI programme goal of sustainably improving access and literacy levels of school aged children and adult learners and other disadvantaged groups.

b) Determine the impact the project has made on access to primary education with a particular focus on girls and minority groups.

c) Assess ownership of the WWI by the Government/regional authorities in Puntland, CECs and the general population and commitment to continue with the support after the expiry of the current initiative.

Sustainability

d) Evaluate the stakeholder participation in the management/implementation of the project;

e) Measure the level of local ownership and confirm whether key stakeholders are committed to continue and build on the achievements of the project;

f) Confirm whether the community has the intent and capacity to continue the support activities (of the current initiative) including financial and technical capacities.

g) Assess if the regional environment, socio-economic, political and cultural factors are conducive to the continuity of the project support activities.

h) Assess to which level sustainability has been considered in planning and execution of activities particularly in respect to institutions, formal and non-formal, as well as technical and cross cutting issues;

i) Validate the exit strategy in place to hand over the Project to support WWI at the end of the programme;

j) To enable a possible replication, the evaluator is to check whether the process and lessons learned are being documented appropriately so they can be translated into a methodology which will enable others to replicate the activities more easily and effective (multiplier effects).

Cross-cutting Issues:

k) Particular attention will be given to special needs, vulnerable and marginalized groups, gender, cultural identity, peace building, environment mainstreaming, HIV/Aids education, and hygiene education in the project.

On the basis of the above mentioned assessment,

• Measure the extent to which the project is successful in achieving its purpose in the targeted regions. Analyse strengths and weaknesses of the project and identify lessons learned.

• Develop detailed recommendations for how to strengthen: Project management (including monitoring); Implementation (including approaches) and relations with stakeholders in the project area. The recommendations should be placed into the context of how the project should be implemented in order to maximize impact going forward.

• EVALUATION METHODOLOGY

CARE Somalia proposes a participatory, multi-stakeholder approach in undertaking this assignment. Both qualitative and quantitative methodologies of data collection should be employed.

a) Literature review

Literature review will involve a study of various project and MoEHE documents including the two documents earlier mentioned (Annex 1). The consultants will also be provided with the following key documents for review and background information: the WWI IILog-frame (Annex 2), Project Proposal (Annex 1), Project Funding Contract between CARE and EC (Annex 3), EC Guideline for the Original CFP (Annex 4) Project Baseline Survey (Annex 5), the project MTR report and various projects implementation reports to the EC plus any other document considered necessary for this exercise.

b) Data collection

Quantitative methods using structured questionnaires and ‘School fact establishment tools’ (adopted from the baseline) will be used to establish quantitative benchmarks reflected in the log frames. Focus Group Discussions, Individual questioning with a scientific representation of all the stake holders will be used to collect detailed qualitative information. It is expected that the consultant will adapt the baseline tools and come up with innovative ways of combining the various research tools and methodologies to get the right and accurate information.

During the study, the consultant will interview all the stakeholders including beneficiary communities, students, Ministry of Education/regional education authorities, Community Education Committees, School children teachers, head teachers and staff of implementing consortium partners (CARE, SAVE, ADRA, VU and MoEHE), men/women groups, religious leaders, opinion leaders, women leaders and other actors including Local and International NGOs in the region.

Sampling Design and Procedure

The eight (9) regions and 12 focus districts in the project areas will form the sample frame. Both simple random and purposive sampling will be used to determine the sample population based on the consensus reached between the Lead Agency, the Consortium members and the MOEHE. Simple random sampling will be applied at school level to select teachers, students and parents, while purposive sampling will be applied to select the head teachers, centre managers, and CECs.

Data Collection Instruments. The consultant will use the five (5) data collection instruments that were used to capture qualitative data at baseline and adopt the same for the EoP evaluation. The instruments include: semi-structured interview questionnaires; one- on-one personalized in-depth interview questionnaires for key informants; focus group discussions (FGDs) with discussion guidelines; institutional fact establishment sheets. The questionnaires will be administered to the respondents, while Institutional fact establishment sheets will be used to collect specific quantitative and qualitative facts at the sampled institutions, including physical and non-physical facilities, enrolment, teacher establishment, etc.

c)REPORTING AND FEEDBACK

After the evaluation, the consultant(s) is expected to do a debriefing with the with the MOEHE and the consortium partners and project staff on the ground and incorporate their relevant inputs. The consultant(s) should then prepare a draft report and make a presentation to the MOEHE and consortium partners and EC delegation in Nairobi for their inputs after which the final report is prepared. Both draft and final report should have a power point summary not exceeding 20 slides. The main text of the final evaluation report should be presented using Arial font 11 and not exceed 50 pages inclusive of Annexes. The Executive Summary in the main report should not be more than five pages with fully cross referenced findings and recommendations. In addition, a short, separate summary of one page should be provided during the presentation of the final report.

The consultants/Consulting firm needs to produce five well bound hard copies of the final report and soft copies of all the reports sent electronically to a list of recipients in the consortium to be provided. The electronic soft copies should be in Adobe Acrobat and MS Word.

d) EXPERTISE REQUIRED

The consultants/consulting firm should be an experienced and have expertise in the following: -

· Extensive experience in research work and in education assessments/evaluations, including previous working experience in Somalia;

· A degree in education planning and management and/or curriculum development or other relevant fields. Postgraduate qualification in the same will be an added advantage.

· Demonstrated experience in community development including community targeting, rapid rural appraisal and project cycle management.

· Knowledge in Research Methodologies and application of various tools including practical experience in assessments, planning, implementation, monitoring and evaluation of community based interventions;

· Have excellent analytical and writing skills

· Previous Evaluations, especially under EC funded projects will be an added advantage

e) WORK PLAN AND TIME SCHEDULE

Under the direct supervision of the Consortium Coordinator and Program Coordinator the consultant will undertake the following tasks;

#

Tasks

Duration (Man days)

Literature review/Desk Review

3

Preparation of data collection tools, translation into Somali and testing/revision of tools

2

Travel to and from Puntland

2

Field data and information collection

10

Data and information analysis

6

Report writing and presentations

7

Total

30

This assignment will be carried out in a total of 30 days as per the breakdown contained in the table above.

1. COST ESTIMATE AND TIMEFRAME

The Consultants are requested to quote their price for this assignment and to mention when they will be available to carry out this assignment. Due to the urgency of the assignment, time will be of essence in the award of this contract.

1. RESPONSIBILITIES OF THE CLIENT – CARE SOMALIA (Lead Agency)

In support to the consultant to undertake the assignment, CARE Somalia will;

• Pay the entire consultancy fee (as per the contract) after the field work, submission and acceptance of the report, outputs and attachments. The consultant will be paid subject to the completion of the report in a satisfactory manner. This payment will also be taxed in accordance with the Kenyan Government laws, exact amounts will be specified in the contract
• Cater for transport and upkeep as well as other related logistics to and from the field.
• Facilitate access to relevant offices in the field through prior arrangements and consultations.

• Access to existing and useful resource materials while in the course of undertaking this assignment.

• CONSULTANT’S RESPONSIBILITIES

During the period of carrying out this assignment, the consultant will need to establish close coordination arrangements with the sub office in Puntland. In particular, the evaluator will be expected to work closely with the Consortium Coordinator, Program Coordinator and Project Managers of the consortium partners (SAVE the Children, ADRA, VU and CARE). The consultant under this assignment will: -

• Be available for the assignment;
• Prepare all their travel documents;
• Commits to complete a fully satisfactory end product within the agreed time frame;
• Provides the necessary material for use during and after the study.

INTELLECTUAL PROPERTY RIGHTS

All documentation related to the assignment shall remain the sole and exclusive property of CARE and the consortium partners.

APPLICATION PROCESS

Applications should be submitted by 1April 2018. The selection committee will review all applications as they arrive. All applicants must meet the minimum requirements described above, and those unable to meet these requirements will not be considered.

Each application package should include the following:

· An application letter addressing the selection criteria including how the firm’s/group’s previous experience matches the consultancy objectives as well as the interest for the position. It should also indicate the candidate’s availability and consultancy rates. The letter should be no longer than two pages.

· A sample of recently written report for a similar assignment.

· Updated CVs for all consultants including relevant work experience and qualifications.

· Contact details of 3 references.

Applications not including all of the above information will not be reviewed.

[i] MoEHE EMIS 2015-2016

All applications should be sent to somconsultant@care.org with the subject line, “***Application for Waxbarashadu Waa Iftiin (Education is Light II) Final Evaluation*.”**The submission date should be by 26 April 2018.