Somalia: TERMS OF REFERENCE END Of PROJECT EVALUATION for SOS Children’s Villages Family Strengthening Programme, Berbera.

1.Background

SOS Children's Villages is a non-governmental social development organisation that has been active in the field of children's rights and committed to children's needs and concerns since 1948. SOS Children’s Villages Somaliland (referred to as SOS Somaliland) has officially been working in Somaliland since 2000.

SOS Somaliland is an active member of the worldwide federation, SOS Children’s Villages International, the umbrella organisation of all national, independent SOS Children’s Villages Associations (based in Innsbruck, Austria). SOS is currently working in 134 countries worldwide, with the objective of supporting children, who are in danger of losing their parents and their families or who have already lost them, in the best possible and in an individual way.

SOS Children’s Villages Somaliland has been implementing a Family Strengthening Programme in Berbera since April 2010, which was co-financed by the German Federal Ministry of Economic Cooperation and Development (BMZ). The Programme has so far undergone two project phases with a duration of three years each. This evaluation shall focus on the results and achievements of phase II of the programme, which built up on the first phase. The goal of the programme is the improvement of the economic conditions and life situation of the parents, caregivers and young people. Parents and caregivers established Community Based Organisations and set-up microcredit schemes. Further they received vocational training courses on the successful implementation of their business ideas. Youth and young adults took part in practical and theoretical vocational training programmes and thus developed vocational perspectives. In addition, awareness raising measures on appropriate health care are implemented.

Specific objectives of the SOS FS Berbera project:

• By taking part in micro-credit schemes and by successfully implementing their business ideas/plans at least 70% of the participants have increased their income by 60% by the end of the project.
• By the end of the project at least 60% of the participants have completed the vocational training programme successfully.
• By the end of the project 90% of the graduates find themselves in employment. 70% are self-employed, 20% in wage-employment.
• 80% of the participants have increased knowledge on health, hygiene, HIV/AIDS and sexually transmitted diseases and apply it in daily life
• Hygiene related diseases have decreased by 30%, especially amongst women.
• 70% of the participants have an increased risk awareness related to organized crime (e.g. piracy)

Overall objective

Sustainable improvement of the living conditions and development opportunities of disadvantaged families in the 4 suburbs of Berbera, Somaliland

Project objective

200 parents and caregivers as well as 300 youth and young adults are able to improve their socio-economic situation and the living conditions of their families

Specific Objectives

1. Establishment of 4 micro-credit institutes by what 200 parents and caregivers reduce their poverty

Target

• By taking part in micro-credit schemes and by successfully implementing their business ideas/plans at least 70% of the participants have increased their income by 60% by the end of the project.

• Development of job perspectives for 300 youth and young adults through the establishment of a vocational training programme

Targets

• By the end of the project at least 60% of the participants have completed the vocational training programme successfully.

• By the end of the project, 90% of the graduates find themselves in employment. 70% are self-employed, 20% in wage-employment

• Prevention of risks in everyday life related to health and organized crime

Targets:

• 80% of the participants have increased knowledge on health, hygiene, HIV/AIDS and sexually transmitted diseases and apply it in daily life.

• Hygiene related diseases have decreased by 30%, especially amongst women

• 70% of the participants have an increased risk awareness related to organized crime (e.g. piracy)

Target Group

1.Direct Target Group

The direct target group consists of 200 low-income parents and 300 (childless) youth at the age of 14-25 years from Darole, Barwaqo, Jamalaye and Burco Sheikh with the following features:

200 parents – participation of micro-credit programme - Low per capita income (below 2 EUR/day) - Low educational level and only very limited access to the formal labour market - Parents/Caregivers of many children (average 8-9 children) - No access to investment capital or credits - Low awareness of risks in daily life related to health and organized crime - High prevalence of infectious diseases - Overcrowded housing situation, characterized by hygiene problems

At least 50% of these parents are single mothers.

300 (childless) youth at the age of 14-25 years– participation in vocational training programmes - Low educational level or dropouts who are however literate - Only very limited access to formal labour market - Low per capita income (2 EUR/day) - Low awareness of risks in daily life related to health and organized crime - High prevalence of infectious diseases - Overcrowded housing situation, characterized by hygiene problems.

At least 50% of the young people are female.

2.Indirect target group

At least 1600 children – out of the families of the direct target group - will benefit indirectly from the measures. Besides the participation in awareness building measures at the 4 project locations will be open to the whole community.

2.Purpose of the external evaluation

The external evaluation will address the following issues:

1. What changes has the project made in the lives of the participating families within target group, their children and their communities? What structural changes has the project brought about (model-character, broad effects, etc.)?
2. How relevant, effective, efficient, sustainable and participatory are the project interventions?
3. What lessons can be drawn from the project that can be taken to further develop the programme?
4. To assess the economic sustainability of families and their performance in income generating activities, sustainability of micro-credit institutes and sustainability of activities.
5. To assess the employability of youths and follow up of their situation after graduation in vocational trainings.

6. To assess the positive changes and impact on the lives of project target group as well as on the lives of surrounding community who have been directly involved in the project activities, including the enrolled families and their children.

7. To assess the progress made towards achievements of specific objectives.

8. To assess the cooperation with local authorities.

9. To measure if the capacity building of target group has contributed to reach project’s objectives as stated in the project document as well contribute to project’s interventions sustainability.

10. To assess whether the project is relevant to the real needs of the intended beneficiaries

11. To identify the best practices, lessons learned, challenges arising from programme implementation as well as drawing conclusions and make recommendations

Likewise, the evaluation should identify the key lessons learned that can be applied to improve the relevance; efficiency; effectiveness; sustainability and impact of the project interventions. The findings of the evaluation will be used as a basis for strategic planning on national level; for improvement of programme design and for future development of family strengthening approach in the country (Somaliland).

Moreover, the external evaluation should look at the following:

o Relevance

• To what extent is the project focussed on target group – low income families and disadvantaged youth?

• To what extent the project interventions respond to the needs and priorities of the project participants?

• To what extent project design address locally defined needs and priorities?

• To what extend is the project extended to reach families and disadvantaged youth in need?

• To what extend are project interventions relevant to multiple social services of the area?

o Effectiveness

To what extent are the objectives of the project being attained?

To what extent have caregivers increased their income by successfully implementing income generation activities?

How many families successfully implemented their business ideas/plans and increased their income by 60% at the end of the project?

How many youth have completed the vocational training programme successfully and found themselves in self-employment or wage-employment?

To what extent project participants (caregivers and youth) have increased their knowledge on health, hygiene, HIV/AIDS and STI’s and applied it in daily life?

To what extent hygiene related diseases have decreased especially amongst women caregivers?

To what extent hygiene and sanitation kits given to caregivers contributed to family health and home conditions?

To what extent have beneficiaries been satisfied with the project interventions?

Is replication or removal of certain project components are required to increase effectiveness of future projects?

o Efficiency

• Is the relation between input of resources and results achieved appropriate and justifiable (cost-benefit ratio)? What are the social benefits for families, youth and CBO’s compared with resources spent for activities?

• How the micro-credit loans have made changes and contributed to the self-reliance of caregivers and their families?

• Have resources been used most economically?

• Could the outcomes and results have been achieved at lower cost through applying a different approach?

• Is the project more efficient in terms of HR and logistic issues after implementing the project, If yes, how?

o Sustainability

• Is the programme supported by local/national institutions? Do these institutions demonstrate commitment and capacity to continue programme activities or replicate it?

• To what extent can activities, results and effects be expected to continue after SOS involvement has ended? Do CBO’s have financial capacity and commitment to maintain programme?

• Is there progress towards SOS withdrawing from its direct involvement and handing over the full responsibility to run the programme to an implementation partner (here CBO’s), or to state structures?

• Is the program more sustainable?

o Participation

• To what extent are stakeholders (participant families, youth, partners, local authority) involved in the design and implementation of the programme? In particular, to what extent are stakeholders involved in the family development planning process?

• To what extent is the programme designed to develop the necessary local institutional (governmental and/or non-governmental) capacity to respond to the problems low income families and disadvantaged youth?

• Is the programme having an active leading role in community supportive programme, how?

3.Key areas for the external evaluation

Within the family strengthening project, children, families, and communities have been supported in various areas, which the external evaluation exercise should look at. The evaluation will focus on following key areas and illustrative set of questions. However, the evaluation consultant (team) needs to decide in cooperation with the national Programme Development Director, HGFD and field team in which of the below mentioned areas and in what extent apply to the particular programme and set of questions will be further elaborated during preparatory phase. Nevertheless, main sections 1-6, which look at the definition of the target group as well as the selection process of participating families; programme interventions supporting families and communities; main approaches; programme partners; allocated resources should apply.

4. Existing information sources:

4.1. Key reference documents:

• FS Berbera project proposal documents.
• Project reports and Impact Matrix
• NA strategic plan and annual plans;
• Programme Operation plan;
• FS Berbera self-evaluation report.
• Database; Family and youth case files; any other collected data by programme team.
• Monitoring and audit reports
• Data from FS Database

4.2. Key persons:

· Caregivers (families) and youth participating in FS program;

· FS Programme team

· Programme partners; community members( CBO’s)

· Programme management on national level

5. Evaluation Approach, Process and Methods:**

5.1 Evaluation Approach

· In his/her inception report, the consultant is expected to use a variety of methods to collect and analyze data. It is further mandatory to incorporate in the inception report the detailed evaluation plan, with precise key respondents, and timeliness.

• The evaluation should use participatory methodologies to involve programme participants (families currently being on the programme as well as families that have already left the programme) and wherever possible children should be involved in the design and implementation of the evaluation.
• The evaluation process must ensure triangulation: findings and results must be cross-checked between the external evaluator and the national Programme team.
• The evaluation process must ensure that results are reported back to the participants in the process
• As this external evaluation is an opportunity for learning for the programme staff, it is critical that the programme staff participate throughout the evaluation process: evaluation design and planning; information collection; and results dissemination.

5.2 Evaluation process/Main responsibilities of external evaluator:

5.2.1 Evaluation plan

External evaluator should plan the evaluation design in coordination with programme staff on local/national level. That includes:

· Propose composition of a survey team;

· Plan and design data collection process:

· Identification of the major stakeholders who are associated with the programme to be interviewed, such as the direct participants of the programme (Care-givers) CBO’s, key local government representatives, and other service providers

· Selection of representative sample of communities or groups to be interviewed on the basis of agreed criteria

· Agreeing on the type of information to be collected

· Preparation of checklists and other tools for data collection

· Develop a methodological tools for data collection and consult that with programme staff on project/national/ level

5.2.2 Data collection and field mission

The evaluation will include a site visit to programme location to meet programme participants (families/children); project team and project stakeholders; and to collect information in accordance with the requirements stipulated in the evaluation design. This visit is expected to be no longer than two weeks in duration. Programme team is to be consulted on visit period. Main tasks include:

· Review the project documentation and other sources of information

· Collecting of data

· Facilitate learning during the data collection exercise

· Field visit with project staff

5.2.3. Data analysis and elaboration of evaluation report

External evaluator will analyse collected data and will prepare an evaluation report that describes the main evaluator’s findings, recommendations and lessons learned. The final report should follow the structure and content as outlined in the terms of references.

That stage includes:

· Analysis of the data

· Preparation of a draft report

· Present the findings to the respective programme staff on local/national level to ensure triangulation

· Finalise the report after inputs from various stakeholders.

5.3. Methodology

The evaluation should use quantitative (e.g. surveys) and qualitative data collection methods such as semi-structured interviews (for example with focus groups, key informant, individual interviews, etc.) as necessary.

The methodology of evaluation will include the following:

· Document review including analysis on key reference documents listed in Terms of Reference;

· Case study of randomly selected beneficiaries files (present and those who exited the programme)

· Interviews (structured and/or semi-structured; in person) with key informants.

· Focus groups with selected key informants – children, care-givers, community CBO members;

· Other methods relevant to evaluation objectives and scope as suggested by consultant

6. Expected deliverables:

External evaluation team will commit to the following key deliverables:

6.1. Evaluation design – contains the evaluation framework; detailed evaluation methodology; work plan and budget

6.2. Developed evaluation tools

6.3. Draft evaluation report – Draft report will be prepared in line with the proposed structure below and should be submitted to national programme management and site project coordinator, electronically via e-mail, in English.

6.4. Final evaluation report - The findings of the external evaluation shall be presented in written report following the proposed outline. Attachments – Templates of applied evaluation tools (questionnaires; main areas for focus groups and etc.). Final evaluation report should be submitted to National program development Director and site project director in soft and hard copy.

7. REQUIRED EXPERTISE

• The Consultant must have previous experience in conducting child rights-based evaluation activities
• The consultant must have a proven conceptual, analytical and evaluative skills, the ability to conduct independent research and analysis, identify issues, formulate options, arrive at conclusions, and make sound recommendations
• He/she must have a proven ability to write, read and speak Somali and English in a clear and concise manner.
• Holding at least Master’s Degree in social sciences or in other relevant development field is required.
• Knowledgeable in quantitative and qualitative research methods
• Sound computer skills including data analysis software

• Knowledge of the local context and culture is additional advantage

  Appendix 1: Structure of final evaluation report

  TABLE OF CONTENTS

  • Table of contents

  • Figures and tables

  • Acronyms

  SUMMARY (4 - 6 pages) * Background and project context

  • Findings and conclusions

  • Recommendations and lessons learned

  1.INTRODUCTION

  1.1.SCOPE OF EVALUATION** * Brief project description

  1.2.BACKGROUND AND RATIONALE** * Reason and justification for evaluation

  • Aim and purpose of evaluation

  • Key guiding questions

  1.3.EVALUATION MISSION** * Time span and process of evaluation

  • Profile, composition and independence (non bias) of evaluation team

  • Participation of partners and target group in evaluation

  • External factors influencing the evaluation process and respective consequences

  2.METHODOLOGY

  2.1 EVALUATION APPROACH AND METHODOLOGY * Methodology and instruments

  • Measures ensuring the protection of the stakeholders involved

  2.2 CRITICAL ASSESSMENT * Measures ensuring the protection of the stakeholders involved

  3.CONDITIONS FOR EVALUATION * Local context, problem statement, initial potentials of project and important changes throughout project period

  • Presence and actions of other stakeholders

  • Risk factors for achieving project objectives

  4.PERFORMANCE OF GERMAN (HGFD) AND IMPLEMENTING PARTNER (SOS Somaliland) * Staff qualification

  • changes at German and implementing partner organization

  5.EFFECTIVENESS

  5.1.RELEVANCE** * needs of the target group are addressed and the objectives of the donor (BMZ), German and implementing partners are attained

  • adequate developmental approach and conceptualization

  5.2.EFFECTIVENESS** * Quality of project planning

  • Quality of system of indicators and objectives

  • Quality of project implementation

  • Motivation, ownership and legitimacy of implementing partner

  • Quality of project management

  • Achievability of project objectives

  • Other effects on output and impact level (incl. negative, if any)

  5.3.EFFICIENCY** * Cost effectiveness of the project

  5.4.DEVELOPMENTAL IMPACTS** * Achievement of overall objective

  • Model-character, structural changes and broad effect

  • Other effects of overall, developmental impact (incl. negative, if any)

  5.5.SUSTAINABILITY** * Viability and durability of positive impacts (once BMZ-funding has ceased); incl. developments in surrounding areas.

  • Risks and potentials of sustainable impacts for the target group as well as on organisational level

  6.CROSS-CUTTING ISSUES * Cross-cutting topics of development cooperation

  • Contribution to organisational goals

  7. CONCLUSIONS AND RECOMMENDATIONS

  7.1CONCLUSIONS

  7.2RECOMMENDATIONS

  7.3LESSONS LEARNED

  ANNEXES * Terms of Reference

  • Composition and independence (non bias) of evaluation team

  • Evaluation matrix

  • Evaluation plan and time diagram

  • List of stakeholders consulted

  • Bibliography/reference

  • Questionnaires/other data collection instruments

  • Debriefing Protocol

  • System of objectives and indicators

  • others if necessary

APPLICATION PROCEDURES

Interested Consultants/Institutions who meet the conditions specified herein are invited to submit their technical and financial proposals including proposed work plans, CVs of lead Consultant and a clear demonstration of understanding and interpretation of this Terms of Reference (TOR) to the HR on: hr@sossomaliland.org and indicate External Evaluation of Family Strengthening Programme, Berbera, in the subject line, to reach us not later than 25^th January 2017.