Country: Somalia
Closing date: 26 Apr 2018
TERMS OF REFERENCE: END OF PROJECT EVALUATION
PROJECT TITLE: ‘*STRENGTHENING CIVIL SOCIETY ENGAGEMENT IN POLITICAL DIALOGUE AND STATE BUILDING PROCESSES – SOUTH WEST ADMINISTRATION AND BANADIR’*
PROJECT ID: FCA_ 11699
1. BACKGROUND OF THE EVALUATION
The project, ‘Strengthening Civil Society Engagement in Political Dialogue and State building processes - South West State (SWS) and Banadir’ is a two years’ project implemented by FCA (FCA) from 1st January 2016 to 30th April 2018. The project was implemented in partnership with the IIDA Women’s Development Organization (IIDA) and the Centre for Research and Dialogue (CRD), both local non Governmental Organizations (NGOs). The NGOs are based in Banadir (Mogadishu) in Somalia, but with operations in other parts of South and Central Somalia. The project was implemented in Somalia South-West State and specifically in Bay, Bakool, and Lower Shabelle regions and Banaadir (Mogadishu).
The design of this project was informed by an assessment and consultation activities to create better understanding of the needs of the target groups and the project beneficiaries. Some of the key findings of the assessment included that there lacked appropriate channels/forums to support civil society and community participation in peace/state building activities; yet it was essential to ensure the needs of communities in Banaadir and SWS regions were communicated to the decision and policy makers for inclusion during the constitution review. There was also need to coordinate advocacy action with other interventions, which focus on social reconciliation in addition to the need to share information on different civic processes in Somali language and use the media/community forums for information sharing. Finally, there was need for citizen engagement from all sections of society - women, youth and minority clans in the state and peace building processes in the country. This action was thus designed to endorse these needs with a focus on building the capacity of civil society and grassroots communities to participate in peace and state building efforts within the South West State and Banadir, thereby addressing the identified gaps.
During the project period, some assessments were done to benchmark the project. These included a Baseline Survey on Civic Engagement in Somalia’s South West State and Banadir and Gender Audit for Bay and Bakool, Lower Shabelle and Banadir regions of Somalia.
In addition to providing the benchmark for the project, these two assessments were also used to inform the content of the Civic Education Toolkit. The Civic Education Toolkit was used to guide training of the Community Advocates in civic education in the SWA and Banadir regions.
Contrary to the planning, no mid term evaluation has been carried out for the project. However, project documentation is contained in the various progress reports (Interim Narrative Reports, monthly updates etc.) and also in the various information communication and education (IEC) materials and other media materials.
The Overall objective of the Action is to Contribute towards strengthening Somali civil society’s capacity to effectively engage with government peace and state building processes to support the implementation of the Somali Compact and Peace and State Building Goals (PSG’s) 1,2,3. The aim was to support civic engagement and the development of new mechanisms to promote better transparency and accountability, through creation of new mechanisms and structures to support advocacy to influence and engage with the Federal government and SWS. In addition, these new mechanisms were aimed at promoting civic engagement and enable better levels of accountability of government actors to their citizens; which in turn would support the development of a peaceful and democratic Somalia. The action also aimed to support improved awareness and transparency of government actors’ decisions and behaviour in relation to the development of the new Federal Constitution.
Specific objective
Specifically, the Action aimed: To increase visibility, voice, and cohesive participation of Somali civil society during the implementation of the Federal Constitution in the South West Administration
The Action’s Key Result Areas were as follows;
Result 1: Improved civic education, knowledge & advocacy capabilities for 120 CSOs leaders; 200 community leaders/members; 400 community members and coordinated engagement activities with government on Federal Constitution (FC) implementation/finalization.
Result 2: Improved mechanisms to enable civic engagement in relation to the implementation & finalization of the Federal Constitution
Result 3: Women, youth, and marginalized groups at the community level have improved civic knowledge and are able to participate in constitutional finalization processes.
The full project proposal is available after signing the consultancy contract.
2. RATIONALE, PURPOSE AND OBJECTIVES OF THE EVALUATION
The purpose of this end-term evaluation is to commission a systematic and objective assessment of the just completed action/project, its design, implementation process and results. The aim is to determine the relevance and fulfillment of the set objectives, efficiency, effectiveness, and to a limited extent, impact/immediate results and sustainability of the interventions post EU financing. It will also capture intended and unintended effects of the action and also investigate the reasons why certain aspects of the action have or have not been implemented as planned.
Through this evaluation, FCA is also interested to deduce lessons learnt from the implementation of the action. These will include but not limited to the approach used, the project design, partnership with local NGOs (IIDA & CRD); the process of implementation and the level of accountability of the project resources and preconditions by the project partners (IIDA&CRD). Please see section 4 for elaboration.
FCA is further keen to establish the overall status of the project at closure for purposes of reporting back/accountability to the back donor, the European Union. This will include feedback not only on the process but also on the outcome results of the Action on the beneficiaries. Furthermore, FCA Somalia programme is also accountable for the action resources internally to the FCA Head Quarters.
The priority issues of the evaluation are founded on the ability of the evaluator to design a very elaborate evaluation framework, a detailed evaluation criteria and an elaborate evaluation methodology; a clearly and most specifically spelt out sampling procedure and sample size, an elaborate GIS coordinate map clearly identifying all the locations from where data will be collected; very elaborate, specific and very clear data collection tools; a well outlined evaluation timeline and finally a very comprehensive outcome evaluation report.
3. SCOPE OF THE EVALUATION
The evaluation will cover the project period 1st February 2016 until 30th April 2018. This period comprises of a three months no cost extension (NCE) period which run from 1st February 2018 to 30th April 2018).
Geographically, the Action was implemented in four locations. These were Mogadishu (Banadir), Lower Shabelle, Bay and Bakool (SWS).
The evaluation shall investigate into the following:
The use of the five OECD DAC evaluation criteria of Relevance, Effectiveness, Efficiency, Impact and Sustainability (see further elaboration below)
Cross-cutting issues will include gender, youth, adherence to FCA’s Rights Based Approach and Do No Harm Principle as well as Core Humanitarian Standards.
The intervention logic (e.g. Log frame) is to be analyzed to ascertain if all the indicators and targets were achieved.
Evaluation against FCAs Somalia Results Framework and Country Strategy.
To assess the coordination between FCA and partners
To identify and document good practices and key lessons learnt and make recommendations for improvement on areas that require such action, based on evaluation findings.
4. EVALUATION QUESTIONS
Relevance
How relevant was the action to the local and national priorities of Somalia including the target beneficiaries? To what extent was the project in line with FCA Country Strategy and FCA Global Program 2015-2017?
Assess the approach used in developing the project (was there effective involvement of the implementing partners? What benefits accrued for or were foregone from the approach adopted? What implications has this had on the effective launch and implementation of the project? What should be done differently in future?)
How has the programme adapted to the changing governance needs and conditions in Somalia?
Was the actual design of the project the best way to structure the action? Was the design context responsive? What should have been done differently to make it more effective? Suggest a better model moving forward
Effectiveness
Were the objectives and the purpose of the action achieved? If yes, to what extent has the expected outcomes and results been achieved against the set indicators? If NOT, what factors undermined the achievement of the project objectives? What should be done in future to avoid such shortfalls?
To what extent did the Action contribute to sustainable capacity building, knowledge transfer as well as to the improvement of the public participation in the state building agendas?
How effective has the coordination and linkages with the implementing partners and other non-partner actors been to date?
Lessons around FCA partnership with the local NGOs (IIDA & CRD), bring out key issues/challenges that marred the partnership, how they were addressed, how effective the strategy for redressing the issues was and a suggest better/more effective approach of avoiding and or addressing similar issues in future
Efficiency
Did the project implementation follow the project timeframes? If not what factors led to the change in the timelines? Were the required deliverables (activity funds, scheduled activities, expected outputs, feedback etc) provided in time to facilitate the action within an appropriate time period?
How cost effective was the action (planned versus actual)? Is the relationship between input of resources and results achieved appropriate and justifiable? What is the cost-benefit ratio?
The process of implementation (did the Action establish effective collaboration arrangements? What were the pros and cons with some of the collaboration arrangements that the implementing agencies entered into? What should we have done differently?)
Establish the level of accountability for the project resources and preconditions by the project partners (IIDA&CRD). Did the partners allocate enough resources (staffs, time, equipment, management resources etc) to the project implementation? Did the partners stick/observe the contractual obligations as spelt out in the various contract documents? What structures did the partners put in place to ensure compliance with the donor requirements in procurement, financial accountability to the Action Coordinator (FCA), reporting etc?
Impact
What were the (concrete) contributions of the action in achieving the state building goals of Somalia?
What are the notable milestones or achievements so far and the likely ones in the long term attributed to the project)? What’s the total reach of the action to date – disaggregated by sex
or other relevant classification such as minorities, youth, women? etc
What difference hast the action made to the beneficiaries’ knowledge, awareness and skills in advocacy and engagement with the state building processes?
What are the exceptional experiences that should be highlighted e.g. case- studies, stories, best practice?
Sustainability
To what extent are the positive changes and effects of the action (summarily) sustainable beyond EU funding?
Is the project supported by the government and local community leaders? To what extent is the buy-in so far?
Have clear exit strategies for the project been put up by FCA and the partners?
5. EVALUATION METHODOLOGY
This evaluation is expected to apply mixed methods approach. These will involve reviewing the relevant project documents and relevant literature, meetings and interviews with a broad range of stakeholders including donor, project partners, local government representatives, the beneficiaries and community opinion leaders among others. The evaluator(s) will be responsible for assessing the project and management issues, briefing and debriefing with key stakeholders and production of the evaluation report. Other methods including use of focused group discussions (FGDs), observations, documentation of the Most Significant Change stories are also highly recommended.
The evaluator/s will be required to carry out the exercise diligently, which will include;
o Provide a detailed elaborate methodology and work plan, which will be shared and agreed upon with FCA before the start of the actual work.
o Participating in the initial briefings with FCA and partner teams to ensure that the evaluation team is clear on the expectations of this evaluation.
o Developing the evaluation inception report highlighting the methodology to be used and the relevant tools which will be submitted to FCA and will be discussed and approved by FCA in consultation with the partners before commencement of the field work.
o Reviewing the project materials and other relevant literature in order to have adequate understanding of the project and maximize incremental learning.
o Discussion with FCA, the implementing partners and other stakeholders including relevant government officials, CSOs, Community Advocates, youths, women and representatives of the marginalized groups, and opinion leaders to get their feedback to reach conclusions against benchmarks to date.
o Conducting field visits to collect data using the agreed upon methods and incorporate gender and adequate representation of the beneficiaries as well as strategic non-beneficiaries in the evaluation.
o Ensuring that the data is collected stored and analyzed professionally.
o Presenting to FCA and partners the key findings and giving the team adequate opportunities to give feedback and agree on action points, lessons and recommendations.
o Submitting draft report to FCA and partners for review, comments and inputs which will be taken into consideration before submission of the final report.
o Document and submit at least four (4) quality case stories focusing on best practices and lessons learnt as evidence from the project locations. In doing this, a good balance of the target beneficiaries and other stakeholders shall be maintained.
5.2 SAMPLE SIZE:
The consultant(s) will determine the appropriate sample size in consultation with FCA and partners, taking into consideration the activities carried out so far, the target project locations and the stakeholders. The consultant(s) will visit all the sites and interview the beneficiaries and stakeholders.
6. EVALUATION PROCESS
The evaluation process comprises of three main phases. These include;
Phase I (Inception Phase): This is the preparatory phase. FCA places a lot of emphasis on this phase to ensure proper grounding of the evaluation exercise. Key activities will include but not limited to the following;
a) Internal consultations with the selected evaluators on the evaluation process. A number of sessions may be held depending on the need
b) Literature review of the project documents
c) Submission & approval of an Inception Report reflecting the reviewed literature, the evaluation methodology including the evaluation framework, detailed evaluation work plan etc.
d) Development of the data collection tools and their review and approval
e) Planning for the logistics (evaluator/s generate logistics needs for discussion with FCA)
Phase II: Field Work: This phase will involve the actual field work. Key activities will include the following;
a) Evaluators security clearance to travel to the field by FCA
b) Actual field data collection
Phase III: Data Analysis and Reporting
This is the final phase whose activities will include the following;
a) Debrief with FCA and partners upon return from the field on preliminary findings
b) Data organization, cleaning, interpretation and analysis
c) Compilation of the report
d) Data validation workshop
e) Finalization of the report and submission to FCA
Please Note: Activities in every phase must be completed before entry to the next phase.
7. EVALUATION LOGISTICS AND COORDINATION
Partners will coordinate all the logistics on behalf of the evaluator as follows;
FCA (Lead Agency)
1) Security clearance to travel to the field and while in the field
2) Support with acquisition of Travel Visa to Somalia (if needed)
3) Support with booking travel flights to the field and to the various project locations (if needed)
4) Support with booking for accommodation while in the field (if needed)
5) Provide the project documents for literature review
6) Organize all the sessions with the evaluator/s in Nairobi
IIDA Women’s Organization (IIDA)
1) Organize meetings & consultations with all the relevant project stakeholders in Mogadishu and Lower Shabelle (Government, media, CSOs, Others relevant to the Action implemented)
2) Mobilize and convene the different project beneficiaries for FGDs in Mogadishu & Lower Shabelle
3) Identify and prepare two (2) suitable beneficiaries/stakeholders (01 in Mogadishu & 01 in L/Shabelle) for recording of the Most Significant Change Stories (Evaluator to provide a selection selection criteria)
4) Support to organize all the local travel arrangements in the two locations in consultation with FCA & the consultant/s.
5) Support to organize for accommodation for the evaluator/s in the two locations in consultation with FCA and the evaluators
6) Avail any communication and media materials (you may have developed) to the evaluator for review
7) Avail relevant project team members for interviews with the evaluators
8) Support to organize for translator for the evaluation while in your location (if need be)
9) Provide office space and working desk to the evaluator while in your location
Centre for Research & Dialogue (CRD)
1) Organize meetings & consultations with all the relevant project stakeholders in Bay and Bakool (Government, media, CSOs, Others relevant to the Action implemented)
2) Mobilize and convene the different project beneficiaries for FGDs in Bay and Bakool
3) Identify and prepare two (2) suitable beneficiaries/stakeholders (01 in Bay and 01 in Bakool) for recording of the Most Significant Change Stories (Evaluator to provide a selection selection criteria)
4) Support to organize all the local travel arrangements in the two locations in consultation with FCA and the consultant/s.
5) Support to organize for accommodation for the evaluator/s in the two locations in consultation with FCA and the consultant/s.
6) Avail any communication and media materials (you may have developed) to the evaluator for review
7) Avail relevant project team members for interviews with the evaluators
8) Support to organize for translator for the evaluation while in your location (if need be)
9) Provide office space and working desk to the evaluator while in your location
8. EVALUATION TIMETABLE
The maximum timeframe for this assignment is 30 Days that include preparation, field work, data analysis and report writing. The process starts on 15th May 2018. The consultant/s is required to submit a detailed work plan on how the task will be rolled out.
9. THE EVALUATION REPORT
Production of the report will be the responsibility of the evaluator(s) while FCA and and the partners will be responsible for coordinating the exercise. The report shall be:
Ø Produced in English language and should be simple in expression (jargon free).
Ø Maximum of 40 pages. Short Annexes to be attached separately
Ø The report format and text, should be on A4 paper size in Arial font size 11.
Ø The evaluator(s) will deliver an electronic version of the evaluation report in Word
ØVersion by the agreed deadline and Four (4) hard copies of the report in full color (where applicable)
9.1 OUTLINE OF THE EVALUATION REPORT
The evaluation report should contain at MINIMUM different elements mentioned below. All parts should be clearly distinguished from each other and of sufficient quality.
a) Cover page
b) Acknowledgement
c) Table of contents
d) List of tables
e) List of figures
f) Executive summary comprehensive enough to highlight all the key aspects of the evaluation. The summary should present a highlight of the methodology, key findings, summary of conclusions and recommendations.
g) Introduction section
h) Literature review (should be precise and well linked to the Action’s focus)
i) Methodology that clearly outlines the sampling procedures and the sample size, how the data was collected, analyzed and presented with clear justification and limitations if any.
j) Findings addressing all the evaluation questions and the unexpected outcomes.
k) Conclusions presenting summary of the findings on the basis of the questions.
l) Lessons learned and recommendations which should be clearly related to the conclusions.
m) Case stories highlighting best practices.
n) Report annexes that include the tools used in the evaluation and the list of respondents.
10. DELIVERABLES
1) Planning meetings
2) Inception report
3) Evaluation tools
4) Zero draft evaluation report for internal review
5) Validation workshop in Nairobi
6) Final evaluation report addressing all objectives, evaluation questions and including clear recommendations (Soft copy & 4 Hard Copies in full color to FCA)
7) Debriefing on the final findings
8) Four case stories
11. SKILLS AND QUALIFICATIONS REQUIRED
The desired specification and qualities of the consultant(s) are as hereunder:
o Post Graduate qualification in Governance, Programmes Management, Political Science, Development Studies or Economics or any other relevant Social Science Degree from a recognizable university.
o Minimum of 5 years’ experience in evaluation work with proven track record
o Skills and experience in evaluating governance and socio-economic projects with in-depth knowledge highly desirable.
o Strong interpersonal and analytical skills.
o Familiarity with the Somalia political, cultural and socio-economic context a must.
o Demonstrable knowledge of Somalia security context, policy environment including human rights, State Building processes, the Provisional Constitution provisions, business environment, public participation, the clan dynamics and resultant segmentation indexes; and the law making processes.
o Experience of effective interaction with local and national organizations, government institutions and marginalized communities in Somalia context.
o Conversant with application of cross cutting themes like gender mainstreaming and gender and competency in advocacy; Rights Based Approach and Do No Harm Principle.
o Good spoken and written communication skills in English. Somali language proficiency is a must. A female team member is highly recommended for purposes of conducting FGDs with women beneficiaries at the community level.
o Proven experience of using participatory methods for data collection and analysis in programme evaluation.
o Ability to show case high quality products of similar tasks in similar contexts and elsewhere
12. BUDGET (IN $) & PAYMENT MODALITIES
Quoted amount should not exceed EUR8,500 professional fee. The consultant will then be required to invoice FCA for costs incurred in accommodation in the field, air travels, local transport, meals, enumerators fee, airport taxes etc. The consultant also bears the tax liability for individual taxes hence the budget should not include VAT.
Payment Modalities
Deliverables
Payment %
Upon approval of the Inception report
30%
Upon Approval of final report
70%
13. BIDS ASSESSMENT
Below is an indicative criterion for bids assessment
CRITERIA
%
Quality of the document/bid
20
Skills / expertise of the evaluation team
35
Methodology for structuring, data collection & analysis
20
Organization of tasks and timetable
10
Price factor: cheapest tender complimented by quality of bid
15
14. TERMS OF CONTRACT
· All documents and the data collected will be the property of FCA and will be treated as confidential and used solely to facilitate analysis. Where necessary, the respondents will not be quoted in the reports without their permission.
· The evaluator/s shall observe a high level of confidentiality of all the information divulged to them as relating to the contractor (FCA) and the partner’s operations in Somalia. No part of the information disclosed should be shared externally without prior approval/consultations with FCA. FCA remains the sole owner of the information generated through this task and will hold such information in trust for all the involved partners in this task.
· The Consultant is responsible for payment of all social costs, other employment related costs and insurance contributions and for all other liabilities of a statutory nature.
· The consultant will have to abide by FCA Conde of Conduct (COC), Child Safeguarding policy; its Security Protocols and any other relevant policies supplied to them verbally or in writing.
· The Consultant shall also adhere to the security advice and protocols spelt out by the partner agencies while working with them in their respective locations. While security is relative, the duty to comply with the laid out procedures lies solely on the individual (contractee).
· The consultant will provide a debriefing session in Nairobi to present the main findings and recommendations
· Copyright for the report will remain with FCA.
15. SUBMISSION PROCESS
Consultant(s) who meet the above requirements should submit bids, which at minimum include the following:
o Updated Curriculum Vitae that clearly spells out relevant qualifications and experience.
o Suitability statement including a short summary of relevant competences and previous evaluation work conducted; and commitment for availability for the entire assignment from 15th May 2018
· Detailed technical proposal outlining your interpretation of the TOR, proposed methodology, sample size and sampling procedure, detailed work plan and the evaluation process among others (using the FCA template)
o Detailed financial proposal, including daily costs.
o Contacts of three organizations that have recently contracted the consultant to carry out relevant evaluation.
o Budget excluding VAT
o Up to 3 examples of recent similar consultancy (from recent evaluation work)
o Up to 3 recommendations from previous contract providers
Please provide all the above documentation in a single consolidated document.
16. REPORTING LINES
The consultant shall work under the supervision of the FCA Project Manager based in Nairobi, who will interphase with the Partner Project Managers in the respective project locations on the ground. A strong security management link shall be maintained with the FCA Security Manager based in Mogadishu while in the field, details of which shall be supplied to the successful applicant.
How to apply:
. APPLICATION PROCESS
Interested and qualified consultants should send their applications before or by 26th April 2018 at 5.00 P.M. (E.A.T). Please Indicate APPLICATION FOR 11699 PROJECT EVALUATION on the subject line to the email address below
Late, incomplete or partial bids will be rejected.
For any clarifications on the proposed task please send an email to the following address: ruth.wangari223@gmail.com with a copy to Jana.Schroder@kua.fi
Annexes
Annex 1: Outline of the technical proposal template
Annexes below will be provided to the successful bidder upon signing of the contract
Annex 2: Outline of the inception report template
Annex 3: Outline of the evaluation report template