TERMS OF REFERENCE: MONITORING AND EVALUTION OF A REGIONAL MULTIMEDIA CAMPAIGN ON EUP

 

TERMS OF REFERENCE

MONITORING AND EVALUTION OF A REGIONAL MULTIMEDIA CAMPAIGN ON EUP

Background

Early and Unintended Pregnancy (EUP) is a global public health concern, with particularly high prevalence rates in Sub-Saharan Africa.  For many adolescents, pregnancy and childbirth are neither planned nor wanted. EUP is driven by multiple factors, including poverty, little or no access to school, information or health care, cultural norms, peer pressure, and sexual coercion and abuse. And although EUP is a problem that primarily affects adolescent girls by negatively impacting their health, social, and economic outcomes it also affects some men and boys, for example by forcing them to drop out of school to support a family.

The ESA region has one of the highest adolescent fertility rates in the world, at 102 per 1,000 live births.[1]  Many of these early pregnancies are not planned, as evidenced by the high rate of unintended pregnancy in Africa at 89 per 1,000 overall and 112 per 1,000 in Eastern Africa, resulting in an estimated 21.6 million unintended pregnancies per year.[2]  Adolescent girls continue to experience the disproportionately high burden of sexual and reproductive ill health, particularly in Sub-Saharan Africa.[3]  Adolescents are likely to have complications during pregnancy including unsafe abortion and more likely to become young mothers a second time.[4] After AIDS, the second highest cause of death for adolescent girls is related to pregnancy.[5] Their babies also face a substantially higher risk of dying than those born to women aged 20 to 24, and are at greater risk of malnutrition, poor mental and physical development, and low educational attainment.[6]

A situation analysis on EUP in 10 countries in ESA was commissioned by UNESCO to assess the magnitude of the problem in the region.[7]  The study revealed that EUP in ESA is very high with at least 15% of girls 15 to 19 years old having ever been pregnant.  Some countries report much higher rates, including Tanzania (59%) and Kenya (39%).  In several countries, teenage pregnancy rates have increased during the last decade.  For example, in Malawi, pregnancy rates have increased from 26% in 2010 to 29% in 2016; in Namibia, rates rose from 15% in 2006 to 19% in 2013; and in Uganda, rates increased from 23.8% in 2011 to 24.8% in 2016.[8]  A majority of adolescents affected by EUP are girls living in rural areas with low educational attainment, thus increasing the risk to perpetuate a cycle of poverty.

An important challenge associated with adolescent pregnancy is unsafe abortion, which can lead to severe complications and death.[9]  A high proportion of women seeking post-abortion services in ESA healthcare facilities are below age 20 (17% in Kenya, 21% in Malawi, up to 58% in Tanzania, 60% in Zambia, and 68% in Uganda).[10] Access to safe abortion is very limited in most countries in the region, even in those countries where abortion is decriminalized.

Negative Consequences of EUP

The key negative impacts of EUP include poor physical, emotional, educational, and economic outcomes.  EUP may lead to such physically harmful outcomes as maternal mortality or morbidity (including obstetric fistula), HIV and other sexually transmitted infections (STIs), gender-based violence (GBV) and poor health outcomes for infants born to adolescent mothers.  AIDS is the leading cause of death for adolescent girls in the region.[11] 

Adolescent girls who experience EUP also generally suffer emotional impacts, especially if they have been rejected by their partner, family, and/or community. Stigma and victimization are highly associated with EUP for many girls who tend to feel shame, lose confidence, and withdraw from society.  Long-term risks include greater vulnerability to violence and abuse, depression and substance abuse, and economic hardship.  Throughout the region, pregnant girls are stigmatized, blamed and shunned without due consideration for the context or structural factors that increase their vulnerability to unintentionally becoming pregnant.

A majority of pregnant adolescents in the ESA region do not complete school and/or have poor school performance, often leading to poor job prospects and a lifetime of poverty.  Less than five percent of girls that drop out of school because of pregnancy are able to return to school once they have given birth.  In Zambia, 95% of young girls ever pregnant did not return to school; in Malawi, Kenya and Uganda, and in Tanzania, this figure was 97%, 98% and 99% respectively.[12]  A key reason that girls did not return to school was because parents and communities were not aware of policies that allowed girls to re-enter schools following pregnancy, and/or school officials that used their own personal discretion, authority and values to prevent re-entry. However, only about half of ESA countries have legislation and policies on how to manage learner pregnancy and re-entry after delivery.[13]  Even where re-entry is an option, girls are still vulnerable to drop-out since the school environment is often hostile and unsupportive.[14] 

Reducing EUP through the ESA Commitment

The ESA Ministerial Commitment[15] that was endorsed and affirmed in December 2013 by the Ministers of Education and Health in ESA, commits governments to increase access comprehensive sexuality education and sexual and reproductive health services for young people provides. The commitment provides an important foundation for addressing early and unintended pregnancy through a multi-sectoral approach. The ESA commitment target of reducing EUP by 75% by 2020 will only be attained if there is a concerted multi-sectoral effort at country level to address EUP. Tackling EUP will contribute to the achievement of some of the Sustainable Development Goals 1, 3, 4,5,10 and 16.

In May 2017, representatives from ESA governments, UN, and civil society partners met during an expanded Technical Coordinating Group (TCG) meeting to discuss progress on the implementation of the ESA Ministerial Commitment for access to comprehensive sexuality education and youth-friendly SRH services (or the ESA Commitment). Among the most pressing issues highlighted at the meeting, was the high levels of EUP in the ESA region. Nearly every government represented raised this challenge and called on the TCG to develop a regional and localised intervention to address EUP as it was agreed that EUP is an area requiring intervention across all ESA Commitment countries.

There was a clear recommendation to implement a Regional EUP campaign across the region with the aim of raising awareness on the consequences of EUP and reducing stigma towards pregnant girls; improving delivery of comprehensive sexuality education; promoting consistent condom use for sexually active young people; and increasing access and use of effective contraception.  

Based on the recommendation, UNESCO collaborated with UNFPA, the Southern Africa HIV and AIDS Information Dissemination Service (SAfAIDS), Save the Children, SADC Parliamentary Forum, Internews, and One Africa, to develop a multimedia campaign on EUP. The campaign takes place within the context of the ESA Ministerial Commitment initiative, whose overall goal is to ensure that adolescents and young people in SSA are empowered, educated, healthy and resilient and have the capacity to reach their full potential and contribute to the development of their community, country and region. The campaign will be implemented in 21 countries across ESA and run until December 2020.

 

Objectives of the Multimedia Campaign

Overall objective: To reduce and prevent early and unintended pregnancy (EUP) and afford girls who become pregnant an opportunity to continue with their education.

 

Specific Objectives:

  1. To advocate for the development and operationalisation of EUP prevention, management and re-entry policies to facilitate the right of girls to complete education
  2. To advocate for the integration of content on “access to contraceptives, gender equality, and power dynamics within relationships” in learning material to develop the learners’ knowledge and skills to prevent pregnancy.

iii. To promote adolescents’ and young peoples’ access to health education and services (incl. contraceptives) by advocating for the establishment of a referral system between schools and health facilities

  1. To challenge school-related gender based violence and engage boys and young men in learning about and practicing pregnancy prevention.
  2. To address cultural norms that put girls at risk of EUP and promote parent-child communication about sexual health.

 

Purpose

UNESCO, in collaboration with UNFPA, SAfAIDS, and Save the children Sweden is seeking the services of a contractor to conduct routine monitoring; mid-term and endline evaluations for a regional multimedia campaign on Early and Unintended Pregnancy (EUP) in East and Southern Africa.

Scope of Work

Guided by the M&E framework developed for the EUP campaign, the contractor is expected to implement monitoring and evaluation activities as follows:

 

Monitoring

Monitoring of activities will be conducted throughout the lifespan of the campaign. Due to the wide scope of the campaign, a sample of at least five (5) countries out of the 21 countries that are implementing will be expected to participate in the monitoring. Convenience/judgemental selective sampling will be used to select these five, with considerations for the category to which the country belongs i.e. focus or supporting; geographic area; and language, amongst other criteria. The final selection of countries will be agreed upon by all partners implementing the campaign.

 

The contractor will be expected to establish an operational monitoring system for the campaign, including development of methods and tools for data collection to be used by both the campaign implementers and the contractor, support data collection in all countries, and conduct data collection as referenced above.

 

The contractor will be expected to produce quarterly monitoring reports which will include but not limited to data and information on: campaign fidelity; how well the campaign components have been carried out; if the targeted audiences are being reached; how they are being reached, and if campaign objectives are being met overall.  Routine monitoring will also include measurements of process outcomes and outputs in relation to the theory of change and the EUP campaign M&E framework.

 

Mid-term evaluation

A mid-term evaluation will be conducted in the first quarter of 2020 with a primary focus of assessing the campaign fidelity, audience reception of the campaign, if campaign objectives are being met, and lessons, as it might be too premature to detect significant changes in intended outcomes of the campaign (i.e. changes in knowledge, attitudes and behaviors). The mid-term evaluation will be used to adjust/change the focus of the campaign to overall improve implementation and results for the second half of the campaign. Key questions which will guide the mid-term evaluation include, but are not limited to:

  • What is the target audience’s overall impression of the campaign?
  • How is the campaign being implemented? Is it being implemented with fidelity?
  • What are the major or critical factors influencing achievement or non-achievement of the campaign objectives?
  • What are the perceptions of primary and secondary stakeholders on communication materials and campaign interventions?
  • What are the target audience’s suggestions for improving the campaign?
  • What lessons have emerged, which will inform the direction of the campaign?

Endline Evaluation

A mixed methods endline evaluation will be conducted towards the end of the campaign (December 2020) with a variety of aims, including: determining impact; assessing the effects of the campaign on knowledge, behaviours/practices, beliefs, norms, attitudes and self-efficacy; and community and policy support for preventing EUP and optimizing adolescent SRH in the ESA region. The evaluation will aim to assess the impact of the intervention on the target audience, including self-reported impact.

 

The key stakeholders of the evaluation are UNESCO; UNFPA, SAFAIDS, Save the Children Sweden, SADC PF, policy makers; community, religious and traditional leaders; health service providers; teachers, educators and school principals; and adolescents and young people (AYP). The contractor will be expected to measure progress on all indicators for the endline evaluation. Key questions which will guide the endline evaluation include, but are not limited to the following:

 

Effectiveness

  • To what extent did the campaign achieve its objectives?
  • Did the campaign result in intended knowledge, attitude and behaviour changes?
  • What were the major or critical factors influencing achievement or non-achievement of campaign objectives?
  • To what extent did the campaign’s monitoring, evaluation, and learning mechanisms contribute to achievement of objectives?

 

Relevance

  • What approaches did the campaign implement to address Early and Unintended pregnancy amongst adolescent girls and young women?
  • Were these approaches (the campaign inputs and strategies) realistic, appropriate and adequate to achieve planned results
  • Were the campaign objectives relevant to the identified needs of adolescents and young people, and other stakeholders?
  • To what extent were gender considerations integrated into implementation of the campaign?

 

Efficiency

  • How did campaign implementation differ across countries? How did implementation evolve over time?
  • What were the strengths and weaknesses of the campaign interventions in each country?
  • To what extent were the outputs achieved in a timely manner?
  • What partnerships were established as a result of the campaign? To what extent did these partnerships contribute to achieved results?
  • To what extent were financial and human resources used efficiently? What measures were taken to ensure that resources were efficiently used?

 

Sustainability

  • To what extent are the benefits of the campaign likely to be sustained after the completion of the campaign, (post 2020)?
  • What key factors will require attention in order to improve prospects of sustainability of outcomes?

 

Impact

  • To what extent did the campaign create an enabling environment for the development and operationalisation of EUP prevention, management and re-entry policies?

 

Lessons Learned

  • How could campaign interventions be improved or strengthened?
  • What lessons were learned during implementation that can inform improvements and/or future campaigns?

 

Methodology

 

Sampling

At least five (5) countries, out of the 21 are expected to participate in the monitoring, mid-term and endline evaluations. Purposive sampling will be used to select the five, which are expected to be representative of focus countries, supporting countries, geographical divide, and language, amongst other criteria. The final selection of countries will be agreed upon by all partners implementing the campaign.

 

Proposed Methods of Data Collection

The Social and Behaviour Change Communication (SBCC) strategy developed for the campaign will serve as the entry point for defining the methodology. The methodology should respond to the scope and overall framing of the campaign and its suggested outcomes and activities in relation to the theory of change (TOC).

The endline evaluation will use a mixed methods approach. The contractor will conduct qualitative data collection in five countries, which will be selected as a sample out of the 21 countries implementing the campaign. The methodology will include, but not limited to:

 

Desk review: This will focus on review of publicly available data sources and copies of other relevant documents such as country specific campaign reports.

 

Secondary Data Analysis: The contractor will conduct secondary data analysis from existing reports and documents. The data sources and documents to be reviewed for data extraction and analysis will differ from country to country. For indicators collected from population based surveys (i.e., DHS), the contractor will use DHS STATcompiler as a first resort and include data available from the most recent survey conducted in each country. The analysis is also expected to generate retrospective baseline data for some of the indicators.

 

Key Informant Interviews (KIIs): These will be conducted with partners and stakeholders, and samples from the different audiences targeted by the campaign, in order to acquire a perspective on the implementation of the campaign. Small group discussions amongst stakeholders may also be used in place of individual interviews. The stakeholders and audience samples are expected to include policy makers; teachers, educators and school principals; parents and guardians; community, religious and traditional leaders; adolescent girls and young women; adolescent boys and young men; and health services providers.

 

Observations: These will also be utilized during field visits, focusing on select stakeholders.

 

Social Media Diagnostics: This will be implemented in order to track output data from digital channels employed by the campaign. This is expected to also include a sentiment analysis and overall perceptions from audience groups to the campaign messages and materials shared online

 

Surveys: These will be used for collecting quantitative data from samples of the different audiences targeted by the campaign.

 

The contractor will be expected to develop a rigorous and systematic methodology to ensure that findings from the evaluation are inferable. Special attention should be paid to sample selection, unit of measurement, and weighting if possible. The methodology will be further elaborated and detailed by the contractor in consultation with UNESCO, UNFPA, SAfAIDS and Save the Children Sweden as part of the inception phase.

Indicative Deliverables and Time Schedule

 

All deliverables will be produced in English language. The contractor will be expected to also produce deliverables in Portuguese language in the case that countries such as Angola and Mozambique are selected to be part of the sample for data collection.

 

Monitoring

 

Monitoring will be conducted throughout the lifespan of the campaign. The deliverables will be as follows for the quarterly monitoring activities:

 

  • Inception Report detailing scope of work and timelines
  • Monitoring matrix and data collection tools
  • Monitoring report with indicator data, findings, emerging lessons and recommendations
  • Timeframe: 3 quarterly monitoring reports produced in January 2020, July 2020 and December 2020 respectively

 

Midterm & Endline evaluations

 

  • Inception Report: The contractor will submit an inception report which details the key scope of the work; a work plan/schedule of tasks designating a team member with the lead responsibility for each task and deliverable (output); evaluation questions; proposed methods showing how each evaluation question will be answered; sources of data; data collection procedures; data collection plan (list of people to be interviewed, their designation and when the interview will be conducted) and tools; and a data analysis plan. The inception report will detail the contractor’s understating of what is being evaluated and why. This will provide the evaluator an opportunity to confirm that they share the same understanding of the evaluation objectives and procedures.
  • A draft mid-term and endline evaluation report will be presented to UNESCO and partners, by the contractor during a stakeholder workshop at a date and venue to be agreed. The contractor will therefore facilitate the workshop which will be aimed at discussing preliminary findings and conclusions of the mid-term and endline evaluations.
  • A final mid-term and endline evaluation report which should present the findings, recommendations and lessons learned covering the scope of the evaluation. The report should follow, but not be limited to the following structure:
  • Executive Summary (usually not more than 3–4 pages, which can also be used independently)
  • Programme description
  • Evaluation purpose and objectives
  • Evaluation methodology
  • Findings
  • Lessons learned
  • Recommendations
  • Annexes (including the list of stakeholders consulted during the evaluation, key documents reviewed, Evaluation Terms of Reference, data collection tools etc.)
  • A PowerPoint presentation summarizing the report.

 

It is expected that the mid-term and endline evaluations will take an estimated duration of 31 and 58 working days respectively. This will include desk reviews, fieldwork, and report writing as illustrated in the table below:

 

Description of Activity

Deliverable

Timeframe

Mid-term Evaluation

Endline Evaluation

Desk review of relevant documents

Inception Report

5 working days,

March 2020

 

10 working days,

September 2020

Development of data collection tools; qualitative & quantitative data collection; and analysis of both existing and new datasets

Data collection tools

10 working days,

March 2020

25 working days,

September - October 2020

Compilation of draft report

Draft report with relevant annexes

10 working days,

March - April 2020

15 working days. UNESCO and partners will have a maximum of 7 working days to submit comments,

 

October - November 2020

Validation of findings (workshop)

Validated report

1 working day,

April 2020

1working day,

November 2020

Revision of report to incorporate findings from validation workshop

Final report with all annexes

5 working days,

April 2020

7 working days,

November 2020

 

 

 

 

 

 

Conditions

 

PCI, UNESCO, UNFPA, SAfAIDS, and Save the children Sweden will be responsible for managing the evaluation. UNESCO, UNFPA, SAfAIDS, and Save the children Sweden will be responsible for providing stakeholder lists and contact information, as well as setting up appointments with the stakeholders for data collection.

 

The contractor will be responsible for own logistical requirements such as office space, administrative and secretarial support, telecommunications, printing of documentation, and travel and accommodation arrangements.

 

The contractor will be paid as follows for each quarterly monitoring:

  • 20% on submission of inception report detailing scope of work, work plan/timelines
  • 20% on submission of monitoring matrix and data collection tools
  • 60% on successful completion of activities and delivery of monitoring report with indicator data

 

The contractor will be paid by as follows for the mid-term and endline evaluations:

  • 20% after successful submission of the inception report to UNESCO and partners
  • 20% on submission of the data collection tools
  • 20% on submission of draft report
  • 40% on successful completion of the activities and delivery of the final validated evaluation report.

 

Qualifications and Requirements

 

The contractor shall have the following expertise and qualifications:

 

Expertise of the firm/institution

  • Mandatory to demonstrate at minimum five years’ experience in programme/project monitoring and evaluations in the field of social sciences at national and international levels
  • Experience working with UNESCO or other UN agencies is desirable
  • Mandatory to have a track record in designing and conducting/leading evaluations;
  • Mandatory to demonstrate regional experience
  • Preferable to have track record in working with the education, social or health sectors.

 

Personnel - individual candidates designated to undertake the assignment

  • The Technical Lead from the consulting firm must have an advanced university degree in one of the social sciences, or public health, with extensive knowledge in qualitative and quantitative research, and analytical skills. Other team members must have at least BA degree in one of the social sciences, or public health, with extensive knowledge in qualitative and quantitative research, and analytical skills.
  • Mandatory to have at least the leading candidate with minimum 5 years of experience in monitoring and evaluation of national programmes or projects related to education, social issues or public health – more than 5 years of experience as specified would be an asset
  • All team members responsible for data analysis and report writing must have strong computer skills including Microsoft Office (Word, Excel, PowerPoint), qualitative and statistical analysis software.
  • At least one member of the consulting firm must demonstrate expertise on the subject matter of the campaign , i.e. EUP and ASRH technical issues.
  • Personnel ideally should be multicultural and consider gender balance and geographical representation.
  • Regional languages other than English will be considered a plus.
  • Experience working on sensitive social subjects and SRHR issues a strong plus

 

Submission of Proposals

 

Consulting agencies/consultant should submit a technical and financial proposal.

Technical Proposal

The technical proposal should be no more than 10 pages and should include:

  1. Background information that includes an interpretation and understanding of the terms of reference;
  2. Methodology and approach outlining a clear conceptual and analytical framework for the work to be undergone;
  3. Proposed work plan outlining clear timeframe and logical steps in conducting the assignment;
  4. Expectations for support from UNESCO and partners in terms of logistical, technical, and other forms of support for the work;
  5. Previous experience of the consultant/institution and expertise that will be of benefit to the proposed assignment;
  6. Writing Sample: Include a sample of a recent evaluation report the consulting firm has written.

 

Financial Proposal

The financial proposal should be no more than 4 pages and should include:

  1. An itemized budget divided into professional fees and direct cost of activities in US Dollars.

Timeframe for submitting proposals

The M&E Consultant should be contracted by October 31, 2019. Proposals must be submitted by email to This email address is being protected from spambots. You need JavaScript enabled to view it. and This email address is being protected from spambots. You need JavaScript enabled to view it. by October 18, 2019. Please include in the email subject: Let’s Talk! Campaign M&E Proposal. The results of the selection will be communicated within 10 days after the closing date.

 

[1] UNESCO (Aug 2018). Baseline Study: Our Rights, Our lives, Our future: Making positive sexual and reproductive health and education outcomes a reality for adolescents and young people in Sub-Saharan Africa. Harare, Zimbabwe: UNESCO.

[2] Singh, S, Remez, L, Sedgh, G, Kwok, L, Onda, T (2018). Abortion Worldwide 2017: Uneven progress and unequal access. New York: Guttmacher Institute. 

[3] Phillips SJ, Mbizvo MT (2016).  Empowering adolescent girls in Sub-Saharan Africa to prevent unintended pregnancy and HIV: A critical research gap.  Int J Gynaecol Obstet, 132(1):1-3.

[4] World Health Organization (2014).  Adolescent pregnancy factsheet.  WHO:

[5] UNSECO (2016).  Fulfilling our promise to young people today: 2013-2015 progress review.  Paris:  UNESCO, UNAIDS, UNFPA.

[6] World Health Organization (2014).  Adolescent pregnancy factsheet

[7] UNESCO (February 2018).  Situation analysis on early and unintended pregnancy in Eastern and Southern Africa.  Paris, France.

[8] UNESCO (February 2018).  Situation analysis on early and unintended pregnancy in Eastern and Southern Africa.  Paris, France.

[9] Birungi H,  Chi-Chi Undie, IM, Katahoire, A, Obare, F, Machawira, P (2015). Education Sector Response to Early and Unintended Pregnancy: A Review of Country Experiences in Sub-Saharan Africa. STEP UP and UNESCO Research Report.

[10] Shah, I, Ahman E (2004).  Age patterns of unsafe abortion in developing regions.  Reproductive Health Matters, 12(suppl. 24): 9-17.

[11] UNESCO (2016).  Fulfilling our promise to young people today: 2013-2015 progress review.  Paris:  UNESCO, UNAIDS, UNFPA.

[12] Birungi H,  Chi-Chi Undie, IM, Katahoire, A, Obare, F, Machawira, P (2015). Education Sector Response to Early and Unintended Pregnancy: A Review of Country Experiences in Sub-Saharan Africa. STEP UP and UNESCO Research Report.

[13] Tanzania has the strictest policy of the ESA countries, requiring expulsion of the pregnant adolescent; the following countries have conditional re-entry policies: Botswana, Burundi, Kenya, Madagascar, Malawi, Mozambique, Namibia, South Africa, Swaziland, Zambia and Zimbabwe.

[14] UNFPA (2016).  Harmonization of the legal environment of adolescent sexual and reproductive health in the East and Southern Africa Region (includes 21 ESA Commitment countries, Comoros and Eritrea).

[15]The ESA Commitment was affirmed by 20 countries in Eastern and Sothern Africa.