This evaluation of the PEPFAR-funded Força à Comunidade e Crianças (FCC) program was a randomized controlled trial (RCT) designed to identify and measure the program’s direct and indirect impacts on households with orphans and vulnerable children (OVCs) in Mozambique. Outcomes of interest included health care utilization, health outcomes, education outcomes, household economic conditions, the ability to cope with shocks and the extent to which any program impacts spilled over to households not directly enrolled.

The results show that the FCC program in Mozambique had no positive impacts for any of its intermediate results and no observable impacts in nearly all outcomes of interest. The evaluation did find that the FCC program had negative outcomes in the form of reduced rates of HIV testing, reduced HIV-related knowledge, and an increase in stigmatizing attitudes about HIV. The evidence from this evaluation suggests that FCC and similar multifaceted programs may be ineffective at accomplishing their purpose or may be unintentionally adding to the challenge of responding to HIV/AIDS in Africa.

Evaluation Executive Summary

The President’s Emergency Plan for AIDS Relief (PEPFAR), initiated in 2003, is the U.S. Government’s most important programmatic response to the international HIV/AIDS crisis. PEPFAR mandates part of its funding be devoted to programs benefiting children orphaned or made vulnerable (OVC) by HIV/AIDS. 

The PEPFAR-funded program Força à Comunidade e Crianças (FCC) in Mozambique began in 2017 as a five-year cooperative agreement between USAID and World Education Initiative (WEI)/Bantwana and has since ended. FCC aimed to improve the health, education, and overall well-being of OVCs in selected districts in Sofala, Manica, Zambezia, and Gaza provinces. FCC also provided referrals and treatment follow-up for health clinic visits for HIV testing and anti-retroviral therapy (ART) and supportive related activities.

PEPFAR-funded programs similar to FCC have taken an integrated approach to supporting OVCs. Interventions have targeted children, families, and communities. They have targeted children’s needs at different developmental stages and have been connected to other development programs related to education, nutrition, and household economic development. In fiscal year 2016, PEPFAR OVC programs supported 6.2 million OVCs and their caregivers worldwide.  

The purpose of this USAID-funded evaluation of the FCC program in Mozambique was to identify and measure the direct and indirect impacts of the FCC program on households in Mozambique communities where the programming takes place.

Evaluation Questions

This evaluation measured the impacts of the FCC program in Mozambique across a selection of specified outcomes:

• Health care utilization (health center visits, HIV testing, diagnosis, and treatment)
• Health outcomes (anthropometrics, morbidity, mortality)
• Education outcomes (attendance, performance, grade progression)
• Household economic conditions (investment in agricultural and non-agricultural activities, use of modern agricultural inputs, farm income, non-farm income, consumption per capita)
• If the economic strengthening interventions in the FCC program affect household ability to cope with health and other (e.g., agricultural, weather) types of shocks. If so, determine what financial instruments households use to cope with shocks (credit, savings, asset accumulation and decumulation).
• The extent to which FCC-program impacts spilled over to households who are not directly enrolled in FCC programming.

Methods

We designed and implemented a randomized controlled trial (RCT) to identify and measure these impacts. The RCT methodology is the gold-standard in impact evaluation. In an RCT, the total sample of participating households are selected based on their similarities of average baseline characteristics. They are then randomly assigned either to a group that receives programming (treatment) or a group that does not (control). Any differences in impacts between the two groups can be attributed to the programming itself.

In this FCC program evaluation, we identified a total sample of 3,658 households across the Manica, Sofala, and Zambezia provinces in Mozambique. The sample includes households with at least one OVC member as identified by the survey procedures. 
Randomization for the Treatment Group and Control Group was cluster randomized at the community level. Half of the communities were randomly assigned to receive the FCC program. Communities not selected to receive the FCC program formed the Control Group. 

Within the Treatment communities, we randomly assigned half of households to receive FCC programming (FCC-enrolled). The remaining households (FCC-ambient) did not directly receive FCC programming but the program took place in their communities. This further randomization made it possible to measure any spillovers from households who directly received FCC programming to households who did not.

Findings

For nearly every outcome of interest, we found no statistically significant impacts of the FCC program in Mozambique. This included for nearly all impacts on health care utilization, health outcomes, education outcomes, and household economic conditions. With no detected impacts on these primary outcomes, the evaluation also did not detect any spillover impacts from FCC-enrolled households to FCC-ambient households.

To identify suggestive mechanisms for this lack of impacts, we surveyed the total sample of households about their contact with the FCC program. FCC-enrolled households were 10.7% more likely to have received services from FCC than households in the Control Group, indicating that the FCC program did reach households in treatment communities.

We did identify negative impacts on rates of HIV testing. In addition to self-reported testing (which may be biased), the evaluation measured HIV testing rates with a low-value monetary coupon (50 MZN/$0.78) for directly observable verification and found that based on this coupon-based measure FCC-enrolled households had 10.5% lower rates of HIV testing than households in the Control Group.

Potential underlying causes for these reduced rates of HIV testing were revealed through detailed surveys with all study participants to measure their knowledge and attitudes related to HIV. Among FCC-enrolled households we identified an increase in HIV-related misinformation and stigmatizing attitudes that may have driven the decline in HIV testing.

We designed an additional intervention after completing the primary FCC evaluation that also indicated increased HIV misinformation and stigma associated with FCC enrollment may have caused the reduction in HIV testing. The outcome of interest for these “minitreatments” was directly observed coupon-based HIV testing, which was the only outcome collected after the main FCC program evaluation endline survey. The result showed that only the high-value HIV-testing coupon (double the value of the original coupon) had a statistically significant effect on HIV testing rates. The effect amounts to 7.29 percentage points above the control-group average of 26.3%. 

Conclusions

The results of this evaluation show that the FCC program in Mozambique had no positive impacts for any of its intermediate results. The evidence also suggests that more research is urgently needed to understand the causes that link programming like FCC and potential unintended negative outcomes such as increased stigmatizing attitudes and misinformation about HIV that may play a role in reduced rates of HIV testing. 

These results from the evaluation of the FCC program in Mozambique may be common for similar multifaceted HIV/AIDS programs across Sub-Saharan Africa. The evidence from this evaluation suggests that FCC and similar multifaceted programs are ineffective at accomplishing their purpose and may be unintentionally adding to the challenge of responding to HIV/AIDS in Africa.

 

This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID) cooperative agreement 7200AA19LE00004. The contents are the responsibility of the Feed the Future Innovation Lab for Markets, Risk and Resilience and do not necessarily reflect the views of USAID or the United States Government.