Evaluation highlights opportunities for Fleming Fund to build on strengths

---

The Fleming Fund’s Regional Grants and Fellowship Scheme are expected to make important contributions to the Fund’s overarching goals but, in a second phase of funding, these workstreams could be strengthened to maximise their effectiveness. These are some of the headlines from a new report by the Fund’s independent evaluators, Itad, now available to read on the Fleming Fund website.

As the Fleming Fund’s evaluation partner, Itad’s role is to inform adaptive management of the current phase and support evidence-based decision making by DHSC on whether to fund a new phase and how to design it. Six high-level evaluation questions drive Itad’s work, but in 2020, DHSC asked the evaluation team to focus on understanding the implementation and strengths and weaknesses of three key areas: the Fleming Fellowship Scheme, the Regional Grants portfolio and the prospects for use of AMR surveillance data at country level. The purpose was specifically to identify lessons for implementation of a second phase of the Fleming Fund, but some of the findings are of potential value to other Regional Grants and Fellowships programmes.

What's working well with in the current phase

To reflect on progress of the Regional Grants and Fleming Fellowship Scheme to the end of October 2020, the evaluation team conducted literature reviews on best practice, analysed data from 16 countries and at regional and global level, including documentation from the Management Agent (Mott MacDonald) and interviews with over 200 stakeholders.

From these various data sources, Itad heard examples of how the Regional Grants align with best practice, which could be relevant to other similar programmes. These include giving grantees space to tailor and adapt their approaches to country needs during a clear inception phase. While it is too early to see many results and the COVID-19 pandemic greatly limited grantees’ ability to implement their planned activities, respondents reported that the portfolio is focused on issues that meet countries’ needs.

Turning to the Fellowship Scheme, the evaluators noted how elements of best practice had been incorporated into the design and implementation of the scheme, offering potential learning for similar capacity building interventions - in particular, the provision of flexible support, tailored to individual needs and national priorities in the face of an extremely diverse operating context. The evaluators also found that the selection processes have been effective in recruiting Fellows with appropriate skills, motivation and positions within the AMR space. The involvement of Beneficiary Institutions (the organisations that the Fellows work for) in this process was considered a strength.

Opportunities presented by a potential second phase

In addition to continuing with what is already working well, the report makes suggestions for ways in which the workstreams could be enhanced, which also provide potential lessons for other similar interventions. For example, there is scope to strengthen the articulation of strategic goals and intervention logic – through the development of a specific theory of change for each Regional Grant intervention; the use of evidence on what works can be strengthened and made more explicit; and the focus on monitoring of the Fund’s higher-level achievements can be improved.

Itad’s report noted the challenges associated with the model of working with Fleming Fellows in their place of work (rather than supporting Fellowship placements in partner institutions as is commonly done in other Fellowship schemes). There are pros and cons of this approach, of potential interest to other Fellowship schemes, with one specific challenge identified related to Fellows’ ability to balance their day-to-day work with the expectations of the fellowship (at present worth 40% of the Fellows’ working time) and the risk that Fellowship work is additional to their existing duties. This could be tackled by careful consideration at the workplan design stage, working closely with Beneficiary Institutions to establish manageable workloads and investigating appropriate incentives.

Regional Grants are currently managed separately from Country Grants and Fleming Fellowships but to maximise coherence, efficiency and effectiveness, the evaluators suggested using a second phase of funding to review and revise management of the grants portfolio. More broadly, the report makes several other suggestions for how to strengthen coherence across the workstreams, including producing country-level Theories of Changes, having joint workplanning processes and joint annual reviews between all Fleming Fund grantees and including all relevant country stakeholders in each country, and ensuring that roles and responsibilities around coherence are clear from the start supported with dedicated resources.

An evolving focus on use of AMR data

The Fleming Fund’s emphasis to date has been on strengthening AMR surveillance systems and the generation of AMR surveillance data. It is unsurprising then, that the evaluation found that the Fund currently lacks an approach to identifying opportunities to ensure that data is actually used to inform policy and practice at country-level. As the grant portfolio matures - with the rollout of a second round of Country Grants, for example – the emphasis is shifting, in particular to a focus on use of data at facility-level.

Evidence suggests that this process of data use is non-linear and unpredictable, and sometimes supported by a ‘policy entrepreneur’*. Itad reported finding opportunities to influence the use of AMR surveillance data, including through engaging in AMR-relevant policy agendas (e.g. relating to antimicrobial stewardship and infection prevention control), in all the countries they looked at. The evaluation team observes that leveraging these in a potential second phase of the Fleming Fund would require the Fund to shift its focus and adjust its ways of working.

* Policy entrepreneurs are the people, such as elected politicians or leaders of interest groups, “with the knowledge, power, tenacity and luck to be able to exploit windows of opportunity and heightened levels of attention to policy problems to promote their ‘pet solutions’ to policymakers” (Cairney et al, 2012). See also, What Is A Policy Entrepreneur?

---