An Assessment of the Monitoring and Evaluation System for the Possible Severe Bacterial Infections Implementation Research Project

Abstract

This study set out to do an assessment of the PSBI implementation research project. The assessment aimed at determining how well the PSBI M&E system meets established standards of a functional M&E system, identify strengths, weaknesses, or challenges, and recommend good practices in setting up and implementing functional M&E system for implementation science research projects. The assessment employed a case study research design with mixed methods approach to data collection and analysis. Data collection was through document review, FHI360 M&E SAT checklist scoring and in-depth interviews with sampled project staff. Data were then analyzed quantitatively and qualitatively to synthesize the results. Overall, based on the checklist, PSBI M&E system scored 167out of possible 232 which is a modest score of 72 percent which was considered functioning moderately well but needs improvement on some aspects highlighted in this report. The scores varied from 50 to 92 percent. The highest scoring domain was data analysis and use domain (92 percent) followed by plans, guidelines, and operational documentations domain (86 percent)while the lowest scoring domains were data verification domain (50 percent) and resources and capacity building domain (55 percent). Key challenges that the PSBI M&E System faced were having too many indicators that later had to be reduced, failure to recognize the actual needs of different context of different facilities, inadequate skilled M&E personnel, use of manual system with a lot of paperwork that progressively became a source of data errors, gaps, and inconsistency. Time constraint was also a challenge in implementation of the project, which meant that some activities had to be implemented simultaneously, an important aspect that directly influences quality of the data and products generated. Another challenge was the COVID-19 pandemic which led to delays and affected how data collection was done especially for the end line survey. Key lessons learnt included importance of a project to involve key stakeholders while developing M&E system for the project with consideration of differences in contextual factors in the study sites and ensuring there are skilled personnel with M&E and data management and analysis expertise. The study recommended that for all projects, whether big or small, efforts should be made to ensure that throughout the project cycle, from conceptualization of the study to the end, there is sufficient resources for M&E which includes personnel and financial resources. Data collection tools should be digitized as much as possible and M&E Personal should have adequate skills on M&E. Monitoring data should be continuously analyzed to improve performance and track the project’s progress and that projects should always invest efforts to developing detailed project M&E documents including M&E frameworks and associated documents.