Many researchers conduct a process evaluation alongside an effectiveness trial of a public health intervention to better understand mechanisms behind observed effects. Yet, there is no standardised, scientifically accepted guideline for reporting such process evaluations, which impedes interpretation and comparison of study results. The aim of this project was to develop a consensus-based and expert-based guideline for reporting process evaluations of public health interventions conducted alongside an effectiveness trial.

We conducted an e-Delphi study with a large panel of international experts.

Based on purposive sampling, we invited 137 international experts that had been involved in the design of process evaluations, researchers who published high-profile process evaluations or frameworks, editors of journals that publish process evaluations, and authors of other reporting guidelines.

Based on a literature search, a first draft of the reporting guideline included 32 items, which was proposed to panel members during the first round. Of the invited 137 invited international experts, 73 (53%) participated in at least one round of the e-Delphi study. Participants rated the inclusion and comprehensibility of the proposed items on a 5-point Likert scale and provided comments and suggestions for relevance and definitions of the items. Adjustments to the items and descriptions were proposed to the e-Delphi panel until consensus of ≥67% for each individual item was reached. In total, 64 (88% of 73) completed round 2, and 55 (76% of 73) completed round 3. This resulted in 19 items that are included in the consensus-based process evaluation reporting guideline for public health intervention studies (CONPHES) guideline. The items cover a detailed description of the intervention that is evaluated, the implementation strategies applied, and underlying causal pathways, and the role of the delivery and support team. The guideline also requires describing the evaluation framework and how evaluation outcomes were assessed. Lastly, the guideline includes items on providing a detailed description of applied analyses (both quantitative and qualitative) and measures for assuring quality. The guideline is accompanied by an Explanation and Elaboration document, with a more detailed explanation of each item.

We expect that the CONPHES reporting guideline for process evaluations of public health interventions can improve the reporting of process evaluations of interventions aimed at promoting public health. This can potentially facilitate more effective translation of public health research into practice and contribute to improving both individual and population health outcomes.

To evaluate the effectiveness of the recommendation improvement matrix (RIM) methodology for improving the quality of recommendations resulting from sentinel event analysis, where we hypothesise that the use of the RIM methodology leads to better quality recommendations.

A before-and-after analysis of the quality of the formulated recommendations after sentinel event analysis.

The study was carried out during the 2023 Saudi Patient Safety Centre International Patient Safety Conference.

36 conference participants, including nurses, medical doctors, pharmacists, dentists, general practitioners and quality officers.

RIM methodology training during a structured 3-hour workshop.

The primary outcome was the proportion of recommendations that using the ² test, passed the RIM filter criteria before and after training. Secondary outcomes included changes in recommendation categorisation within the matrix and participant ratings of feasibility and usability on a five-point Likert scale using a t-test for comparison.

Prior to training, 49 recommendations were generated, of which 63.3% met the filter criteria. Post-training, the proportion of recommendations passing the filter increased significantly to 83% (p=0.00543). Adjustments to recommendations primarily improved alignment with the filter criteria, though limited improvements were observed in matrix categorisation. Participants rated the methodology’s feasibility and usability highly, with average scores of 4.39/5 and 4.43/5, respectively. However, 46% expressed a need for additional training, particularly on the matrix application.

The RIM methodology significantly improves the quality of recommendations following sentinel event analyses. To enhance its impact, further training focusing on matrix application is necessary. Incorporating the methodology into healthcare education and professional development could strengthen patient safety practices.