by Wenshu Li, Jeffrey A. Leibowitz, Shuoguo Wang, Louisa Walker, Chang Xu, Kuei-Ting Chen, Alexa B. Schrock, Jason Hughes, Nimesh Patel, Julia A. Elvin, Lauren L. Ritterhouse, Ethan Sokol, Garrett Frampton, Lucas Dennis, Bahar Yilmazel, Brennan Decker
Homologous recombination repair (HRR) is a cellular pathway for high-fidelity double strand DNA break repair that uses the sister chromatid as a guide to ensure chromosomal integrity and cell viability. Deficiency in the HRR pathway (HRD) can sensitize tumors to poly (ADP-ribose) polymerase inhibitors (PARPi) and platinum-based chemotherapy, offering an avenue to identify patients who may benefit from targeted therapies. HRD signature (HRDsig) is a pan-solid-tumor biomarker on the FoundationOne®CDx (F1CDx®) assay that employs a DNA scar-based approach to calculate a score based on copy number features (e.g., segment size, oscillation patterns, and breakpoints per chromosome arm) and does not rely on HRR gene alterations, enabling detection of genomic and epigenetic mechanisms of HRD. After finalizing the HRDsig algorithm, analytical validation was conducted in a CAP-accredited, CLIA-certified laboratory on 278 solid tumor and normal tissue specimens. HRDsig results were compared with an independent HRD biomarker, defined by the presence of a reversion mutation restoring HRR gene function. In this evaluation, 100 HRD-positive and 126 HRD-negative samples showed a positive percent agreement of 90.00% and a negative percent agreement of 94.44%. The limit of detection (LoD) was estimated at 23.04% tumor purity, with the limit of blank (LoB) confirmed as zero in 60 normal tissue replicates. Reproducibility testing on 11 positive and 11 negative samples across multiple labs, reagent lots, and sequencers yielded agreement in 99.49% of positive and 99.73% of negative replicates. HRDsig status remained consistent in the presence of interfering substances, demonstrating 100% concordance in spiked samples. These validation results underscore the high analytical concordance, low false-positive rate, and overall robustness of HRDsig for reliable assessment of homologous recombination deficiency.Communication skills assessment (CSA) is essential for ensuring competency, guiding educational practices and safeguarding regulatory compliance in health professions education (HPE). However, there appears to be heterogeneity in the reporting of validity evidence from CSA methods across the health profession that complicates our interpretation of the quality of assessment methods. Our objective was to map reliability and validity evidence from scores of CSA methods that have been reported in HPE.
Scoping review.
MEDLINE, Embase, PsycINFO, CINAHL, ERIC, CAB Abstracts and Scopus databases were searched up to March 2024.
We included studies, available in English, that reported validity evidence (content-related, internal structure, relationship with other variables, response processes and consequences) for CSA methods in HPE. There were no restrictions related to date of publication.
Two independent reviewers completed data extraction and assessed study quality using the Medical Education Research Study Quality Instrument. Data were reported using descriptive analysis (mean, median, range).
A total of 146 eligible studies were identified, including 98 394 participants. Most studies were conducted in human medicine (124 studies) and participants were mostly undergraduate students (85 studies). Performance-based, simulated, inperson CSA was most prevalent, comprising 115 studies, of which 68 studies were objective structured clinical examination-based. Other types of methods that were reported were workplace-based assessment; asynchronous, video-based assessment; knowledge-based assessment and performance-based, simulated, virtual assessment. Included studies used a diverse range of communications skills frameworks, rating scales and raters. Internal structure was the most reported source of validity evidence (130 studies (90%), followed by content-related (108 studies (74%), relationships with other variables (86 studies (59%), response processes (15 studies (10%) and consequences (16 studies (11%).
This scoping review identified gaps in the sources of validity evidence related to assessment method that have been used to support the use of CSA methods. These gaps could be addressed by studies explicitly defining the communication skill construct(s) assessed, clarifying the validity source(s) reported and defining the intended purpose and use of the scores (ie, for learning and feedback, for decision making purposes). Our review provides a map where targeted CSA development and support are needed. Limitations of the evidence come from score interpretation being constrained by the heterogeneity of the definition of communication skills across the health professions and the reporting quality of the studies.