Trauma Service Guidelines – Post Implementation Evaluation

Introduction

The clinical mission of the Division of Acute Care Surgery is to provide outstanding patient-centered care. _i To fulfill this mission, the Division aims to create a learning health care system, in “which science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the care process… and new knowledge captured as an integral by-product of the delivery experience.” _ii

Post Implementation Evaluation

As a learning health care system, continuous improvement and innovation is woven into the fabric of clinical care. As such, the implemented guidelines and algorithms require prospective, rigorous evaluation to ensure that they are effective. History is littered with examples of interventions thought to be effective but then found to be harmful (e.g. tight glycemic control, post-menopausal hormones to prevent coronary artery disease and stroke, dopamine as first line vasopressor in septic shock).

Given the fallibility of expert opinion, biologic plausibility, prospective observational studies, and small clinical trials, it is imperative that the clinical effectiveness of guidelines and algorithms be assessed in the most rigorous method possible. This may be done via quality improvement studies, clinical trials, quasi-experimental studies, or joint quality improvement-research studies.

The post-implementation of all guidelines and algorithms, however, is not entirely feasible given limited resources and a limited number of trials that may be performed at one time. To balance, the demand for post-implementation evaluation and to implement best available practices, we will categorize guidelines and algorithms into one of five categories based upon strength of recommendation.

Category	Description & Recommendation	Typical Supporting Evidence	Approach to Implementing Therapy	Assessment of Treatment Effects	Significant Variation Encouraged
1	Therapy recommended based upon high certainty that the net benefit is substantial.	Major benefit exceeding hazards identified in all RCTs (>1) or in a meta-analysis of all RCTs, particularly large, well-designed trials	Implement for all patients for whom benefit has been established.	Patient outcomes need not be monitored to assess benefit. QI may be needed to assure the therapy is fully & properly implemented.	No
2	Therapy recommended based upon high certainty that the net benefit is moderate or that there is moderate certainty that the net benefit is moderate to substantial.	Benefit identified in multiple well designed cohort studies, in a single high quality RCT, or in some but not all well designed RCTs.	May implement in all patients with plan to retrospectively identify effects on outcomes. However, assurance that benefits exceed hazards is much greater if implemented in a RCT or at least a prospective cohort study using concurrent controls or a time series comparison.	Assess all important outcomes. Interpret findings cautiously, particularly with retrospective comparisons.	No
3	A therapy for which the evidence is so weak that neither a recommendation for or against use is warranted.	Benefit identified only in some observational studies, in flawed RCTs, or well designed RCTs with conflicting results.	Either conduct an RCT or await further published studies, (RCTs if feasible)	If used, assess all important outcomes, within an evaluation carefully designed to avoid biased results.	Yes
4	Therapy not recommended based upon moderate or high certainty that the service has no net benefit or the harms outweigh the benefits.	Likely harms or significant costs in; absence of well performed RCTs or meta-analyses showing overriding benefit.	Ensure that therapy is not routinely used.	QI may be needed to avoid routine clinical use. If used in research, rigorously assess all important outcomes.	No
5	Current evidence for the therapy is insufficient to assess the balance of benefits and harms.	Insufficient evidence available to assess for recommendation.	Either conduct an RCT or await further published studies, (RCTs if feasible)	QI may well be needed to ensure the therapy is not used clinically. If used in research, rigorously assess all important outcomes.	Yes

References

_i https://med.uth.edu/about-us/

_ii Roundtable on Value & Science-Driven Health Care. The Roundtable. Washington, DC: Institute of Medicine; 2012.