Randomized Controlled Trials, vs Pragmatic Trials, vs Health Insurance Trials – NEJM Sept 2017

Randomized, Controlled Trials in Health Insurance Systems

N Engl J Med 2017; 377:957-964 September 7, 2017DOI: 10.1056/NEJMra1510058
Niteesh K. Choudhry, M.D., Ph.D.

 Download the PDF from Vitamin D Life

The well-recognized limitations of traditional randomized, controlled trials (RCTs), including their cost, the nature of the patients and providers included in them, and even the types of interventions that they can evaluate, have led to the search for alternative methods and settings for conducting these types of studies. Pragmatic trials,1 also referred to as “practical” or “effectiveness” trials, have been widely advocated as means of addressing these limitations. These designs rely on simplified data-collection processes, strategies such as broad eligibility criteria for both patients and providers, and an acceptance of protocol “violations” such as crossover, nonadherence, and loss to follow-up that make the trial conditions similar to the way in which care is delivered in routine practice.2-5

Many of the pragmatic trials that have appeared in the peer-reviewed literature have recruited patients individually from traditional care settings such as physician offices or hospitals and have prospectively collected baseline and outcome data.6-8 As a result, although aspects of their design provide tremendous efficiencies and greatly enhance generalizability, many pragmatic trials share the fundamental features of traditional RCTs that make them cumbersome to conduct. To address this problem, “registry randomized trials” that leverage the existing participant-identification and data-collection efforts of disease registries have been proposed,9 but registries themselves usually require an expensive infrastructure and trials embedded in them cannot, by definition, be conducted when no relevant registry exists.

Another alternative is to embed trials within health insurance systems and to use the massive amounts of data that insurers generate and collect in the process of administering health benefits (Figure 1)

Typical Features of Different Types of Randomized, Controlled Trials (RCTs).). Information from “claims” submitted to insurers by health care institutions, providers, or diagnostic facilities is commonly used in observational comparative-effectiveness studies and health services research studies but can also provide efficiencies (e.g., the evaluation of study outcomes without the need for prospective data collection) for RCTs. In addition, because the way in which patients interact with insurers is very different from the way in which patients interact with providers in traditional clinical environments, trials that are based in health insurance systems may provide new ways of administering the interventions to be tested and may in fact be the most rigorous way to determine how health insurance itself should be structured. Of course, the potential advantages of trials that are conducted with the use of health insurance data and that are based in health insurance systems create new methodologic challenges. This article outlines some of the considerations, with an emphasis on studies that leverage the data and infrastructure of health insurance systems to identify and evaluate a broad range of clinically relevant and policy-relevant questions.

---

References

1. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis 1967;20:637-48.
2. Ware JH, Hamel MB. Pragmatic trials — guides to better patient care? N Engl J Med 2011;364:1685-7.
3. Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol 2009;62:464-75.
4. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003;290: 1624-32.
5. Koppenaal T, Linmans J, Knottnerus JA, Spigt M. Pragmatic vs. explanatory: an adaptation of the PRECIS tool helps to judge the applicability of systematic reviews for daily practice. J Clin Epidemiol 2011;64:1095-101.
6. Zwarenstein M, Schoeman JH, Vun- dule C, Lombard CJ, Tatley M. Randomised controlled trial of self-supervised and directly observed treatment of tuberculosis. Lancet 1998;352:1340-3.
7. Trivedi MH, Fava M, Wisniewski SR, et al. Medication augmentation after the failure of SSRIs for depression. N Engl J Med 2006;354:1243-52.
8. Elley CR, Kerse N, Arroll B, Robinson E. Effectiveness of counselling patients on physical activity in general practice: cluster randomised controlled trial. BMJ 2003; 326:793.
9. Lauer MS, D’Agostino RB Sr. The randomized registry trial — the next disruptive technology in clinical research? N Engl J Med 2013;369:1579-81.
10. Baicker K, Taubman SL, Allen HL, et al. The Oregon Experiment — effects of Medicaid on clinical outcomes. N Engl J Med 2013;368:1713-22.
11. Newhouse JP, Manning WG, Morris CN, et al. Some interim results from a controlled trial of cost sharing in health insurance. N Engl J Med 1981;305:1501-7.
12. Choudhry NK, Avorn J, Glynn RJ, et al. Full coverage for preventive medications after myocardial infarction. N Engl J Med 2011;365:2088-97.
13. Wennberg DE, Marr A, Lang L, O’Malley S, Bennett G. A randomized trial of a telephone care-ma nagement strategy. N Engl J Med 2010;363:1245-55.
14. Milkman KL, Beshears J, Choi JJ, Laibson D, Madrian BC. Using implementation intentions prompts to enhance influenza vaccination rates. Proc Natl Acad Sci U S A 2011;108:10415-20.
15. Lauffenburger JC, Lewey J, Jan S, et al. Rationale and design of the ENhancing outcomes through Goal Assessment and Generating Engagement in Diabetes Mel- litus (ENGAGE-DM) pragmatic trial. Con- temp Clin Trials 2017;59:57-63.
16. Zwarenstein M, Hux JE, Kelsall D, et al. The Ontario Printed Educational Message (OPEM) trial to narrow the evidence-practice gap with respect to prescribing practices of general and family physicians: a cluster randomized controlled trial, targeting the care of individuals with diabetes and hypertension in Ontario, Canada. Implement Sci 2007;2:37.
17. Angrist JD, Imbens GW, Rubin DB. Identification of causal effects using instrumental variables. J Am Stat Assoc 1996; 91:444-55.
18. Choudhry NK, Shrank WH. Implementing randomized effectiveness trials in large insurance systems. J Clin Epidemiol 2013;66:Suppl:S5-S11.
19. Schneeweiss S, Rassen JA, Glynn RJ, et al. Supplementing claims data with outpatient laboratory test results to improve confounding adjustment in effectiveness studies of lipid-lowering treatments. BMC Med Res Methodol 2012;12:180.
20. Zwarenstein M, Shiller SK, Croxford R, et al. Printed educational messages aimed at family practitioners fail to increase retinal screening among their patients with diabetes: a pragmatic cluster randomized controlled trial ISRCTN72772651. Implement Sci 2014;9:87.
21. Zwarenstein M, Grimshaw JM, Pres- seau J, et al. Printed educational messages fail to increase use of thiazides as first- line medication for hypertension in primary care: a cluster randomized controlled trial ISRCTN72772651. Implement Sci 2016;11:124.
22. Choudhry NK, Krumme AA, Ercole PM, et al. Effect of reminder devices on medication adherence: the REMIND randomized clinical trial. JAMA Intern Med 2017;177:624-31.
23. Hlatky MA, Boothroyd DB, Melsop KA, et al. Economic outcomes of treatment strategies for type 2 diabetes melli- tus and coronary artery disease in the Bypass Angioplasty Revascularization Investigation 2 Diabetes trial. Circulation 2009;120:2550-8.
24. Hynes D, Reda D, Giobbie-Hurder A, et al. Measuring costs in multisite randomized controlled trials: lessons from the VA Cooperative Studies Program. Med Care 1999;37:Suppl Va:AS27-AS36.
25. Petrou S, Murray L, Cooper P, Davidson LL. The accuracy of self-reported healthcare resource utilization in health economic studies. Int J Technol Assess Health Care 2002;18:705-10.
26. Seidl H, Meisinger C, Kirchberger I, Burkhardt K, Kuch B, Holle R. Validity of self-reported hospital admissions in clinical trials depends on recall period length and individual characteristics. J Eval Clin Pract 2016;22:446-54.
27. Suki WN, Zabaneh R, Cangiano JL, et al. Effects of sevelamer and calcium-based phosphate binders on mortality in hemodialysis patients. Kidney Int 2007;72: 1130-7.
28. St Peter WL, Liu J, Weinhandl E, Fan Q. A comparison of sevelamer and calcium- based phosphate binders on mortality, hospitalization, and morbidity in hemodialysis: a secondary analysis of the Dialysis Clinical Outcomes Revisited (DCOR) randomized trial using claims data. Am J Kidney Dis 2008;51:445-54.
29. National Death Index. Atlanta: Centers for Disease Control and Prevention (http://www.cdc.gov/nchs/ndi.htm).
30. Choudhry NK, Isaac T, Lauffenburger JC, et al. Rationale and design of the Study of a Tele-pharmacy Intervention for Chronic diseases to Improve Treatment adherence (STIC2IT): a cluster-randomized pragmatic trial. Am Heart J 2016;180:90-7.
31. Petersen LA, Wright S, Normand SL, Daley J. Positive predictive value of the diagnosis of acute myocardial infarction in an administrative database. J Gen Intern Med 1999;14:555-8.
32. Birman-Deych E, Waterman AD, Yan Y, Nilasena DS, Radford MJ, Gage BF. Accuracy of ICD-9-CM codes for identifying cardiovascular and stroke risk factors. Med Care 2005;43:480-5.
33. Hlatky MA, Ray RM, Burwen DR, et al. Use of Medicare data to identify coronary heart disease outcomes in the Women’s Health Initiative. Circ Cardiovasc Qual Outcomes 2014;7:157-62.
34. Seeger JD, Quinn S, Earnest DL, et al. The Zelnorm Epidemiologic Study (ZEST): a cohort study evaluating incidence of abdominal and pelvic surgery related to tegaserod treatment. BMC Gastroenterol 2012;12:171.
35. Solomon DH, Iversen MD, Avorn J, et al. Osteoporosis telephonic intervention to improve medication regimen adherence: a large, pragmatic, randomized controlled trial. Arch Intern Med 2012;172:477-83.
36. Love D, Custer W, Miller P. All-payer claims databases: state initiatives to improve health care transparency. Issue Brief (Commonw Fund) 2010;99:1-14.
37. Choudhry NK, Krumme AA, Ercole PM, et al. Rationale and design of the Randomized Evaluation to Measure Improvements in Non-adherence from Low- Cost Devices (REMIND) trial. Contemp Clin Trials 2015;43:53-9.
38. Developing approaches to conducting randomized trials using the Mini-Sentinel distributed database: Mini-Sentinel and Clinical Trials Transformation Initiative, 2 014 (http ://www.min i- s en tinel.o rg/wo rk_ products/Statistical_Methods/Mini-Sentinel _Methods_CTTI_Developing-Approaches -to-Conducting-Randomized-Trials-Using -MSDD.pdf).
39. U.S. Code of Federal Regulations: Protection of Human Subjects. Title 45 CFR, Part 46 (http://www.hhs.gov/ohrp/ humansubjects/guidance/45cfr46.html).
40. Nayak RK, Wendler D, Miller FG, Kim SY. Pragmatic randomized trials without standard informed consent? A national survey. Ann Intern Med 2015;163:356-64.
41. Kim SYH, Miller FG. Informed consent for pragmatic trials — the integrated consent model. N Engl J Med 2014;370: 769-72.
42. Faden R, Kass N, Whicher D, Stewart W, Tunis S. Ethics and informed consent for comparative effectiveness research with prospective electronic clinical data. Med Care 2013;51:Suppl 3:S53-S57.
43. Olsen LA, Aisner D, McGinnis JM, eds. The Learning Healthcare System: workshop summary. Washington, DC: National Academies Press, 2007.