Using meta-analyses for comparative effectiveness research

The coming years will be an ideal time for investigators in radiology to develop a comparative effectiveness research (CER) initiative and to establish a lasting precedent for CER in our field.

Instrumental variable (IV) methods have been proposed as a potential approach to the common problem of uncontrolled confounding in comparative studies of medical interventions, but IV methods are unfamiliar to many researchers. The goal of this article is to provide a non-technical, practical introduction to IV methods for comparative safety and effectiveness research. We outline the principles and basic assumptions necessary for valid IV estimation, discuss how to interpret the results of an IV study, provide a review of instruments that have been used in comparative effectiveness research, and suggest some minimal reporting standards for an IV analysis. Finally, we offer our perspective of the role of IV estimation vis-à-vis more traditional approaches based on statistical modeling of the exposure or outcome. We anticipate that IV methods will be often underpowered for drug safety studies of very rare outcomes, but may be potentially useful in studies of intended effects where uncontrolled confounding may be substantial.

Back to table of contents Previous article Next article Professional NewsFull AccessAMA Backs 'Medical Home' Despite Specialists' ConcernsMark MoranMark MoranPublished Online:19 Dec 2008https://doi.org/10.1176/pn.43.24.0004The AMA is supporting the concept of the "medical home," a patient-centered philosophy of universal access to care championed by four primary care medical organizations. AMA delegates at last month's Interim Meeting in Orlando, Fla., voted to adopt the Joint Principles of a Patient-Centered Medical Home, formulated in 2007 by the American Academy of Pediatrics, American Academy of Family Physicians, American Osteopathic Association, and American College of Physicians.The medical home concept was endorsed by the AMA with the addition of a resolution that the association continue to study the concept "with particular emphasis on funding sources and payment structures"—a caveat added by delegates in response to considerable anxiety in the House of Delegates about whether the medical-home concept would put primary care physicians in the position of a "gatekeeper" with power to control access to specialty services and whether it would divert finite resources toward primary care and away from specialty services (see Original article: AMA Suggests Ways to Change How Medicare Pays M.D.s).The concept was supported by John McIntyre, M.D., chair of the Section Council on Psychiatry. McIntyre is a member of the AMA Council on Medical Services, which is charged with studying the financing structure of the medical-home model, with a report on the subject due at the AMA's 2009 Annual Meeting in June."My feeling is that it's a very good concept," McIntyre told Psychiatric News. "I think we need to move in this direction.""The Joint Principles of a Patient-Centered Medical Home" include the following fundamental concepts, as elucidated in a March 2007 press release issued by the four primary care organizations:Personal physician: Each patient has an ongoing relationship with a personal physician trained to provide first contact and continuous and comprehensive care.Physician-directed medical practice: The personal physician leads a team of individuals at the practice level who collectively take responsibility for the ongoing care of patients.Whole-person orientation: The personal physician is responsible for providing for all the patient's health care needs or taking responsibility for appropriately arranging care with other qualified professionals. This includes care for all stages of life, acute care, chronic care, preventive services, and end-of-life care.Coordination and integration of care: Care is coordinated and/or integrated across all elements of the complex health care system (for example, subspecialty care, hospitals, home health agencies, nursing homes) and the patient's community (for example, family and public and private community-based services). Care is facilitated by registries, health information exchange, and other means to assure that patients get the indicated care when and where they need and want it in a culturally and linguistically appropriate manner.In reference committee hearings and on the floor of the House of Delegates, some physician groups—especially obstetrician-gynecologists and surgical specialists—cautioned that endorsement of the medical home was inherently an endorsement of a financing structure that would favor primary care within a system in which financing was a "zero sum game.""If we adopt these principles, we are in effect adopting a financing structure," said Erin Tracy, M.D., M.P.H., an obstetrician and at-large representative from Boston. "We are prematurely deciding how to allocate resources."Peter Schwartz, M.D., also an ob-gyn, added that "Medicare Part B is not expanding any time soon" and that funds diverted to primary care would inevitably mean less money for specialty care. He urged that the concept not be adopted until the Council on Medical Services came back with its report in June.McIntyre said that the medical-home concept and other issues related to health system reform were bound to be front and center at the annual meeting of the house in June 2009.Also approved by the AMA delegates—but again with concerns about how it might impact payment to physicians—was a report by the AMA's Council on Medical Services calling for the creation of an independent entity to do comparative effectiveness research (CER). The report outlined 11 principles around which such an entity should be formed (see Original article: What Principles Should Guide Treatment Effectiveness Study?)."Most research on medical treatments to date has focused on comparing the effects of a given treatment to no treatment, rather than comparing alternative treatments to each other," the CMS report stated. "The lack of comparative research on the effects of alternative services and treatments has led to a lack of knowledge about whether new treatments outperform existing treatments. In cases in which there are multiple, alternate interventions to treat a health condition that have been proven to be effective, studies have also shown that the treatment patients receive often depends on where they live.... Ultimately, having evidence that compares outcomes of alternative treatments, as well as treatment costs, will not only increasingly integrate value into the health care decision-making process, but will also equip physicians with the ability to achieve the right care for each patient."The report also noted that there is already significant federal legislation introduced in this area and that it is very likely that some sort of body to perform comparative research will be established. Most prominent is the Comparative Effectiveness Research Act of 2008, introduced earlier this year by Sens. Max Baucus (D-Mont.) and Kent Conrad (D-N.D.). The bill would establish an independent comparative effectiveness research institute.The council report, and much testimony at the house, emphasized the importance of the composition of whatever entity will be created. "This will be independent from [industry and other influences] and would have on it consumers, physicians, and health care executives," McIntyre told Psychiatric News."I think the whole idea of comparative effectiveness research is a step beyond developing practice guidelines," said McIntyre, who has led APA's practice-guideline effort. "The next step is comparing treatments with each other to determine which is more effective and being able to do that independently and without being dominated by [the pharmaceutical or device manufacturing] industry."But as with the medical-home resolution, the report on CER raised some concern about how such an entity would affect physician payment for various treatments, particularly if the body doing the research incorporates cost-effectiveness research."The way the council felt about this is that the CER body shouldn't be making coverage or payment decisions, but we did think there has to be some recommendation that items that are of very, very high cost be looked at first," McIntyre explained. "So the exact role of cost-effectiveness wasn't refined at this point."McIntyre noted, however, that the house did defeat an amendment that would have made cost-effectiveness off limits as a subject for the proposed new CER entity.As one delegate on the house floor, Robert Hughes, M.D., of New York, put it, "Cost is an everyday part of my practice, and I would like some guidance.""The Joint Statement of Principles of a Patient Centered Medical Home" is posted at<www.medicalhomeinfo.org/Joint%20Statement.pdf>. Reports and resolutions from the AMA Interim Meeting are posted at<www.ama-assn.org/ama/pub/category/16552.html>.▪ ISSUES NewArchived

Patient-Centered Outcomes Research: Early Evidence From A Burgeoning Field.

HEALTH REFORM LEGISLATION AND an earmarked $1.1 billion have set the stage for the commencement of federally endorsed comparative effectiveness research. But it remains to be seen just how such research will be carried out—and how the results might change health care. The engine for federally sponsored comparative effectiveness research is the new Patient-Centered Outcomes Research Institute, whose 21-member board of governors, which includes the directors of the Agency for Healthcare Research and Quality and the National Institutes of Health, was announced September 23. The institute, established by the Patient Protection and Affordable Care Act of 2010, is a nonprofit public-private endeavor charged with disbursing funding for comparative effectiveness research. According to the act, the purpose of the institute is “to assist patients, clinicians, purchasers, and policymakers in making informed health decisions.” Institute staff are to apply research and evidence to improve methods for preventing, diagnosing, treating, monitoring, and managing health conditions. The institute is also charged with disseminating research findings on health outcomes, clinical effectiveness, and the appropriateness of medical treatments, services, and items. To achieve these goals, the institute will create a standing methodology committee to develop and update scientifically based standards for research conducted through the institute. In addition, the institute will ensure peer review and make research findings publicly available within 90 days. The institute will also allow for public comment periods prior to such actions as the adoption of national priorities, the research project agenda, the methodological standards, and the peer review process, and after the release of draft findings of reviews of existing research and evidence. As clear as these expectations are, the institute must still navigate political waters roiled with charges that comparative effectiveness research is a tool for the continued takeover of health care by the federal government and a way to justify health care rationing. Such attitudes among certain members of Congress resulted in the passage of a law that severely restricts one of the perceived benefits of comparative effectiveness research: the possibility of cost savings in health care. Savings would presumably follow from the identification of proven therapies that are less expensive than those in common use, although such research could also identify more effective treatments that are more expensive. The law stipulates that the institute “shall not develop or employ a dollarsper-quality adjusted life year (or similar measure that discounts the value of a life because of an individual’s disability) as a threshold to establish what type of health care is cost effective or recommended” under Medicare. With such restrictions, little wonder that researchers and economists are looking at other ways to bring cost into the comparative effectiveness research conversation. Some of those approaches are discussed in Health Affairs, which dedicated its October issue to a variety of articles exploring issues surrounding comparative effectiveness research.

Comparative effectiveness research encompasses research that compares two interventions to each other, and takes place in real-world settings without strict inclusion and exclusion criteria, according to the established standard of care. There is a need for comparative evaluations of the treatments for obstructive sleep apnea, a disease associated with increased cardiovascular risk, stroke and metabolic derangement. This article reviews the recent, representative literature that addresses obstructive sleep apnea intervention and treatment, paradigms for diagnosis and randomized control trials addressing the efficacy of interventions, in an effort to demonstrate examples of both traditional observational and randomized control trials, as well as to illustrate the considerable overlap between some traditional studies and comparative effectiveness research. Despite methodological challenges, the potentially large clinical and public health impact of obstructive sleep apnea, accompanied by considerable cost, mandates that randomized controlled trials and comparative effectiveness research be systematically applied to identify both the efficacy and effectiveness of alternative diagnosis and treatment strategies.

While the approval of biopharmaceuticals hinges on a demonstration of safety and efficacy through randomized clinical trials, other stakeholders have historically evaluated the approved product based on repackaged clinical trial data with limited ‘real-world’ validity. The US healthcare system is now evolving into a more complex ‘ecosystem’ where regulatory approval is just a first step toward successful market access. Indeed, the adoption and financial success of a product relies on stakeholder preference modulated by environmental and competitive factors, such as the clinical burden of a disease or the availability of cheaper alternative therapies. One market challenge faced by the biopharmaceutical industry is that of measuring product performance under the imperfect research conditions of the real world, replete with its confounders and effectmodifiers. This has led to discordance in the healthcare ecosystem; regulators have the data they need but other stakeholders are left with blind spots when making decisions. Comparative effectiveness research (CER) has the potential to fill these data gaps, helping to ensure optimalmarket access for innovative products. This article examines how companies can meet increasing demands for CER using a three-step planning approach involving CER diagnosis, CER prognosis and CER prescription. The approach assesses (i) the likelihood of a drug or device being included in a CERstudy, the parameters of such a study (e.g. comparators, endpoints, patient population) and the groups likely to conduct and fund the research; (ii) the potential for the drug or device to receive a positive evaluation as well as the conditions that can optimize the evaluation; and (iii) the activities to be undertaken throughout the clinical development process to best position a drug or device for CER. If initiated prior to approval, these steps will ensure that the data developed for each intervention is relevant to the web of healthcare stakeholders that will determine its success in the marketplace.

The purpose of this study was to investigate publication patterns for comparative effectiveness research (CER) on spine neurosurgery. The authors searched the PubMed database for the period 1980-2012 using the key words "cost analysis," "utility analysis," "cost-utility," "outcomes research," "practical clinical research," "comparator trial," and "comparative effectiveness research," linked with "effectiveness" and "spine neurosurgery." From 1980 through April 9, 2012, neurosurgery CER publications accounted for 1.38% of worldwide CER publications (8657 of 626,330 articles). Spine neurosurgery CER accounted for only 0.02%, with 132 articles. The journal with the greatest number of publications on spine neurosurgery CER was Spine, followed by the Journal of Neurosurgery: Spine. The average annual publication rate for spine neurosurgery CER during this period was 4 articles (132 articles in 33 years), with 68 (51.52%) of the 132 articles being published within the past 5 years and a rising trend beginning in 2008. The top 3 contributing countries were the US, Turkey, and Japan, with 68, 8, and 7 articles, respectively. Only 8 regular articles (6.06%) focused on cost analysis. There is a paucity of publications using CER methodology in spine neurosurgery. Few articles address the issue of cost analysis. The promotion of continuing medical education in CER methodology is warranted. Further investigations to address cost analysis in comparative effectiveness studies of spine neurosurgery are crucial to expand the application of CER in public health.

Evaluation of Pharmacy and Therapeutic (P&amp;T) Committee member knowledge, attitudes and ability regarding the use of comparative effectiveness research (CER) in health care decision-making

Screening is an established strategy for reducing death from colorectal cancer. The availability of multiple guidelines-recommended for colorectal cancer increases the complexity of screening decision-making for patients, clinicians and policy makers. This is compounded by differences in the strength of the evidence supporting the use of currently available tests in actual clinical practice. Therefore, colorectal cancer screening represents an important focus for comparative effectiveness research: the available tests also vary in cost, complexity and risk of complications. Another unique situation with colorectal cancer screening is the increasing use of colonoscopy for screening, the effectiveness of which has been recently called into question, while tests with proven efficacy such as sigmoidoscopy and fecal-based tests are now less frequently used. This presentation will discuss colorectal cancer comparative effectiveness research from the context of an NCI-funded project. The specific aim of that project is to estimate the effectiveness of screening colonoscopy in reducing death from colorectal cancer among average risk adults when compared to no screening. Of particular interest is to determine if colonoscopy is effective in reducing death from right-sided colon cancers and to examine the impact of quality of colonoscopy on its effectiveness. The presentation will discuss the following concerning colorectal cancer screening comparative effectiveness research: 1) the disconnect between health care policies, clinical practice and scientific evidence; 2) the methodological challenges of studying the comparative effectiveness of colorectal cancer screening tests; and 3) review the design of the funded observational study. Citation Information: Cancer Epidemiol Biomarkers Prev 2010;19(10 Suppl):B114.

The importance of the design of observational studies in comparative effectiveness research: Lessons from the GARFIELD-AF and ORBIT-AF registries

Introduction: Comparative effectiveness and efficacy research for analysis and practice (CEERAP) was performed to assess the effects of penicillin-based versus erythromycin-based antibiotic treatments in patients with skin and soft tissue infections (SSTIs) including cellulitis, impetigo, and erysipelas. Because SSTIs, especially orofacial cellulitis, are volatile infectious diseases of a life-threatening nature, research on the most efficacious remedies is necessary. Methods: The stringent bibliome yielded three systematic reviews, which were examined for quality of research synthesis protocol and clinical relevance. Results: The sample size of three, rendered the statistical analyses and cumulative meta-analysis problematic. Conclusion: The systematic review outlined here should aid in increasing clinical awareness, improving patient health literacy, and promoting consensus of the best evidence base (BEB) to mitigate the threat of sepsis and potential death caused by cellulitis infections.

This paper discusses an important element that is missing from the existing algorithm of preconception care, namely, comparative effectiveness research (CER). To our knowledge, there has been limited assessment of the comparative effectiveness of diverse interventions that promote preconception health, conditions under which these are most effective, for which particular populations, and their comparative costs. CER can improve the decision making process for the funding, development, implementation, and evaluation of comprehensive preconception care programs, specifically by identifying the most effective interventions with acceptable costs to society. This paper will examine the framework behind preconception care and how the inclusion of comparative effectiveness research and evaluation into the existing algorithm of preconception care could foster improvement in maternal and child health. We discuss challenges and opportunities regarding the utilization of CER in the decision making process in preconception health, and finally, we provide recommendations for future directions.

Introduction Part I: Evidence 1. The Generation of Evidence in Medicine: Evaluating the Benefits and Harms of Treatment 2. The Use of Evidence: Federal Regulation of Prescription Drugs and Medical Devices 3. The Subversion of Evidence: The Rosiglitazone Story 4. Operate First, Evidence Later: The Special Case of Surgery 5. Doctors and Evidence: Evidence-based Medicine Part II: Politics and Policy Introduction to Part II 6. Comparative Effective Research: A New for U.S. Health Care Costs 7. The Legislative Odyssey of Comparative Effectiveness Research: Birth in 2002 to the First Federal Mandate in 2003 8. 2005 to 2007: The Health Services Research Community Stirs, the Word Awakens Policymakers, and the House CHAMP Act Creates Divisions 9. 2008 to 2010: The Baucus-Conrad Bills, the American Recovery and Reinvestment Act, and the Patient Protection and Affordable Care Act 10. The Scope, Financing, and Processes of Federally-mandated Comparative Effectiveness Research and Why They Matter Part III: Interests Introduction to Part III 11. Aligning Clinical Practice with Best Evidence of Comparative Effectiveness: Payment Policy for Evidence-based Medical Care 12. Dealing with Uncertainty: Payment Policy for the Generation of Evidence as Part of Clinical Practice Epilogue

Comparative Effectiveness Research in Practice and Policy for Radiation Oncology