menu

Overview of CMTP’s Priority-Setting Process

Center for Medical Technology Policy · 2009

Download the original PDF (8 pages). This page shows the full text of the PDF. The figures, some tables and the original layout are in the PDF.

Overview of CMTP’s Priority-Setting Process

© 2009 Center for Medical Technology Policy. Unauthorized use or distribution prohibited. All rights reserved.

Introduction

The primary goal of CMTP is to improve the process for generating high-quality and useful information about the risks, benefits, and costs of promising new medical technologies. Because the pace of innovation in medicine is so rapid and the numbers of technologies that need evaluation far outweigh CMTP’s available resources, we have developed an in-house priority-setting framework to identify and select the most promising medical technologies for consideration. To that end, it is essential that the priority-setting process incorporate the opinions and expertise of all relevant stakeholders. Selected technology topics are then used in the various products and services that CMTP provides, such as Effectiveness Guidance Documents (EGDs) and Coverage with Evidence Development (CED) initiatives.

Depending on the intent and purpose of the project(s) for which priority setting is required, typically we will choose a disease category (i.e. cardiology, oncology, etc.) before the process begins. To select disease categories, CMTP staff looked at three decisive criteria: (1) overall disease burden, (2) overall cost impact, and (3) references by technology assessment organization and payers during the past 12 months. Based upon these criteria, cardiology and oncology were chosen as the two disease categories for initial focus. In no particular order, cardiology was chosen for the first round of prioritization in summer 2009 and oncology for the second round later this year. Both of these disease areas affect a large number of patients, have significant costs, and challenge payers in their coverage decisions.

Step 1: Horizon-Scanning and Selection of Topics CMTP staff compile a comprehensive list of emerging technologies within the selected disease category. The only criterion for this initial list is that the technology has undergone assessment by a horizon scanning, technology assessment, or payer organization within the past year. Potential topics are taken from a variety of sources, including: − Topic nominations from external stakeholders using CMTP’s web-based form − Horizon-scanning literature from free international and domestic resources within the past 12 months − Recent technology assessments. − Technologies that did not receive favorable coverage determinations by large health plans. − Contracting with a health technology research/consulting company for a list of technologies that fit the relevant criteria. For the round of priority setting in cardiology, this process yielded approximately forty potential topics.

Step 2: Narrowing List of Topics CMTP staff trim the comprehensive list of potential technologies to a list of 10 promising topics to move to the next round of selection. While no formal criteria or process has been developed specifically for the purpose of narrowing the broad list of technology topics, common reasons for removal of technologies from the initial list will include: (1) there is only one vendor producing the technology, (2) the technology is already being covered by most payers, (3) the evidence gaps are not significant. Steps of the Priority Setting Process

  • Horizon-Scanning and Selection of Topics
  • Narrowing List of Topics
  • Creation of Expert Workgroup
  • Background Rand Technology Briefs
  • Creation of Criteria List
  • Priority-Setting 6.1 Initial round: Web-survey, Based on Criteria 6.2 Final round: In-Person Meeting Step 3: Creation of Expert Workgroup Upon the narrowing of the list of potential topics to ten technologies, the process moves from an internal CMTP process to one that incorporates the expertise of a range of stakeholders within the chosen disease area. Expert workgroups are charged with ranking the remaining 10 topics. With an eye towards incorporating a range of perspectives, workgroups typically consist of around four clinicians in the disease area, a payer representative, and a disease-specific patient representative.

Step 4: Background Research and Technology Briefs With the goal of establishing uniform and easily accessible information for the expert workgroup, CMTP prepares brief but high-level reports on each of the remaining 10 technologies. In the past, these reports have proven vital to the workgroup’s initial prioritization of topics and in the topic selection meeting. Sections within each report include:

  • Description of the technology, approved indications, known information about diffusion
  • Description of the patient group (with estimated patient numbers), expected utilization rates
  • Current diagnostic or treatment alternatives,
  • Known over-, under-, misuse use of the technology, variations in use or outcomes
  • High level summary of the current research evidence of clinical effectiveness, benefits, harms, and any known evidence gaps, patient safety issues
  • Details of any ongoing or related research activities,
  • Direct and indirect health care costs (annual and/or lifetime),
  • Potential cost-effectiveness as compared to alternative treatments,
  • Known information about reimbursement (public and private payers)
  • Political climate surrounding the technology, particular reasons for urgency,
  • Social or ethical concerns related to the technology,
  • List of important references These briefs are also adapted into shorter, less technical one-page summaries to be distributed to nonclinician members of the workgroup. See Appendix A for an example.

Step 5: Creation of Criteria List CMTP staff will give the expert workgroup a list of criteria to guide their efforts, further described in Step 6. Potential criteria were culled from well-establish healthcare and technology assessment organizations, as well as related work done by CMTP for previous projects. Criteria are organized as either 1) broadly applicable to all CMTP products, 2) applicable to a specific product due to differences in purpose and scope, or 3) secondary (non-absolute) criteria. Criteria are listed in no particular order.

General Criteria:

  • Clinical Impact: The technology has potential to provide a clinically significant improvement in net health outcome compared to the most effective or other widely used alternatives.
  • Cost Impact: The technology may offer a cost-effective alternative to existing treatments or offers significant cost savings to patients, health plans or public health programs.
  • Need for more evidence: The proposed research will contribute to reducing the uncertainties identified in any current technology assessment, and will likely change the outcome of the technology assessment and/or coverage decisions. (i.e. the concept of improvability of evidence or value of information)
  • Disease Burden: The technology is used to prevent, diagnose, treat, or provide palliative care for a condition or disease that poses a high burden of illness or high costs to society Step 6: Priority-Setting CMTP uses a modified Delphi/group-nominal method for the prioritization process, which is a systematic method involving a panel of experts and multiple rounds of prioritization. Please reference Appendix B for brief summaries of these priority-setting methods. CMTP’s Product- Specific Criteria:

Coverage with Evidence Development (CED)

  • Potential for unregulated diffusion: Market factors suggest that relevant evidence would not become available until after the technology is already in use
  • Pressure on payers: There is a high degree of pressure on payers to cover this technology, even with limited evidence
  • Feasibility: Tentative study designs can be identified and are feasible, timely, and affordable
  • Lack of evidence early on: The technology is new enough so that proposed research can be used to inform decisions
  • Multiple vendors: Preference will be given to technologies with multiple vendors/sources
  • Trial design options: If randomization is the only appropriate way to answer the question, the technology is not the sole treatment option and the condition is not life threatening
  • FDA approval: The technology is FDA approved or will likely to be in the next 1-2 years
  • Applicability to other technologies: A CED initiative would have potential to be a template for other CED projects Effectiveness Guidance Documents (EGDs)
  • Need for study design guidance: Recent policy or political events have highlight the absence of a shared evidentiary framework between developers and healthcare decision-makers for evaluating the effectiveness of the category of technologies
  • Impact of technology over time: The category of technologies is an area of active clinical development, and a substantial importance because of potential for clinical and/or economic impact over the next 5-10 years

Secondary Criteria:

  • Feasibility: Political and operational feasibility of conducting the research (e.g., whether there may be significant ethical, legal or social barriers or equity concerns)
  • Overlap/Redundancy: The availability ongoing clinical research or likelihood that such research will be initiated in the near future.
  • Risk of Waiting: Potential impact or risk of waiting Step 6.1: Initial round: Web-survey Based on Criteria The expert workgroup will be provided with the ten technology briefs and other optional background information for their review. After a two week reading period, workgroup members will complete a web-based survey tool to assess each technology based upon the priority setting criteria outlined in Step 5. After this initial round, CMTP staff will provide workgroup members with average scores, and a sense of group consensus. Additionally, workgroup members will be given the opportunity to speak with key staff and advisors to address any questions or issues. If workgroup members indicate that they have greatly altered their opinions based on conversations with key staff, a second round of survey-based prioritization may take place and new results will be distributed before the in-person meeting.

Step 6.2: In-Person Meeting - Final Round In the final round of prioritization, an in-person meeting takes place. CMTP staff lead the workgroup in discussions of each technology at length. The content of each discussion draws from both the technology briefs and the experts’ own personal knowledge and experiences. While not explicitly following the criteria in a formal way, the facilitators ask questions based largely on the criteria. At the end of the meeting, participants are asked to each produce lists of their preferred topics for the various CMTP projects and products they are asked to consider. Following this meeting, a follow-up teleconference will be used to share the rankings of the entire group, come to a final consensus, and address any final questions or concerns on the specific technologies.

APPENDIX A – Example of One-page Technology Brief

PHARMACOGENETIC TESTING FOR ESTIMATING OPTIMAL INITIAL WARFARIN DOSAGE

BACKGROUND. Pharmacogenetics, or the study of how people respond to drug therapy given their genetic make-up, can be applied to numerous conditions. Tests are currently being explored that could influence treatment of various cancers, infectious diseases, and general heritable syndromes. Integrating genetic analysis and corresponding dosing algorithms into clinical practice will ultimately require a significant amount of coordination across clinical departments.

OVERVIEW OF TECHNOLOGY. Warfarin is the most widely used oral anticoagulant agent worldwide. The use of genetic testing to analyze a patient’s metabolism of warfarin allows for the personalization of dosage. The FDA updated the label on warfarin in August 2007 to encourage, but importantly, not require pharmacogenetic testing to be used in dosing. While studies have shown positive results, many clinicians are calling for more robust clinical data before incorporating what would be a significant shift in treatment patterns. Thus, the traditional approach of determining the appropriate dose of warfarin by obtaining repeated international normalized ratio (INR) measurements and making successive dose adjustments seems to be the standard of care. This is currently the only alternative to the pharmacogenetic tests available.

IMPACT ANALYSIS Clinical Implications: No prospective studies have yet shown pharmacogenetic warfarin testing to be effective. Prospective studies will be needed to determine whether testing can improve patient outcomes, identify patient subgroups and clarify risks and costs associated with the use of the tests. Randomized controlled trials are necessary to evaluate the impact of pharmacogenetics on dosing accuracy, monitoring requirements, and other factors. Financial Implications and Cost–Effectiveness. In 2006, the American Enterprise Institute-Brookings Joint Center released a paper focusing on the use of genetic testing specifically for warfarin. They concluded that integrating these tests could “avoid 85,000 serious bleeding events and 17,000 strokes annually.” This was estimated to equal $1.1 billion in cost savings annually. Although these estimates are impressive for sure, they are just that – estimates. As emphasized in the editorial by Henry Bussey, these and other conclusions are merely projections with no clinical data to support their findings. One of the major safety arguments employed by Bussey was the risk of misusing genetic information leading to less frequent INR monitoring and a higher risk of blood clot or bleeding complications Reimbursement. CMS reimburses testing for all patients prescribed with warfarin at about $140. Cigna, a commercial carrier, covers pharmacogenetic testing for warfarin metabolism as medically necessary to assist with induction dosing of warfarin therapy. Industry recognizes slow adoption of the DNA technologies, citing pushback from physicians who are not eager to change practice patterns in response to an unproven application. Payers are also dragging their feet, not providing any additional reimbursement.

APPENDIX B – Summary of Priority Setting Methods

Taken from Amie Shen’s “Report on Consumer Input in Priority Setting for Clinical Research”:

Delphi Method The Delphi method was developed by RAND Corporation in the 1950s. Participants are involved in an iterative process of responding to mail questionnaires and receiving feedback about the group’s responses (and their own initial judgment). This process may be repeated several times, and the responses are aggregated and sometimes weighted for different levels of expertise. Participants do not interact directly. Oftentimes, participants are asked at the onset to contribute to the development of the agenda. The logic behind this method is that combined numerical estimates of participants’ views will generally result in more reliable estimates than estimates from a single person. This method is relatively low cost, but it lacks the potential benefits of face-to-face interactions and exchanges of information.

Nominal Group Technique The Nominal Group Technique (NGT) was initially developed for committee decisionmaking. The primary objective of this approach is to structure the group interaction by having each individual record his/her ideas independently and privately, presenting each idea in a round-robin format, discussing each idea in turn by the group, and having individuals privately record their judgments. Additional discussion and/or voting may take place, and these responses are aggregated statistically to derive the group judgment. Variations of this technique are often used; e.g. individuals are asked to express their views privately via mailed questionnaire before the group meets. This technique has the advantages of allowing all ideas to be presented and of potentially minimizing individuals’ inhibition about sharing ideas. A facilitator is present to ensure that everyone has an opportunity to participate and that no one person dominates the discussion.

Download the original PDF