Pharmacovigilance Observed: Why Watchful Waiting will Work

Kowalski CJ and Mrdjenovich AJ^*

Institutional Review Board for Health and Behavioral Sciences, Office of Research, University of Michigan, USA

Corresponding Author:

Mrdjenovich AJ
540 East Liberty St.
Suite #202 Ann Arbor
Michigan 48105, USA
E-mail: chuckk@umich.edu

Received Date: July 22, 2014; Accepted Date: October 30, 2014; Published Date: November 10, 2014

Citation: Kowalski CJ, Mrdjenovich AJ (2015) Pharmacovigilance Observed: Why Watchful Waiting will Work. J Clin Diagn Res 3:114. doi:10.4172/2376-0311.1000114

Copyright: ©2014 Kowalski CJ, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Visit for more related articles at JBR Journal of Clinical Diagnosis and Research

Abstract

This paper reviews issues of pharmacovigilance, defined as “all scientific and data gathering activities relating to the detection, assessment, and understanding of adverse events”, including pharmacoepidemiologic studies, which are “undertaken with the goal of “identifying adverse events and understanding, to the extent possible, their nature, frequency, and potential risk factors”. Some of these activities could be carried out more effectively. Particular attention is paid to the use of prospective observational studies and registries as important tools in this regard. Indeed, in studying drug safety, there are many situations in which the use of observational research has definite advantages over the randomized controlled clinical trial. While some controversy persists about the usefulness of observational research in the study of beneficial, intended effects, fewer objections have been raised about the usefulness of observational research in the study of drug safety, i.e., in assessing harmful, unintended, usually unanticipated outcomes.

Keywords

Drug safety; Adverse drug reactions; Pharmacoepidemiology; Observational studies; Registries

Introduction

Vigilance is alert watchfulness. To be vigilant about the safety of marketed drugs is to practice pharmacovigilance[1]. We need to do a better job of this. Prescription drugs are one of the leading causes of death in the United States [2]. In fact, medications taken according to doctors’ instructions kill more Americans than diabetes or Alzheimer’s disease. An estimated 100,000 Americans die each year from prescription medications; this corresponds to more than 270 deaths each day—more than twice the number of deaths caused by automobile accidents. An additional 1.5 million Americans are injured by their medications each year, so severely that they require hospitalization [3]. Such statistics led to warnings that the time had come to “act on drug safety” [4]. This paper reviews some of the activities involved in pharmacovigilance, and indicates how we can do some of these better. We begin with an outline of how the safety of marketed drugs is currently monitored.

Identifying drug safety problems: existing activities and resources

Pharmacovigilance activities center on safety signals: The WHO (2005) defines a signal as “reported information on a possible causal relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously [5].” These signals, representing an excess of adverse events (“noxious and unintended and occurs at doses used in man for prophylaxis, diagnosis, therapy, or modification of physiologic functions” [WHO, 1984]) compared to what would be expected from the use of a drug, can come from post-marketing data arising in the context of a formal study, but can even be triggered by a single well-documented case report should the event be extremely rare in the absence of drug use. After a signal is identified, it is studied further, to determine whether it represents a safety risk and what additional action is warranted.

The FDA’s primary resource for identifying drug safety problems is the Adverse Event Reporting System, MED Watch [6]. Voluntary reports of any problem associated with any FDA-regulated product can be made by mail, phone or on-line using a single one-page reporting form by patients, consumers, health professionals and drug companies. It receives some 400,000 reports annually, primarily from drug manufacturers who are required to report serious, unexpected safety events within fifteen days. A smaller proportion of these socalled adverse drug reactions (ADRs) come from health care providers and patients. This system is plagued by massive under-reporting and an inability to distinguish between drug-induced and naturally occurring adverse events. It has been estimated that only about 1% of all ADRs and about 10% of all serious ADRs are reported [7]¹. And, underreporting is only part of the problem: the ADRs that are reported do not accurately reflect the universe of all ADRs [8]. While spontaneous, passive reporting systems like MED Watch can be effective in discovering unusual or rare ADRs that occur with the use of medications, they do not reliably detect ADRs that occur widely separated in time from the original use of the drug, or that represent an increased risk of an ADR that occurs commonly in populations not exposed to the drug. For a comprehensive list and discussion of both the strengths and weakness of spontaneous reporting systems [9,10]. Kessler [11] himself noted, some fourteen years after he introduced the MED Watch system, “One key reason drugs may be used for years by millions of patients before risks become evident is that the United States has no active drug-surveillance system” [11]. Active drug surveillance occurs when “Participants are asked about the occurrence of specific adverse events in structured questionnaires or interviews, or predefined laboratory or other diagnostic tests are performed at prespecified time intervals” [12]. They also defined passive systems: “Participants are not specifically asked about or tested for the occurrence of adverse events. Rather, adverse events are identified based on patient reports made on their own initiative.” Active methods are more likely to identify adverse events than passive methods [13]. For a comprehensive review of the development of active drug safety surveillance systems – including potential component data sources – Platt et al. [14].

It is perhaps not surprising, then, that many have called into question the effectiveness of the FDA’s (passive) approach to drug safety. Brewer and Colditz [15] pointed to the limitations of MEDWatch, and thought that “methods to evaluate ADRs using data from clinical trials, medical records, and computerized databases of medication users and nonusers must be developed to complement spontaneous reporting systems [15]. Without these methods, potentially important ADRs will remain undetected, and spurious associations between adverse outcomes and medications will remain unchallenged”.

Drug safety issues: who will oversee and who will pay?

Wood et al. pointed to the need for an independent (independent of both the pharmaceutical industry and the FDA) drug safety board [16]. Such a board would, among other things, oversee formal prospective mandatory post-marketing surveillance; gather and analyze comparative data allowing comparison of the safety profiles of different drugs that are used for the same purpose; and validate the surrogate endpoints, often used in pre-marketing studies, with evidence that the drug reduces morbidity and/or mortality. Ray and Stein went a step further, calling for the establishment of a reformed regulatory authority with three distinct functions: drug approval, postmarketing studies, and drug information [17]. They envisioned that these tasks would be the responsibility of three independent but cooperative centers within a unified agency, but recognized that other administrative structures were possible. Griffin et al. pointed to the need for a distinct entity with the mission of monitoring all new and existing drugs after marketing in a proactive, systematic way [18]. They thought that this entity should be independent of the pharmaceutical industry, but “could work cooperatively with federal agencies, including the FDA, Agency for Healthcare Research and Quality, and Centers for Disease Control and Prevention; with groups representing practitioners; with pertinent consumer organizations; and with other stakeholders”. Fontanarosa et al. detailed the many inadequacies of the MEDWatch system, and argued that these inadequacies stemmed from the fact that “drug manufacturers are largely responsible for collecting, evaluating, and reporting data from postmarketing studies of their own products” [19]. Nor did they think that the FDA would be a wise choice to entrust with drug safety: “… the drug approval process must be decoupled from the postmarketing safety and surveillance system. It is unreasonable to expect that the same agency that was responsible for the approval of drug licensing and labeling would also be committed to actively seek evidence to prove itself wrong”. Strom suggested that new drugs with potential safety problems be given only conditional approval; and that there be a mechanism in place to ensure that industry commitments to conduct post-marketing safety studies were honored [20]. He based these recommendations on the observation that, “51% of drugs have label changes because of major safety issues discovered after marketing, 20% of drugs get new black box warnings after marketing, and 3% to 4% of drugs are ultimately withdrawn for safety reasons”. The Institute of Medicine (IOM, 2007) not only agreed with the notion of conditional approval, they suggested the use of a symbol² to be included in the labeling that would signal the still-to-be-determined safety profile of the drug [21]. They thought this symbol should be used for 2 years and that, during that time, direct-to-consumer advertising would be limited. We would suggest that no definite time period be attached here: If drugs received ‘conditional approval’ until such time as adequate safety data were available, this would provide an incentive for the industry to collect such data. Roth-Cline considered the idea that safety problems could be reduced if stricter statistical criteria for documenting efficacy were required [22]³. While it seems intuitively clear that shrinking required levels of significance will require more time to attain significance, thereby giving more time for potential problems to appear, she concluded that there were better ways to assess drug safety than adopting a small P-value for efficacy. Avorn also doubted this approach would work, albeit for different reasons, and suggested instead a two-step approval process, similar to that followed in several European countries, that requires a review two- to three-years after the initial approval [23]. Again, we believe it possible and desirable to continue the conditional approval designation until such time that the FDA (or the independent drug safety board) was satisfied that the drug had a favorable risk/benefit ratio.

Furberg et al., in summarizing these suggestions, thought the recommended solutions were limited in scope and generally failed to recognize that “the US Congress holds the key to a markedly improved and comprehensive national drug safety program” [7]. They urged Congress⁴ to implement the following five recommendations: (1) give the FDA more direct legal authority to pursue violations, (2) authorize the adoption of a conditional drug approval policy, at least for selected drugs, (3) provide additional financial resources to support the safety operations, (4) mandate a reorganization of the agency (establish a Center for Drug Safety) with emphasis on strengthening the evaluation and proactive monitoring of drug safety, and (5) require broader representation of safety experts on the FDA’s advisory committees.

The Institute of Medicine (IOM, 2007) also recognized Congress as the necessary driving force behind initiatives for improving drug safety, outlining a number of action steps that Congress should take in order to fulfill a lifecycle approach⁵ to the study, regulation, and communication about the risks and benefits of drugs [21]. Such an approach recognizes that all drug approvals are provisional, and that there is a responsibility to continually gather, analyze and interpret appropriate data. We are currently neglecting this responsibility. Psaty and Furberg provide a thoughtful summary of the problems we now face and what needs to be done to alleviate them [24].

“For an approved drug, the FDA currently engages in protracted negotiations with manufacturers (1) to change a product label, (2) to conduct patient or physician education, (3) to limit advertising to patients or physicians, (4) to modify approved indications, (5) to restrict use to selected patients, (6) to complete post-marketing agreed on at the time of approval, (7) to conduct additional post-marketing studies or trials, and (8) to suspend marketing or immediately withdraw a drug. The FDA has recently claimed to lack adequate authority in these areas. … Congress needs to provide the FDA with the necessary authority and also to create an independent Center for Drug Safety with new authority and funding. … Provisional approval and regular repeated review would provide valuable opportunities to reevaluate risk and benefit.”

It should be possible to be unambiguous about which powers the FDA does and does not have; Congress needs to provide the FDA with the necessary means and mandates to do its job of protecting the public. It would seem clear that Congressional action is appropriate and long overdue. Changes need to be made, and as stated by Avorn, “all that is lacking is the political will to implement them” [23]. There remains the question of how best to accomplish this. We turn our attention to this question next.

Methods for Studying Drug Safety

It is important to realize that the aim here is to complement, not replace, spontaneous reporting systems. Systematic study of reported ADRs, using so-called data mining techniques, can pinpoint an excess of adverse events associated with the use of a drug, and for uncovering patterns, time trends and events associated with drug-drug interactions. Data mining methods usually are based on comparing the fraction of all reports of a particular event (e.g., kidney failure) for a specific drug, the so-called observed reporting fraction, to the fraction of reports for the same event for all drugs, the expected reporting fraction. The data mining score quantifies the mismatch between the observed and expected for a given product-event combination [25]. This approach is called the proportional reporting ratio method. It has been extended for use in longitudinal databases other than those based on spontaneous reporting by Zorych et al. [26]. Other data mining methods exist and may be considered for use in various applications, e.g., the Multi-Item Gamma Poisson Shrinker algorithm (Szarfman et al. apply this to the FDA’s spontaneous reports database), and the neural network approach [27-30].

These techniques will not prove able to establish causality; but they can be useful in identifying safety signals worthy of further investigation⁶. A good overview of this process is given by Wilson et al. [30]. Next steps once a signal has been identified were discussed by Waller and Lee [31].

These post-marketing safety studies can be designed and categorized in a variety of ways, each having its own peculiar set of strengths and weaknesses; the choice among them will be (or, at least, should be) driven by the purpose of the study [32-34]. In many situations, an observational study will prove advantageous. Before taking a closer look at observational studies, it is worth pausing to note that when studying drug safety, observational studies may avoid some of the usual complaints made about them, including the extreme view that they are in every respect inferior to randomized, controlled clinical trials (RCTs). Perhaps the biggest potential limitation of observational studies of efficacy is confounding by indication, the idea that persons choosing (perhaps with the advice and consent of their physician) one treatment (medical management) over another (surgery) may do so because of the seriousness of their condition (unlikely to survive surgery). In such cases, it may be difficult to determine if differences in outcomes are due to the treatments themselves or to the other factors that led to the treatment choices. When the outcome of a study is an ADR – recall that adverse events are always unintended, their risks are either not known at all or they were unpredictable in a particular subgroup of patients – the choice of drug is not made taking the (unintended and possibly unknown) ADR into account⁷. As stated by the IOM, “In the evaluation of unintended or previously unsuspected effects of drugs, however, observational safety studies are less likely than studies of known effects to be influenced by confounding by indication” [34]. The reasons for this have been spelled out in a number of publications, e.g., Miettinen, Grobbee and Hoes, and Vandenbroucke [35-37]. Often, it comes down to the simple observation that the choice of the drug was not made with the unanticipated ADR in mind. If there is a factor thought to increase the risk of an identified ADR, the study may have to be restricted to those not having the factor: As pointed out by Psaty et al., “Confounding by indication simply cannot occur among subjects who do not have the potential confounding factor, and when the indication can be measured accurately, restriction of the analysis to subjects without the indication excludes the possibility of confounding by that indication” [38]. Note that a special case of this restriction occurs when two drugs that have the same indications are compared.

In addition to avoiding a serious negative, there are a number of positives. The IOM noted that some observational studies of safety may have distinct advantages over randomized, controlled trials (RCTs): “They can often be much larger than randomized controlled trials, involve longer patient followup, include a broader diversity of patients and care settings … observational studies evaluating infrequent outcomes that occur long after exposue can sometimes provide higher-quality safety evidence than randomized controlled trials” [34]. They recommended that “the FDA should require a randomized controlled trial be conducted to provide additional evidence about an approved drug’s efficacy and safety only [emphasis added] when (i) uncertainty about the risk-benefit balance cannot be made based either on the existing evidence or on evidence from new observational studies, and (ii) the trial is properly designed and implemented to reduce uncertainty about the risk-benefit balance sufficiently for a reasonable policy decision to be made”. It would appear, then, that RCTs for establishing the safety of a marketed drug should be the exception, rather than the rule. As stated by Vandenbroucke, [39].

“Thus, to understand the full spectrum of adverse events – those that occur late, that were not known beforehand, and that are rare but nevertheless serious – and to be able to investigate the true incidence of known adverse advents in circumstances of actual prescribing, welldesigned observational studies will always be necessary.”

The FDA recognizes specific classes of observational studies, viz., (1) Pharmacoepidemiologic studies, and (2) Surveys, which will often suffice to monitor a drug’s safety [1]. Pharmacoepidemiologic studies can be of various designs, including cohort, case-control, nested casecontrol, case-crossover, or other models. These are distinguished from case series in that they are designed to assess the risk attributable to a drug exposure. In particular, they follow a written protocol and usually include a control group, allowing the testing of pre-specified hypotheses. An epidemiologic approach to drug use and safety allows assessment of how drugs function in the real world [40].

Surveys can be used to gather a variety of safety-related information, including data concerning safety signals, awareness of labeled adverse event warnings, and use of a product as labeled, particularly when the indicated use is for a restricted population or numerous contraindications exist. Like a registry, a survey can be initiated by a sponsor at any time.

Thus, there are a number of possible forms that pharmacovigilance may assume, depending upon what one is watching, and what one is looking for. Our attention will be on designed, prospective observational cohort studies (including open label extension studies), and registries⁸. These have the desirable properties that they are carried out in the real world (i.e., are more pragmatic than explanatory – Kowalski,), can involve large numbers of heterogeneous patients, and can be conducted over long periods of time, allowing the discovery of slow-developing and/or rare ADRs that may occur in only select subgroups of the treated population [41]. We cannot use randomized clinical trials to study long term ADRs because we just cannot study a large enough group of people for a long enough period of time⁹. Randomization might be “nice” in that it would prevent confounding by indication, but there are ways to control for this and, as pointed out by Feinstein, “All of the other features usually associated with randomization are due not to the act of randomization, but to the act of planning for everything else that we do in the usual randomized trial [42]. We are careful in observing performance, detecting outcome events and collecting the data”. He also thought that, “Much of the major bias that one sees in nonrandomized comparisons is not simply because they were nonrandomized, but because the patients in the groups to be compared did not fulfill the same eligibility criteria on entering the study”.

Kowalski previously described some of the ways in which observational studies could be strengthened so as to prove useful even for the study of intended, desirable outcomes [43]. These include such things as being prospectively designed, and carefully structured, using a protocol. The basic idea is to mimic the many features of the RCT, short of randomization, that support its credence. This is in the spirit of Hlatky et al., who noted, “much of the high scientific quality of the modern RCT is due to strict attention to detail in its conduct, and not simply from the act of randomization per se” [44]. We believe that this fastidious approach can be largely successful when studying intended outcomes, but potential shortcomings are even further mitigated when studying harms – unintended, undesirable outcomes.

What, then, are the issues that a successful observational study will need to pay attention to in relation to PV? We cite two lists that cover the basics:

Chou et al.: For cohort studies, important factors include (i) assembly of an inception cohort, (ii) complete follow-up, (iii) appropriate assessment of potential confounders, (iv) accurate determination of exposures and outcomes, and (v) blinded assessment of outcomes [12].

Feinstein: Observational studies can be improved by insisting on the same scientific principles used in randomized trials to select groups and reduce bias. These principles are: (i) the use of admission criteria, (ii) the identification of variables important for stratification, (iii) attention to changes that might occur in intervening therapy, and (iv) attention to aspects of surveillance and detection which, in the best of randomized trials, do receive adequate attention today [42].

Application of these principles to design observational studies of drug safety should provide the quantity and quality of information necessary to establish meaningful safety profiles for drugs. In particular, when designing observational studies of drug safety, we should (C (j) refers to the jth factor listed by Chou et al.; F (k) to Feinstein’s k^th):

C (i) and F (i): Establish appropriate and unambiguous inclusion criteria for the study; in particular, ensure that the patients in the groups to be compared fulfill the same eligibility criteria on entering the study. Typical eligibility criteria include age, gender, diagnosis, comorbid conditions and concomitant medications [39].

C (ii): Complete follow-up is the aspirational goal, but when data points are missing or subjects drop out of the study altogether, an attempt to explain what happened should be made whenever possible. It is important to ensure that the same procedures to minimize nonresponse are used in the exposed and unexposed groups.

C (iii) and F (ii): All suspected confounding factors should be assessed. In addition, following Moses, the record should include why the patient was given the drug selected. This information may not always be readily available, but it should prove to be a powerful adjustment variable, and is worthy of considerable effort to obtain.

F (iii): Drug therapies, environmental factors, and even study methods can change over time. Any such changes must be part of individuals’ data records if they are to be properly assessed.

C (iv) and F (iv): Explicit definitions of exposures (drug/dose/ schedule/indication) and outcomes (ADRs) need to be made and observed. Disease outcomes require adequately detailed diagnostic criteria [39].

C (v): Whenever possible and appropriate, it is best if ADRs are verified without knowledge of drug history [39].

Further elaboration of these (and some other) ways to design informative observational studies is available ([39] see also http:// www.effectivehealthcare.ahrq.gov/ehc/products/31/103/ DEcIDEPharmacovigilanceFinal.pdf for a research report from AHRQ)¹⁰. Of particular interest is the work of the Observational Medical Outcomes Partnership (OMOP, www.omop.org) who, over the course of a 5-year project, quoting from its website, successfully achieved its aims to (1) conduct methodological research to empirically evaluate the performance of various analytical methods on their ability to identify true associations and avoid false findings, (2) develop tools and capabilities for transforming, characterizing, and analyzing disparate data sources across the health care delivery spectrum, and (3) establish a shared resource so that the broader research community can collaboratively advance the science. The results of OMOP’s research have been widely published and detailed references are available on the website. This work has been continued by the Observational Health Data Sciences and Informatics (OHDSI, pronounced “Odyssey” collaborative. For more information, go to http://ohdsi.org.

Open label extension studies

Before proceeding to a discussion of registries, we want to pause to consider a particular kind of observational study (not always recognized as such) that has often been employed in the name of drug safety assessment. Open label extension studies (OLESs) typically follow (and are thus an extension of) a double blind, placebo controlled, randomized clinical trial of a new medication. Upon completing their participation in the trial, subjects are invited to continue in the study for a further period of time, during which they are given the investigational drug. This apparently simple idea has generated a perhaps surprising number of strong reactions pro and con. On the nay-saying side, Schneider wrote, “Claims of efficacy and safety based on open label extensions of clinical trials create misinformation and unrealistic expectations about important treatments, can undermine otherwise compelling trials evidence, and have no place in academic journals” [45]. His choice of words in describing OLESs as “case series of the survivors of the trials” leaves no doubt that he lacks confidence in the scientific rigor and representativeness of these studies. On the other end of the spectrum, Casarett et al., who were concentrating mainly on the ethical questions¹¹ raised by OLESs, thought that “there are reasons to look more favorably on, and perhaps to require, open label extension studies” [46]. Thus the spectrum of opinion ranges from misinformation to requiring that such studies be done. A balanced view was given by Day and Williams [47]. While recognizing that, to many (e.g., Taylor and Wainwright, 2005), these studies were more a marketing tool than a serious attempt to generate safety data, they thought that OLESs “do have a legitimate but limited place in the clinical development of new medicines [48]. Increased human exposure to a new medicine under reasonably controlled circumstances is an acceptable rationale for an open-label extension study, and a useful activity to increase the knowledge of the safety profile of a new medicine. However, this goal is increasingly being achieved by means other than open-label extension studies”. One of the reasons other methods are being used more often is that the size of the OLES is by definition limited by the size of the parent RCT, and this will generally be inadequate to detect rare ADRs. Another limitation is the lack of representativeness of OLES participants. They will have already volunteered for the parent RCT (the well-known “volunteer effect” arises from the fact that those who volunteer to participate in a given trial will often differ from those declining to participate), and their willingness to volunteer again squares the magnitude of the volunteer effect, and already is a strong indication that they are able to tolerate the drug. It is probably not a stretch to suggest that OLES participants will not be representative of the original study population, let alone the target population of all those who will eventually be prescribed the drug. In addition, OLESs often are time-limited; continuing only until marketing approval is given. Thus OLESs suffer from considerable scientific limitations. There are other concerns involving the true motivation for embarking on them. When sponsors describe an OLES, they will often use phrases like “long-term safety” but Day and Williams point out that, “other possible motives, whether overt or not, include: collecting data for pharmacoeconomic analyses; familiarizing prescribers and patients with the use of the product and, thereby, achieving marketing objectives; … generating pressure on funding providers (e.g., government and hospitals) to purchase the drug (a further marketing objective)” [47]. Thus OLESs will often be sponsored by the marketing division, rather than the research arm, of the company, and will often be used for a “me-too” drug looking to capture market share in a crowded field. Day and Williams note, “to many sponsors the marketing benefit of having clinician researchers who are often ‘opinion leaders’ continuing to use the new drug is a compelling motive for performing an open-label extension study” [47]. This places OLESs too close for comfort to so-called “seeding studies” [6], studies that are designed largely to expose clinicians to new medicines or to expand the marketing potential and patent life of a medicine by identifying new clinical indications with little or no scientific warrant [6]. These are described in more detail – and thoroughly discredited – by Relman and Angell, and Sox and Rennie [49,50]. On the other hand, when strictly advertising/marketing motives can be ruled out, a valid clinical question might be: What is the long-term safety profile for patients who can tolerate the drug? In this case, an OLES might contribute some useful knowledge so should not be summarily dismissed. Often, however, even this question can be answered in ways that are not subject to volunteerism, small samples, and time limitations. The strengths of prospective observational studies include that patients have been treated entirely in ways agreed upon with their physicians and, often, many patients are available and time is not an issue.

Registries

A registry (FDA, 2005) “is an organized system for the collection, storage, retrieval, analysis, and dissemination of information on individual persons exposed to a specific medical intervention who have either a particular disease, a condition (e.g., a risk factor) that predisposes [them] to the occurrence of a health-related event, or prior exposure to substances (or circumstances) known or suspected to cause adverse health effects” [1]. Whenever possible, a control or comparison group should be included, i.e., individuals with the disease or risk factor that are not treated or are exposed to medical interventions different from the intervention under consideration.

The Agency for Healthcare Research and Quality (AHRQ) defines a patient registry for evaluating outcomes as, “an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves a predetermined scientific, clinical, or policy purpose(s).” Solomon et al. noted that the development of registries can be traced back at least as far as 1086 to the preparation of England’s “Domesday Book.” They suggested the simplified definition: “A registry is a data base of identifiable persons containing a clearly defined set of health and demographic data collected for a specific public health purpose” [51]. Note that both definitions emphasize that the data are to be collected for a specific purpose. According to Solomon et al., the first thing one should look at when evaluating proposals for new registries is the stated purpose of the registry. In any case, patient registries collect clinical outcomes in populations defined by a particular disease, condition, or exposure [51]¹². Clinical data are collected prospectively for specific research purposes using active methods to identify outcomes. Registries can be particularly useful in identifying longterm or uncommon ADRs, and they may be supplemented by information from other sources, including administrative databases. These databases differ from registries in that they contain information routinely collected during clinic, hospital, laboratory or pharmacy encounters, and not for specific research purposes. The advantage of registries is that their observational and inclusive design may allow for surveillance of a diverse patient population that can include groups typically excluded from initial clinical trials such as pregnant women, minorities, older patients, children, or those with co-morbidities and taking concomitant medications. For disease registries, it will pay dividends to collect data on as many factors that are thought to affect prognosis as is feasible. Hlatky et al. thought that a suitably designed registry “Answers the questions that the patient (with coronary artery disease) asks the physician: ‘How long am I likely to live? Is there a difference depending on how I am treated? How big a difference can I expect?’ We believe that an observational database can contribute important information to answer these questions, for it is designed to account for the myriad of factors known to affect prognosis, not only the treatment given, by analyzing the experience within the entire spectrum of patients seen in practice” [44]. Ellwood built on these ideas, suggesting the development of a national program for “outcomes research,” that would involve a permanent national medical database, linking medical interventions with health outcomes, including ADRs [52]. This approach, based on the idea that we should continue to learn about both efficacy and safety by tracking the results of our interventions, deserves much thought and thoughtful application, but is beyond the scope of the present discussion. A possible outcome of efforts in this direction was described by Berry: “Each MD could have a computer that is part of a national network … Each patient’s characteristics, diagnosis, treatment and follow-up visits would be entered into a national data base. These data bases would be open to the public. Medical journals would publish periodic summaries and analyses of the data bases. When a new therapy is introduced, control data of good quality will be accessible in the data base. Information concerning comparability with current patients will also be available” [53]. In another forum, Berry recognized that registries did have certain disadvantages, but thought these signaled only that “care is necessary when using databases. But learning from databases is possible. And they can play a valuable role in medical research. In our view, establishing and analyzing databases is the future of medical research” [54].

Significant strides in this direction have already been made. We cite two examples of already established data bases that can be expected to impact healthcare generally and, in particular, drug safety issues, viz., PCORnet and i2b2.

PCORnet, a database assembled by PCORI (the Patient-Centered Outcomes Research Institute, www.pcori.org), combines patient medical records from a number of sites across the country, that may contain as many as 30 million patient records by 2015. These data will facilitate comparative effectiveness research and also provide a way to pinpoint patients with specified characteristics satisfying the inclusion/ exclusion criteria for proposed clinical trials.

Informatics for Integrating Biology and the Bedside (www.i2b2.org) is an NIH-funded Center for Biomedical Computing that seeks to combine clinical data and genomic information in order to facilitate targeted therapies for individual patients with diseases having genetic origins.

Concluding Thought

In an earlier work, Kowalski provided a number of quotations from and references to arguments that observational studies, including registries, could provide much useful information concerning treatment effectiveness, i.e., they could be used for meaningful comparisons of intended, beneficial outcomes.We agree with these sentiments. For the reasons given earlier, however, when studying drug safety – comparing unintended, unanticipated adverse reactions – there is even less reason to suspect that observational studies might mislead; and solid reasons for believing they offer benefits that other study designs cannot. Watchful waiting will work. We need to mobilize enough political power to see that it is given a chance to do so [43,52-56].

Acknowledgements

¹One might think reporting would be better in research contexts, where detailed record keeping should be the norm. Shamoo studied the potential magnitude of adverse events in the United States among human subjects enrolled in research and characterized adverse events reporting as “The tip of an iceberg.”

²In the UK a black triangle appearing after the trade name of a drug indicates that the medication is new to the market, or that an established drug is being used for a new indication, or via a new route of administration. We might want to use a different symbol. Doing a simple Google search on black triangle shows why.

³That this strategy was in fact considered by planning committees for cardiovascular trials, in consultation with the FDA, was communicated to Roth-Cline by the well-known statistical consultant David DeMets, who also served on Ms. Roth-Cline’s thesis committee.

⁴The solution must come from Congress, for it is they who control both the FDA’s legal authority and its funding.

⁵A lifecycle approach to drug risk and benefit requires continuous availability of new data and ongoing, active reassessment of the risks and benefits to drive regulatory action, and regulatory authority that is strong both before and after approval.

⁶Data mining has been disparaged by some as “fishing expeditions.” There is no gainsaying the fact that a net is (purposefully) being cast in the hope of capturing as many interesting fish as possible. Those not meriting further study are thrown back. Others require – and receive – further scrutiny.

⁷If an adverse effect of a drug is known, doctors might avoid prescribing it for patients who are at higher risk for the effect. Confounding by indication might still obtain here, but there are methods available to correct for this: matching, stratification, restriction, and multivariate adjustment [24].

⁸Other pharmacoepidemiologic study designs and surveys will often prove useful, but will not be considered further here

⁹The ‘rule of three’ is useful here. If P (ADR)=1/N, then to be 95% confident that one will observe at least one ADR requires that we study 3N subjects. Thus if P(ADR)=1/10 000, we need to study 30 000 subjects.

¹⁰The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines are to observational studies what the CONSORT statement is to RCTs. The complete stories of each are available at www.strobe-statement.org and www.consort-statement.org respectively.

¹¹These include possible coercion to participate in the parent RCT by promise of continued free access to the study medication at trial’s end; and an inability to provide informed consent if the blind in the parent RCT is not broken when entering the OLES.

¹² Schmitt-Egenolf argued that disease registries, rather than product registries, and especially those managed by healthcare professionals, are more likely to be successful in collecting unbiased, interpretable long-term safety data as they allow objective comparisons across generations of drugs, devices or other interventions. At the risk of repeating ourselves, what we want to get out of the registry should drive what we put into it.

References

1. FDA (2005) Guidance for Industry: Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment.
2. Lazarou J, Pomeranz BH, Corey PN (1998) Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA 279: 1200-1205.
3. Petersen M (2008) Our Daily Meds. New York: Picador.
4. Moore TJ, Psaty BM, Furberg CD (1998) Time to act on drug safety. JAMA 279: 1571-1573.
5. WHO (World Health Organization) (2005) The Safety of Medicines. Geneva: WHO.
6. Kessler DA (1993) Introducing MEDWatch. A new approach to reporting medication and device adverse effects and product problems. JAMA 269: 2765-2768.
7. Furberg CD, Levin AA, Gross PA, Shapiro RS, Strom BL (2006) The FDA and drug safety: a proposal for sweeping changes. Arch Intern Med 166: 1938-1942.
8. GAO (2000) Surveillance Systems for Adverse Events and Medical Errors. GAO/T-HEHS-00-61.
9. Meyboom RH, Egberts AC, Gribnau FW, Hekster YA (1999) Pharmacovigilance in perspective. Drug Saf 21: 429-447.
10. Poluzzi E, Raschi E, Piccinni C, De Ponti F (2012) Data mining techniques in pharmacovigilance: Analysis of the Publicly Accessible FDA Adverse Event Reporting System (AERS). In Karahoca A (ed) Data Mining Applications in Engineering and Medicine.
11. Kessler DA (2007) Drug safety reform at the FDA–Pendulum swing or systematic improvement? NEJM 356: 1700-1702.
12. Chou R, Aronson N, Atkins D, Ismaila AS, Santaguida P, et al. (2010) AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. J ClinEpidemiol 63: 502-512.
13. Bent S, Padula A, Avins AL (2006) Brief communication: Better ways to question patients about adverse medical events: a randomized, controlled trial. Ann Intern Med 144: 257-261.
14. Platt R, Madre L, Reynolds R, Tilson H (2008) Active drug safety surveillance: a tool to improve public health. Pharmacoepidemiol Drug Saf 17: 1175-1182.
15. Brewer T, Colditz GA (1999) Postmarketing surveillance and adverse drug reactions: current perspectives and future needs. JAMA 281: 824-829.
16. Wood AJ, Stein CM, Woosley R (1998) Making medicines safer--the need for an independent drug safety board. N Engl J Med 339: 1851-1854.
17. Ray WA, Stein CM (2006) Reform of drug regulation--beyond an independent drug-safety board. N Engl J Med 354: 194-201.
18. Griffin MR, Stein CM, Ray WA (2004) Postmarketing surveillance for drug safety: surely we can do better. ClinPharmacolTher 75: 491-494.
19. Fontanarosa PB, Rennie D, DeAngelis CD (2004) Postmarketing surveillance--lack of vigilance, lack of trust. JAMA 292: 2647-2650.
20. Strom BL (2006) How the US drug safety system should be changed. JAMA 295: 2072-2075.
21. IOM (Institute of Medicine) (2007) The future of drug safety: promoting and protecting the health of the public. Washington DC: The National Academies Press.
22. Roth-Cline MD (2006) Clinical trials in the wake of Vioxx: requiring statistically extreme evidence of benefit to ensure the safety of new drugs. Circulation 113: 2253-2259.
23. Avorn J (2006) Evaluating drug effects in the post-Vioxx world: there must be a better way. Circulation 113: 2173-2176.
24. Psaty BM, Furberg CD (2005) COX-2 inhibitors--lessons in drug safety. N Engl J Med 352: 1133-1135.
25. Evans SJ, Waller PC, Davis S (2001) Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 10: 483-486.
26. Zorych I, Madigan D, Ryan P, Bate A (2013) Disproportionality methods for pharmacovigilance in longitudinal observational databases. Stat Methods Med Res 22: 39-56.
27. Szarfman A, Machado SG, O'Neill RT (2002) Use of screening algorithms and computer systems to efficiently signal higher-than-expected combinations of drugs and events in the US FDA's spontaneous reports database. Drug Saf 25: 381-392.
28. Bate A, Lindquist M, Edwards IR, Olsson S, Orre R, et al. (1998) A Bayesian neural network method for adverse drug reaction signal generation. Eur J ClinPharmacol 54: 315-321.
29. Trifirò G, Pariente A, Coloma PM, Kors JA, Polimeni G, et al. (2009) Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor? Pharmacoepidemiol Drug Saf 18: 1176-1184.
30. Wilson AM, Thabane L, Holbrook A (2004) Application of data mining techniques in pharmacovigilance. Br J ClinPharmacol 57: 127-134.
31. Waller PC, Lee EH (1999) Responding to drug safety issues. Pharmacoepidemiol Drug Saf 8: 535-552.
32. Glasser SP, Salas M, Delzell E (2007) Importance and challenges of studying marketed drugs: What is a phase IV study? Common clinical research designs, and self-reporting systems. J ClinPharmacol 47: 1074-1086.
33. Berlin JA, Glasser SC, Ellenberg SS (2008) Adverse event detection in drug development: recommendations and obligations beyond phase 3. Am J Public Health 98: 1366-1371.
34. IOM (Institute of Medicine) (2010) Ethical Issues in Studying the Safety of Approved Drugs: A Letter Report. Washington DC: The National Academies Press.
35. Miettinen OS (1983) The need for randomization in the study of intended effects. Stat Med 2: 267-271.
36. Grobbee DE, Hoes AW (1997) Confounding and indication for treatment in evaluation of drug treatment for hypertension. BMJ 315: 1151-1154.
37. Vandenbroucke JP (2004) When are observational studies as credible as randomised trials? Lancet 363: 1728-1731.
38. Psaty BM, Koepsell TD, Lin D, Weiss NS, Siscovick DS, et al. (1999) Assessment and control for confounding by indication in observational studies. J Am GeriatrSoc 47: 749-754.
39. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, et al. (2007) Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and elaboration. Epidemiology 18: 805-835.
40. Guidelines for Good Pharmacoepidemiology (2004) International Society for Pharmacoepidemiology.
41. Kowalski CJ (2010) Pragmatic problems with clinical equipoise. PerspectBiol Med 53: 161-173.
42. Feinstein A (1984) The role of observational studies in the evaluation of therapy. Stat Med 3: 341-345.
43. Kowalski CJ (2013) When ethics precludes randomization: put prospective, matched-pair observational studies to work. PerspectBiol Med 56: 184-197.
44. Hlatky MA, Lee KL, Harrell FE Jr, Califf RM, Pryor DB, et al. (1984) Tying clinical research to patient care by use of an observational database. Stat Med 3: 375-387.
45. Schneider LS (2006) Open-label extension studies and misinformation. Arch Neurol 63: 1036.
46. Casarett D, Karlawish J, Sankar P, Hirschman KB, Asch DA (2001) Open label extension studies and the ethical design of clinical trials. IRB 23: 1-5.
47. Day R, Williams KM (2007) Open-label extension studies: do they provide meaningful information on the safety of new drugs? Drug Saf 30: 93-105.
48. Taylor GJ, Wainwright P (2005) Open label extension studies: research or marketing? BMJ 331: 572-574.
49. Relman AS, Angell M (2002) America's other drug problem: how the drug industry distorts medicine and politics. New Repub 227: 27-41.
50. Sox HC, Rennie D (2008) Seeding trials: just say "no". Ann Intern Med 149: 279-280.
51. Solomon DJ, Henry RC, Hogan JG, Van Amburg GH, Taylor J (1991) Evaluation and implementation of public health registries. Public Health Rep 106: 142-150.
52. Ellwood PM (1988) Shattuck lecture--outcomes management. A technology of patient experience. N Engl J Med 318: 1549-1556.
53. Berry DA (1989) Ethics and ECMO. Statistical Science 4: 306-310.
54. Berry DA (1993) A case for Bayesianism in clinical trials. Stat Med 12: 1377-1393.
55. Truog RD (1992) Randomized controlled trials: lessons from ECMO. Clin Res 40: 519-527.
56. Weinstein MC (1974) Allocation of subjects in medical experiments. N Engl J Med 291: 1278-1285.