Journal of Community Medicine & Health Education
Open Access

Self-controlled case series; Self-controlled risk interval; Critical appraisal; Pharmacoepidemiology

A Self-controlled Case Series (SCCS) or case series method is an observational epidemiological study design used to study the temporal association between a time varying exposure and a sudden onset of events. It’s a study design where the observed cases act as their own controls [1-5].

It was originally developed in 1995 for the evaluation of vaccine safety in pharmacoepidemiology and nowadays it is used in general epidemiology [1,6,7]. It is derivative of a cohort study design where individuals who have experienced the event of interest are observed over time, within a pre-established observation period, where the exposure history and the occurrence of the event of interest are identified. During the observation period, the risk period and the control period are established, which usually do not have the same length of time. The risk period is defined a priori and is considered as the time during or after exposure where the individuals are at a higher or reduced risk of the events of interest after an exposure. The control period constitutes any time within the observation period (before, after or between) the risk period [1,2,8]. For example, in a study to evaluate the risk of GBS after Human Papilloma Virus (HPV) Vaccination in England, the risk period started with vaccination (day 0) and ended on day 91 and a control period considered any time outside the risk period [9].

Normally, information from past events may help define the risk period [1]. For example, in the case of influenza vaccines and the development of Guillain-Barré syndrome (GBS), the prespecified risk period is 42 days after vaccination. This risk period was stipulated after a cluster of GBS cases occurred in the United States in 1976 after the population was vaccinated with A/New Jersey influenza vaccine [10]. During this risk period, it is presumed that there will be increased risk of the vaccinated individuals to develop the GBS and the time after as the control period [1,11].

A SCCS design is suitable for independent recurrent events as well as rare non-recurrent events. Some of the assumptions that make SCCS applicable include: the observed events are rare; the occurrence of an event must not alter the probability of subsequent exposure that is, if the events are consecutive, they should be independent; the occurrence of the event of interest must not censor or affect the observation period and that the observation period for each individual is independent of the event times. The latter can be violated in the event of death [1,2,4,5,7].

The SCCS estimates the relative incidence (RI) of events of interest in a defined time period after a transient exposure and the control period. The RI is the ratio of events of exposure in the risk period and the control period [1,6,11].

The main advantage of SCCS is that time invariant confounders that act on the standard rates are rigorously controlled. These factors are like: sex, location, genetics, socio economic factors and underlying health condition [1,5,12]. It is also cheaper and easier to collect data since the design only requires cases with the event of interest. Also, as an advantage is the methodological efficiency compared to the cohort design, for example, in the surveillance of adverse events after vaccination, a traditional cohort study may not be applicable for full coverage of the effects since it would be a challenge to recruit unvaccinated controls as the surveillance systems normally collect data on individuals who reported an adverse event. The SCCS requires a small sample of the population to be studied thus it can produce results that are clinically and statistically valid from just a few cases [1,4]. Some of the limitations of the SCCS design include: it does not produce estimates of absolute incidence but only estimates of relative incidence; as one of its assumptions, it requires that the probability of exposure is not affected by the occurrence of an outcome event, the effect of exposure has to be transient, variations over time are not adjusted for, it is prone to selection and information bias (normally uses information from surveillance databases). For non-recurrent events, the SCCS design works only when the event risk is small over the observation period [1,12].

The Self-controlled risk interval (SCRI) is a variant of a SCCS design where cases also act as their own controls. There is a risk period and a control interval. The control period is either before or after the exposure. The SCRI design has a reduced control interval, for example, in the case of vaccines, after the risk period is determined; a selected short period is selected before or after vaccination and close to the risk period. The reduced control interval is chosen to avoid time confounding issues like age and seasonality. The control interval is usually similar to the risk period [13,14]. For example, in a study to evaluate the adverse events following varicella vaccine in Taiwan, the risk interval was day 1 to day 42 after vaccination and the control interval was the period between day 43 to day 84 post vaccination [15].

The striking difference between the two study designs is the observation period. In the SCRI design, the index date is the vaccination date, and it is used to define the risk and control period. On the other hand, the SCCS design choses an observation period independent of the vaccination date and all cases are identified in the observation period [14].

Just like the SCCS, the SCRI design controls for fixed confounding factors like sex, race, genetic factors, preexisting health conditions and geographical locations. For time varying confounders like seasonality and age, there is a need for explicit adjustments, like in the case of vaccines, the adverse effects vary especially among children and can also vary over the follow up period and be confounded with other factors giving a false positive impression of the vaccine effects [13,14]. Unlike SCCS that includes both the vaccinated and unvaccinated individuals in the case of the analysis of adverse effects after vaccination, the SCRI design only includes vaccinated cases [15]. This is the primary strength of the SCRI design as it reduces bias that can arise among the vaccinated and unvaccinated cases [16].

SCCS and SCRI study designs are crucial in epidemiology where other study designs may not be quite suitable like the case of adverse events after vaccination in pharmacovigilance and thus, they require their quality evaluated and reported considering the basic assumptions of such study designs.

In evidence synthesis, the critical appraisal of the studies included is essential so as to access the credibility of the findings which is a consequence of the methodological rigor applied. It also helps to analyze the transparency and reproducibility of the published evidence. The absence of the critical appraisal is a barrier to the consumers of the evidence generated since there is hinderance in the ability to interpret the research findings which consider the strengths and weaknesses of the study in question [17,18].

To the best of our knowledge and after contacting two renowned researchers in SCCS/SCRI design, it came to our knowledge that no specific methodological assessment tool that has been reported and validated for use in evaluating the methodology of SCCS/SCRI designs. In 2018, Wachira and collaborators adapted the New Castle Ottawa Scale for quality assessment of cohort studies to evaluate SCCS study designs in their systematic review of the etiology of Guillain-Barré [19,20]. The present study aims at proposing a methodological quality assessment tool that can be used in quality assessment of SCCS/SCRI study designs.

The quality assessment tool is an adaptation of the Newcastle Ottawa Scale (19) for the quality assessment of cohort studies as presented by Wachira and colleagues and considers the basic assumptions of SCCS/ SCRI [21]. The proposal was adapted by V.W and assessed by H.M and M.R. Two other scholars used the tool in their systematic reviews studies [22,23].

There are two approaches that are widely used in the quality assessment of a primary study: the component approach that evaluates the individual items of each domain of the assessment tool and the composite approach that considers the quality scores of each of the domains and gives an overall score [24].

The proposed quality assessment tool is divided into three sections: Selection, Comparability and Outcomes as shown in Table 1. In the Selection section, the tool evaluates the representativeness of the selected cases in the SCCS/SCRI in relation to the total cases coming from the study population, the definition of the cases, the ascertainment of exposure and the absence of the outcome of interest at the start of the observation period. In the comparability section, confounding factors that vary over time are evaluated such as age and seasonality. The study should report if these factors were considered and if any adjustments were made in the analysis of the results. In the outcome section, there is evaluation of the ascertainment of the outcome of interest, clear indication of the risk and control periods and adequacy of the observation periods.

For the overall/composite assessment in each section, stars are allocated. For section 1, a maximum of three stars can be awarded to a study if the cases are representatives of the cases in the general population like in the case of vaccine safety studies, if there is a clear ascertainment of the exposure of interest and if there is a demonstration that the outcome of interest occurred in the observation period. In section 2, a maximum of two stars can be awarded to a study if it reports that time varying confounders were accounted for or if the follow up period was short enough to mitigate time confounding factors. In section 3, a study can be awarded a maximum of five stars if there is a clear way of confirming the outcome, if the risk and control periods are well stated, if the time in the risk and control periods was long enough for the outcome of interest to be analyzed and if there was complete follow up of the cases or accountability of the cases lost during follow up. In total, a study can be awarded a maximum of 10 stars.

In this first proposal of the assessment tool, the overall assessment of a study can be considered to be of “poor quality” if the study is awarded 3 stars, “moderate quality”, if the study is awarded up to 6 stars and “high quality” if the study gets 7 to 10 stars. This is just a suggestion of how to grade the overall quality of the study evaluated but the users of the tool are at liberty to decide how this can be applied in the studies analyzed depending on the specificities of the research question tackled in those studies.

There were no weights accorded to the sections in the tool thus giving the user the opportunity to be flexible in determining the overall quality of the study taking into consideration the need of the quality assessment and the nature of the evidence synthesis product in question. Table 1 shows the sections of the quality assessment tool, an explanation of the items considered and how to award stars to these items.

Table 1: The proposed quality assessment tool for the evaluation of the methodological quality of studies with a self-controlled case series and selfcontrolled risk interval designs

There are different quality evaluation tools for both quantitative and qualitative studies. At times, researchers modify existing tools or develop their own to meet their needs and this was our case. In 2018, we had to evaluate SCCS and SCRI included in a systematic review, and we couldn’t find an existing tool that could be used. This led us to adapting the NOS for cohort studies.

This is the first version of the proposed quality assessment tool for SCCS and SCRI. It has been used in five studies: one master’s degree dissertation, one published article, two projects of a scientific initiation program (not published), one thesis and one article submitted for publication [20-23]. It is based on the NOC that has never been published in a journal, thus has not undergone peer review. On the other hand, it is worth noting that the instrument was elaborated via Delphi method, has been tested in many systematic reviews and has been modified over this process [25]. Besides the critics the NOS receive, it has been used to evaluate innumerous studies included in published systematic reviews, thus confirming its validity and applicability.

In this first version of the tool, quality scores are recommended but no weighting scores have been attributed to the individual items in the domains evaluated. This allows flexibility of the evaluation process depending on the type of evidence synthesis being elaborated. Quality scores have previously been criticized as being poor predictors of the overall quality of a study especially when they are considered as a factor for the performance of a meta-analysis [26,27].

The SCCS and SCRI study designs are used in pharmacovigilance and are useful in analyzing rare events. They have been commonly used in studying populations after vaccine exposure like in the case of influenza vaccines and Human Papillomavirus vaccines [28,29]. Since 2020, the world has been facing the COVID-19 pandemic and in 2021, COVID-19 vaccines started being administered in all parts of the world. There has been temporal association of the COVID-19 vaccines or SARS-CoV-2 infection with the development of events like GBS, cardiovascular events like stroke, encephalitis, psychiatric events among others [30-32]. A huge number of both primary and secondary studies have been produced to report these events. The SCCS and SCRI could also be used in such cases and the use of the proposed quality assessment tool would be useful in ascertaining that the methodological rigor was adhered to, thus making the findings more reliable.

This proposed quality assessment tool has been used in a few studies, some of which have been published or are undergoing peer review. It is worth highlighting that this is just an adaptation based on necessities that arose to evaluate self-controlled study designs. In the course of time, it will be subjected to a Delphi process to validate its use and enhance its applicability. It is our expectation that the tool will contribute to critical assessment of the methodological quality of SCCS and SCRI studies and that it will be continuously improved and adapted by the scientific community.

The SCCS and SCRI are important study designs in pharmacoepidemiology and are useful in studying rare events reported after transient exposures, a good example are the influenza vaccines and the development of GBS. This paper presents a first proposal of a quality assessment tool for such studies. This came after the author’s necessity to evaluate the methodological quality of such designs a few years back and no instrument was found, thus, the authors adapted an already existing tool and took into consideration the premises of SCCS/SCRI to design the tool. It is expected that the tool may guide researchers in assessing the methodological rigor of SCCS/SCRI study designs especially now with the COVID-19 pandemic where these designs could be widely used especially in studying the events being reported after the administration of the COVID-19 vaccines.

Wachira V.K: Conceptualization, Investigation, Methodology, Visualization, Writing Original Draft, Writing-review Peixoto H.M: Conceptualization, Supervision, Validation, Visualization, Writingreview De Oliveira M.R.F: Conceptualization, Supervision, Validation, Visualization, Writing-review.

This study was supported by Coordenação de Aperfeiçoamento de Pessoal de Nivel Superior (CAPES), Fundação de Apoio à Pesquisa do DF (FAP-DF) (Edital 04/2016-Demanda Induzida-Aedes Aegypti e as arboviroses Zika, Chikungunya e Dengue) and the National Institute for Science and Technology for Health Technology Assessment (IATS).

The authors do not have no conflicts of interest to declare that are relevant to the content of this article.

1. El-Gilany (2019) Self-controlled case series study (SCCSS): A novel research method. Asploro J Biomed Clin Case Reports 2(1):29-32.

   [Crossref] [Google Scholar]

2. Farrington CP, Hocine MN (2010) Within-individual dependence in self-controlled case series models for recurrent events. J R Stat Soc Ser C Appl Stat 59(3):457-75.

   [Crossref] [Google Scholar]

3. Farrington CP, Whitaker HJ, Hocine MN (2009) Case series analysis for censored, perturbed, or curtailed post-event exposures. Biostatistics 10(1):3-16.

   [Crossref] [Google Scholar] [PubMed]

4. Whitaker HJ, Hocine MN, Farrington CP (2009) The methodology of self-controlled case series studies. Stat Methods Med Res 18(1):7-26.

   [Crossref] [Google Scholar           ]

5. Petersen I, Douglas I, Whitaker H (2016) Self controlled case series methods: An alternative to standard epidemiological study designs. BMJ 354354:4515-4515.

   [Crossref] [Google Scholar] [PubMed]

6. Whitaker HJ, Hocine MN, Farrington CP (2009) The methodology of self-controlled case series studies. Stat Methods Med Res 18(1):7-26.

   [Crossref] [Google Scholar] [PubMed]

7. Takeuchi Y, Shinozaki T, Matsuyama Y (2018) A comparison of estimators from self-controlled case series, case-crossover design, and sequence symmetry analysis for pharmacoepidemiological studies. BMC Med Res Methodol. 2018;18(1):1-15.

   [Crossref] [Google Scholar] [PubMed]

8. Petersen I, Douglas I, Whitaker H (2016) Self controlled case series methods: An alternative to standard epidemiological study designs. BMJ 354:i4515.

   [Crossref] [Google Scholar] [PubMed]

9. Andrews N, Stowe J, Miller E (2017) No increased risk of Guillain-Barre? syndrome after human papilloma virus vaccine: A self-controlled case-series study in England. Vaccine 35(13):1729-32.

   [Crossref] [Google Scholar] [PubMed]

10. Schonberger, Honberger LB, Bregman DJ, Sullivan-Bolyai JZ, Keenlyside RA, et al. (1979) Guillain-Barré syndrome following vaccination in the national influenza immunization program, united states, 1976-1977. Am J Epidemiol 95(1):65-73.

    [Crossref] [Google Scholar] [PubMed]

11. Hunsberger S, Proschan MA (2017) Simple approaches to analyzing self-controlled case series (SCCS) data. Stat Biopharm Res 9(1):65-72.

    [Crossref] [Google Scholar]

12. Whitaker HJ, Farrington CP, Spiessens B, Musonda P (2006) Tutorial in biostatistics: The self-controlled case series method. Stat Med 25(10):1768-97.

    [Crossref] [Google Scholar] [PubMed]

13. Li L, Kulldorff M, Russek-Cohen E, Kawai AT, Hua W (2015) Quantifying the impact of time-varying baseline risk adjustment in the self-controlled risk interval design. Pharmacoepidemiol Drug Saf 24(12):1304-12.

    [Crossref] [Google Scholar] [PubMed]

14. Li R, Stewart B, Weintraub E (2016) Evaluating efficiency and statistical power of self-controlled case series and self-controlled risk interval designs in vaccine safety. J Biopharm Stat 26(4):686-93.

    [Crossref] [Google Scholar] [PubMed]

15. Liu CH, Yeh YC, Huang WT, Chie WC, Chan KA (2020) Assessment of pre-specified adverse events following varicella vaccine: A population-based self-controlled risk interval study. Vaccine 38(11):2495-502.

    [Crossref] [Google Scholar] [PubMed]

16. Baker MA, Lieu TA, Li L, Hua W, Qiang Y, et al. A vaccine study design selection framework for the postlicensure rapid immunization safety monitoring program. Am J Epidemiol. 2015;181(8):608-18.

    [Crossref] [Google Scholar] [PubMed]

17. Tod D, Booth A, Smith B (2021) Critical appraisal. Int Rev Sport Exerc Psychol 1-21.

    [Crossref]

18. Goldstein A, Venker E, Weng C (2017) Evidence appraisal: A scoping review, conceptual framework, and research agenda. J Am Med Informatics Assoc 24(6):1192-203.

    [Crossref] [Google Scholar] [PubMed]

19. Wells GA, Shea B O´Connell, Peterson J, Welch V, Losos M, et al. (2014) Newcale-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses 8159.

    [Google Scholar]

20. Wachira VK, Peixoto HM, de Oliveira MRF (2019) Systematic review of factors associated with the development of guillain-barré syndrome 2007-2017: What has changed?. Trop Med Int Heal. 24(2):132-42.

    [Crossref] [Google Scholar] [PubMed]

21. Wachira VK (2018) Etiologia da síndrome de Guillain-Barré-uma revisão sistemática de literatura: O que mudou em 10 anos?. StatPearls Publishing.

    [Crossref] [Google Scholar]

22. Landim P (2020) Revisão Sistemática da Literatura: Etiologia da Síndrome de Guillain-Barré Temporalidade ou Causaliddae com procedimentos cirúrgicos e arboviroses. Landim1-14.

    [Google Scholar]

23. Rudney M, Ferreira (2020) Revisão Sistemática da Literatura : Etiologia da Síndrome de Guillain-Barré-temporalidade ou causalidade com as vacinas HPV e Influenza. SciELO

    [Crossref] [Google Scholar]

24. Ul Haq SM (2008) Assessing the quality of primary studies in systematic reviews. Int J Heal Promot Educ 46(4):139-41.

    [Crossref] [Google Scholar]

25. Stang A (2010) Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol. 25(9):603-5

    [Crossref] [Google Scholar] [PubMed]

26. Greenland S (1994) Quality scores are useless and potentially misleading. Am J Epidemiol 140(3):290-6.

    [Crossref] [Google Scholar]

27. Greenland S (2001) On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions. Biostatistics 2(4):463-71.

    [Crossref] [Google Scholar] [PubMed]

28. Caldeira D, Rodrigues B, David C, Costa J, Pinto FJ, et al. (2019) The association of influenza infection and vaccine with myocardial infarction: Systematic review and meta-analysis of self-controlled case series. Expert Rev Vaccines 18(11):1211-7.

    [Crossref] [Google Scholar]

29. Hviid A, Laksafoss A (2021) Quadrivalent human papillomavirus vaccination and non-targeted infectious disease hospitalisation: Population-based self-controlled case series analysis. Lancet Reg Heal-Eur 10:100189.

    [Crossref] [Google Scholar]

30. Varatharaj A, Thomas N, Ellul MA, Davies NWS, Pollak TA, et al. (2020) Neurological and neuropsychiatric complications of COVID-19 in 153 patients: A UK-wide surveillance study. The Lancet Psychiatry 7(10):875-82.

    [Crossref] [Google Scholar]

31. Munhoz RP, Pedroso JL, Nascimento FA, De Almeida SM, Barsottini OGP, et al. Neurological complications in patients with SARS-CoV-2 infection: A systematic review. Arq Neuropsiquiatr 78(5):290-300.

    [Crossref] [Google Scholar]

32. Patone M, Handunnetthi L, Saatci D, Pan J, Katikireddi SV, et al. Neurological complications after first dose of COVID-19 vaccines and SARS-CoV-2 infection. Nat Med 27(12):2144-53.

    [Crossref] [Google Scholar] [PubMed]