Evaluation of the Metrological Quality of Medico-Administrative Data for Perinatal Indicators: A Pilot Study

Goueslard K¹, Revert M^2,3, Pierron A¹, Vuagnat A^1,4, Cottenet J¹, Benzenine E¹, Fresson J⁵ and Quantin C^1,6,7*

¹CHRU Dijon, Biostatistics and Medical Information Department (DIM), University of Burgundy, Dijon, France

²School of Midwifery, St. Anthony Hospital Saint - Antoine, 184 rue du Faubourg Saint Antoine, Paris, France

³Research Unit EA7285, Clinical Risk and Health of Women in Security and Perinatal Health, University of Versailles St – Quentin, 2 Avenue of the Source-of-the-Bièvre, Montigny-le-Bretonneux, France

⁴Research Studies, Evaluation and Statistics, Ministry of Health, Paris, France

⁵Department of Medical Information, Regional Maternity Hospital of Nancy, France

⁶INSERM, CIC 1432, Dijon, France; Clinical epidemiology/Clinical Trials Unit, Dijon University Hospital, Clinical Investigation Center, Dijon, France

⁷INSERM UMR 1181, Biostatistics, Biomathematics, Pharmacoepidemiology and Infectious Diseases, (B2PHI), University Bourgogne, F-21000 Dijon, France

*Corresponding Author:

Catherine Quantin
CHRU Dijon, Biostatistics and Medical Information Department (DIM)
University of Burgundy, Dijon, France
Tel: 33380293465
E-mail: catherine.quantin@chu-dijon.fr

Received date: April 29, 2016; Accepted date: June 06, 2016; Published date: June 15, 2016

Citation: Goueslard K, Revert M, Pierron A, Vuagnat A, Cottenet J, et al. (2016) Evaluation of the Metrological Quality of Medico-Administrative Data for Perinatal Indicators: A Pilot Study. J Community Med Health Educ 6:437. doi:10.4172/2161-0711.1000437

Copyright: © 2016 Goueslard K, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Community Medicine & Health Education

Abstract

Background: In order to assess public health policies for the perinatal period, routinely produced indicators are needed for the whole population. These indicators are used to compare French national public health policy with those of other European countries. French medico-administrative data are straightforward and may be a valuable source of information for research. The study aimed to assess the metrological quality of medico-administrative data for perinatal indicators in three university hospitals.
Methods: The hospital data were compared with medical records for 2012 for 300 live births after 22 weeks of amenorrhea, drawn at random from three university hospitals. The variables were chosen according to the Europeristat Project’s core and recommended indicators, as well as those of the French National Perinatal survey conducted in 2010. The information gathered blindly from the medical records was compared with the medicoadministrative data. The positive predictive value (PPV) and the sensitivity were used to assess data quality.
Results: Data on maternal age, parity and mode of delivery as well as the rates of premature births from the two sources were superimposable. The PPV was 100.0% for pre-existing diabetes, 88.9% [74.3-100] for gestational diabetes and 100.0% for hypertension disorders with a rate of 9.0% in hospital data and 6.3% in the medical records. The positive predictive value for pre-eclampsia and HELLP syndrome was also 100% but the sensitivity was only 81.3%. The positive predictive value was 81.3% [67.8-94.8] for obesity and 90% [79.8-99.2] for postpartum hemorrhage.
Conclusion: This pilot study showed variability between establishments and between indicators, which reinforces the need for specific training in coding for activities. It confirms the importance of conducting such studies at the national level.

Keywords

Medico-administrative data; Perinatal indicators; Perinatal health; Individual data; Positive predictive value; Sensitivity

Introduction

In perinatal health, exhaustive and standardized information is required to compare indicators with those in other European countries and for public health studies.

France has many sources of data relative to the perinatal period. Data from the civil registry can be used to determine the number of live births, the number of multiple pregnancies and various parental characteristics (report of the National Institute of Statistics and Economic Studies, 2010). The National Perinatal Surveys (NPS) conducted by the National Institute for Health Research (Institut National de la Santé et de la Recherche Médicale, INSERM) in collaboration with Ministry of Health (Direction de la Recherche, des Etudes, Evaluation et des Statistiques, DREES) make it possible to follow numerous indicators by meeting the public health objectives. These surveys are conducted at regular intervals on a representative sample of births. The DREES exploits the medical and sociodemographic data via the eighth-day certificate established in the first week of life of the child. The Institut National de Veille Sanitaire gathers the data of registries for malformations and handicaps in certain regions. Death certificates allow the INSERM to determine the number of neonatal deaths. This fragmented information system presents certain advantages but also as many drawbacks, notably in terms of the annual monitoring of a certain number of indicators.

The medico-administrative database is of particular interest in France since almost all 800,000 births take place in hospital, which represent 99.6% of French births [1]. The use of medico-administrative databases for epidemiological purposes has already been the subject of numerous studies in various medical disciplines, notably in a perinatal setting [2-20]. In a first study, our team compared these data with aggregated statistics from the civil registry and the NPS of 2010 for the principal indicators of health status at birth [5]. It showed the interest of using these hospital data even though the results were very close, notably for maternal age, the type of delivery, the birth weight and the gestational age. However, we concluded that a complementary study on the quality of individual data was necessary before engaging in epidemiological studies that use hospital data, as aggregated data cannot take into account variations in coding at the individual level.

The aim of our study was thus to assess the metrological quality of medico-administrative data for perinatal indicators in three university hospitals.

Methods

The principle of this transversal multicenter study was to compare data from the medico-administrative database with data from medical records, which we considered the gold standard.

This pilot study was undertaken in three university hospitals, which accurately reflect maternity departments in French university hospitals in terms of size and level of activity.

Deliveries were selected from procedures of « delivery », according to the French Common Classification. We developed a specific computer program to randomly select 100 live births per hospital for the year 2012. Each medical information department ran this program on all stays for « delivery » in its hospital database to extract the data of discharge abstracts linked to these 100 stays for delivery and the list of the corresponding 100 medical records. A list of 20 additional stays made it possible to compensate for possibly inaccessible records.

The medical record consisted of an electronic or paper document that retraced prenatal care, the delivery and the post-delivery stay, the report of the procedure and the discharge letter. The data from each record were collected on a standardized form in accordance with national predefined guidelines regarding the collection of medicoadministrative data. The variables studied corresponded to the characteristics of the hospital stay, the mother, the pregnancy, the delivery and the new-born: the mode of admission and discharge, the length of stay, the age of the mother, the existence of maternal obesity (BMI>30), the weight and gestational age of the new-born and the parity for vaginal deliveries. Maternal diseases concerned diabetes before the pregnancy, gestational diabetes, arterial hypertension (AHT), premature rupture of membranes (PROM), the possible premature delivery (PPD) and post-partum hemorrhage. The characteristics of the labor and delivery included the type of pregnancy (singleton or multiple), the type of presentation, the mode of delivery (spontaneous vaginal, instrumental extraction, caesarean: emergency or not), epidural anesthesia for vaginal deliveries.

Statistical analysis

Means or proportions were calculated for each source of data. To evaluate the metrological quality of the medico-administrative databases, the positive predictive value (PPV) and the sensitivity were calculated for dichotomous data. Continuous data were assessed by the concordance rate. The medical record was considered the reference.

The PPV corresponded to the probability that the variables recorded in the discharge abstracts were also present in the medical record. Sensitivity corresponded to the probability that variables recorded in the medical record were also present in the discharge abstracts. The rates of false negatives (FN) and false positives (FP) were also calculated. The rate of concordance corresponded to the number of discordant cases between the discharge abstracts and the medical records over the total number of records examined.

This study was approved by the National Committee for data protection (registration number 913291). To meet the requirement of the CNIL (the data protection agency), the family names and first names were removed, the date of birth was replaced by the age at the delivery, and the dates of admission and discharge were replaced by the length of stay.

Results

This study showed that, in the three hospitals, the coding of diagnoses in medico-administrative data was usually done by experienced personnel who had been specifically trained in coding for obstetrics-gynecology. The procedures were coded in real time by the practitioners. In addition, the persons in charge of coding checked the quality and exhaustiveness of data, for example, by regularly comparing information with that in the registers for the obstetrics operating rooms.

Table 1 presents the characteristics of stays and the home post code for the women. Concerning the mean length of stay (5.2 days in the medical record versus 5.8 in the discharge abstract), the concordance rate between these two sources was 89%. Though the post codes for the department of the mother’s home agreed in 96% of cases, the rate of agreement for the post code of the district was only 94.3%.

Table 1: Characteristics of hospital stays.

The maternal characteristics are presented in Table 2. Concerning the mean age of the women, the two sources of data agreed in 93% of cases. The positive predictive value and the sensitivity for age classes were between 93 and 100%.

Table 2: Maternal characteristics.

In women who had vaginal delivery, the PPV were 91.4% (95% Confidence Interval (CI) [86.3-96.5]) for primiparous women and 96.6% [93.3-99.9] for multiparous women.

Table 3 presents the results relative to maternal morbidity. Though the PPV was 100% for diabetes overall, it was only 88.9% [74.3-100] for gestational diabetes.

Table 3: Maternal morbidity.

We studied hypertension disorders during pregnancy and found a positive predictive value of 100%. The positive predictive value for pre-eclampsia and HELLP syndrome was also 100% but the sensitivity was only 81.3%. The positive predictive value for obesity was 81.3%, 95% CI [67.8-94.8] and that for postpartum hemorrhage was 90%, 95% CI [79.8-99.2].

The results relative to characteristics of the delivery are presented in Table 4. The number of deliveries before 37 weeks of amenorrhea was identical in both sources but the PPV was only 93.9% [85.8-100]. The PPV was 100% for coding for the presence of a scarred uterus, the distinction between singleton and twin births, head presentation and the need for instrumental extraction.

Table 4: Characteristics of delivery.

The data relative to eutocic delivery showed a PPV greater than 99.6%. The PPV was lowest for PROM and breech presentation (42.0 and 60.0%, respectively). The PPD presented a FN rate of 21% and a PPV rate of 68%. For perineal tears, the PPV was 94.3% but the FN were high (7.7%). The number of episiotomies for the two sources was very close with a PPV equal to 89.0%.

For some variables like gestational diabetes and obesity, the positive predictive value varied considerably between hospitals. The PPV ranged from 74% to 100% for gestational diabetes and from 67.8% to 94.8% for obesity.

Discussion

Analysis of individual data

This type of study to evaluate individual data is essential when aggregated data do not allow false positives and false negatives to be assessed as they can compensate for each other. In our study, the frequencies observed in discharge abstracts were often very close to those observed in medical records, whereas the PPV was not 100%. This situation was encountered for gestational age less than 37 WA, for which the frequency observed in the medical record was the same as that in the discharge abstracts (11%), whereas the PPV was 93.9%. It was also the case for post-partum hemorrhage for which the percentages in discharge abstracts and in medical records were very close (around 13%) whereas the PPV was only 89.5%. Only individual comparisons, by going back to the medical record as we did, make it possible to assess the PPV and sensitivity, and thus the potential impact of these errors. It is important to remember that even when the PPV and/or sensitivity are low, if the total numbers of patients are close to those expected, the data can at least be used for description purposes for certain indicators. However, it is recommended to be extremely vigilant in the use of these data for longitudinal epidemiological studies.

The quality of coding for mode of admission and discharge in the discharge abstracts is usually considered unreliable with an underestimation of transfers, which was confirmed in our study. This information needs to be made more reliable in obstetrics-gynecology given the regionalization of care with the organization of healthcare establishments based on the level of care they provide. The evaluation of transfers is of major interest for the Perinatal Healthcare Network.

Inter-hospital variability

Our results brought to light marked inter-hospital variability in the PPV for certain variables. It is important to be careful when studying professional practices from hospital databases as comparisons of professional practices using discharge abstracts could lead to erroneous conclusions if differences in coding practices between teams are not taken into account.

Inter-indicator variability

Our results differed of course from one indicator to another.

The indicators based on consensus definitions and having an influence on the hospital budget (gestational age, singleton or twin births, mode of delivery, presentation) were of course better collected. It is clear that the information used for DRG classifications, and taken into account in the algorithm, is more carefully collected by teams given the impact of this information on the finances of the hospital.

Some indicators with less standardized definitions were less well documented. This was the case for diseases for which the wording in the medico-administrative database may differ from the terms used in clinical practice, and thus are underestimated, even if these diseases involve the mobilization of material and human resources and thus costs.

Perspectives

Specific training and validation seem to be needed to achieve the objectives of hospital financing, but also to meet the ever-increasing use of hospital databases for epidemiological purposes. Practitioners involved in the coding of hospital activity must be offered specific and regularly updated training.

Conclusion

Since 2012, in France, medico-administrative data have become more reliable for two reasons. On the one hand, the importance of this data for budgetary purposes in hospitals has increased. On the other hand, the use of this information for statistical and epidemiological purposes by research and government institutions is becoming more common. This study produced preliminary results, which should be confirmed by a national study which will concern major indicators of perinatal health, such as prematurity and stillbirth.

Acknowledgments

Support grants

"This work was supported by the Fondation pour la Recherche Médicale (FRM), the Direction de la Recherche, des Etudes, Evaluation et des Statistiques (DREES) of the Ministry of Health and the French National Research Agency (Agence Nationale de la Recherche, ANR) and was conducted in collaboration with the DRUG SAFE platform funded by the Agence Nationale de Sécurité du Médicaments et des produits de santé (ANSM)."

Acknowledgments

To the Steering Committee of this study: Pr. P. Sagot (Service de gynécologie-obstétrique, CHU de Dijon), Pr. O. Morel (Service de gynécologie-obstétrique, CHU de Nancy), Pr. F. Goffinet (Service de gynécologie-obstétrique, CHU Port-Royal, Paris), Dr J. Frenkiel (Département d’information médicale, CHU Port-Royal, Paris), et Dr J. Epstein (Département d’information médicale, CHU de Nancy).

We would like to thank P. Bastable for his thorough proofreading of this article.

References

1. Blondel B, Drewniak N, Pilkington, Zetlin J (2011) Out-of-hospital birth and the supply of maternity units in France. Health Place 17: 1170-1173.
2. Chantry AA, Deneux-Tharaux C, Cans C, Ego A, Quantin C, Bouvier-Colle MH (2011) Hospital discharge data can be used for monitoring procedures and intensive care related to severe maternal morbidity. J ClinEpidemiol 64: 1014-1022.
3. Fresson J, Blondel B (2013) Stillbirths soon to emerge from statistical limbo in France. J GynecolObstetBiolReprod (Paris) 42: 1-4.
4. Chantry AA, Deneux-Tharaux C, Bal G, Zeitlin J, Quantin C, Bouvier-Colle M-H (2012) French hospital discharge database: data production, validity, and origins of errors in the field of severe maternal morbidity. Rev EpidemiolSantePublique 60: 177-188.
5. Quantin C, Cottenet J, Vuagnat A, Prunet C, Mouquet MC, et al. (2014) Quality of perinatal statistics from hospital discharge data: comparison with civil registration and the 2010 National Perinatal Survey. J GynecolObstetBiolReprod (Paris) 43: 680-690.
6. Serfaty A, Bénifla J-L (2013) What can be able to expect in stillbirth registration in Hospital Discharge Data System? A feedback from Armand-Trousseau Hospital (Paris). J GynecolObstetBiolReprod (Paris) 42: 101-104.
7. Bernier M-O, Mezzarobba M, Maupu E, Caër-Lorho S, Brisse HJ, et al. (2012) Role of French hospital claims databases from care units in epidemiological studies: the example of the "Cohorte Enfant Scanner" study. Rev EpidemiolSantePublique 60: 363-370.
8. Quantin C, Benzenine E, Ferdynus C, Sediki M, Auverlot B, et al. (2013) Advantages and limitations of using national administrative data on obstetric blood transfusions to estimate the frequency of obstetric hemorrhages. J Public Health 35: 147-156.
9. Goueslard K, Cottenet J, Mariet As, Giroud M, Cottin Y, et al. (2016) Early cardiovascular events in women with a history of Gestational Diabetes Mellitus. Cardiovascular Diabetol 15: 15.
10. Abdulmalak C, Cottenet J, Beltramo G, Georges M, Camus P, et al. (2015) Haemoptysis in adults: a 5-year study using the French nationwide hospital administrative database. EurRespir J 2015 46: 503-511.
11. Schnitzler A, Woimant F, Nicolau J, Tuppin P, de Peretti C (2014) Effect of rehabilitation setting on dependence following stroke: an analysis of the French inpatient database. Neurorehabil Neural Repair 28: 36-44.
12. Lainay C, Benzenine E, Durier J, Daubail B, Giroud M, et al. (2015) Hospitalization within the first year after stroke: the Dijon stroke registry. Stroke 46: 190-196.
13. Lorgis L, Cottenet J, Molins G, Benzenine E, Zeller M, Aube H, et al. (2013) Outcomes after acute myocardial infarction in HIV-infected patients: analysis of data from a French nationwide hospital medical information database. Circulation 127: 1767-1774.
14. Tuppin P, Cuerq A, de Peretti C, Fagot-Campagna A, Danchin N, et al. (2014) Two-year outcome of patients after a first hospitalization for heart failure: A national observational study. Arch Cardiovasc Dis 107: 158-168.
15. Allaert FA, Benzenine E, Quantin C (2016) More than one in two venous thromboembolism (VTE) treated in French hospitals occurs during the hospital stays. Phlebol 31: 390-396.
16. Tuppin P, Ricci-Renaud P, de Peretti C, Fagot-Campagna A, Alla F, et al. (2014) Frequency of cardiovascular diseases and risk factors treated in France according to social deprivation and residence in an overseas territory. Int J Cardiol 173: 430-435.
17. Pagès PB, Cottenet J, Mariet AS, Bernard A, Quantin C (2016) In-hospital mortality for lung cancer resection: nationwide administrative database. EurRespir J 47: 1809-1817.
18. Creuzot-Garcher C, Benzenine E, Mariet As, De Lazzer A, Chiquet C, et al. (2016) Incidence of Acute Postoperative Endophthalmitis after Cataract Surgery: A Nationwide Study in France from 2005 to 2014. Ophthalmol.
19. Quantin C, Benzenine E, Velten M, Huet F, Farrington P, et al. (2013) Self-controlled case series and misclassification bias induced by case selection from administrative hospital databases: application to febrile convulsions in pediatric vaccine pharmacoepidemiology. Am J Epidemiol 178: 1731-1739.
20. Tuppin P, Samson S, Fagot-Campagna A, Lukacs B, Alla F, et al. (2014) Prostate cancer outcomes in France: treatments, adverse effects and two-year mortality. BMC Urol 14: 48.

--