Evaluation of a Curriculum in an Internal Medicine Residency and Patient- Centered Medical Home Rotation

Nawaz H, Via C, Wild D, Costales V, Gentile C^*, Petraro P and Lim L

Department of Medical Education, Griffin Hospital, Derby, USA

*Corresponding Author:

Christina Gentile, MPH
Department of Medical Education, Griffin Hospital
130 Division St., Derby, Connecticut, 06418, USA
Tel: 203-735-1265 Ext: 301
E-mail: cgentile@griffinhealth.org

Received date: April 28, 2017; Accepted date: May 24, 2017; Published date: May 25, 2017

Citation: Nawaz H, Via C, Wild D, Costales V, Gentile C, et al. (2017) Evaluation of a Curriculum in an Internal Medicine Residency and Patient- Centered Medical Home Rotation. J Community Med Health Educ 7:523. doi:10.4172/2161-0711.1000523

Copyright: © 2017 Nawaz H, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Community Medicine & Health Education

Abstract

Introduction: The Patient-Centered Medical Home (PCMH) model aims to transform primary care by delivering high-quality, safe, comprehensive, patient-centered, and accessible care. Few evaluations of PCMH curricula for internal medicine residency programs have been published. The objective of this study was to develop and implement a PCMH curriculum, a new PCMH rotation, and to redesign the resident ambulatory care clinic and private primary care practices of the residency program. Effects were evaluated by examining: 1) residents’ attitudes towards primary care and PCMH, 2) residents’ skills in communicating and managing care and 3) patients’ assessments of practice accessibility.

Methods: From 2010-2015, we implemented a multi-modal program combining practice redesign, a lecture series, a one-month PCMH rotation, clinical activities, a quality improvement project, and self-reflection at an academic community hospital, a Federally Qualified Health Center (FQHC) and faculty practice offices to internal medic ne residents. Evaluations were conducted through resident self-assessment, patient assessment of practice accessibility, and objective structured clinical examinations.

Results: Twelve residents [41% (12/29)] completed all components of the curriculum. After the rotation, residents thought their PCMH skills improved in 13 components of PCMH (all domains: p<0.02). Residents’ attitudes towards primary care and PCMH did not improve from 2010 to 2012 [response rate: 38% (11/29) and 44% (13/29), respectively]. Patient ratings improved, mostly in practice accessibility (p<0.001) and test notification (p=0.01).

Conclusion: By engaging in the PCMH curriculum and practice redesign, residents showed improvement in PCMH skills across all domains, and patients demonstrated increased satisfaction with the practice. This curriculum in PCMH can be utilized as a useful model for other internal medicine programs.

Keywords

Patient centered medical home; Primary care; Caredelivery redesign; Graduate medical education; Resident rotation; Evaluation

Abbreviations

CSMS: Connecticut State Medical Society; EHR: Electronic Health Record; FQHC: Federally Qualified Health Center; GFP: Griffin Faculty Physicians; HHS: Health and Human Services; HRSA: Health Resources and Services Administration; IT: Informational Technology; NCQA: National Committee for Quality Assurance; OSCE: Objective Structured Clinical Examinations; PCMH: Patient Centered Medical Home; PEAT: Patient Experience Assessment Test; QI: Quality Improvement

Introduction

The model known as Patient-Centered Medical Home (PCMH) aims to transform primary care by changing how it is organized and delivered [1]. PCMH encompasses these five functions and attributes: Comprehensive care, patient-centered, coordinated care, accessible services, and quality and safety [1]. Patient outcomes have been shown to improve as a result of the PCMH model [2]. PCMH teams can optimize the model's potential if they learn to coordinate, communicate, and collaborate [3]. Since the introduction of the PCMH, residency programs have been challenged to develop curricula and provide a clinical experience that will prepare residents to practice in such settings [4]. Physicians must learn to work in interdisciplinary teams and share decision-making [5] to improve care processes, to enhance quality and safety, and to exercise managerial skills, such as managing relationships and finances. Few residents are familiar with the PCMH model [6] and few residency programs offer their residents a fully realized PCMH [7].

Some residency training programs have reported work in redesigning their curricula and clinical components to train residents to engage with chronically ill patients and to measure their proficiency in doing so [8,9]. Previous publications have also described curricula that address individual components of the PCMH, such as pain management [10], team-based care [11] and group visits [12]. However, prior published studies did not include resident selfassessments of their mastery of specific PCMH components, nor do they provide patient assessments [6]. Lastly, most of the published studies are based on family practice residency programs, rather than internal medicine [12].

Our aim was to develop and implement a comprehensive PCMH curriculum, including a new PCMH rotation, and to redesign the resident ambulatory care clinic and the community ambulatory office practices affiliated with the internal medicine residency program. We evaluated the impact of the PCMH curriculum and practice redesign on 1) residents’ attitudes towards primary care and PCMH, 2) residents’ skills in communication and patient care management and 3) patients’ assessments of practical accessibility and self-care support.

Materials and Methods

Setting and participants

Griffin Hospital is a 140-bed, mid-size academic community hospital in Connecticut, and it is a teaching affiliate of the Yale School of Medicine. The residency programs consist of an internal medicine program (12 residents); a preliminary year program (8 residents); and a combined internal medicine/preventive medicine program (12 residents). Residents receive training in outpatient medicine primarily at two sites: A Federally Qualified Health Center (FQHC) and the primary care practices of the Griffin Faculty Physicians (GFP). The residents complete approximately 12,000 patient encounters per year at the FQHC. The majority of these patients are Caucasian (63%), followed by Hispanics (22%) and African Americans (15%). Most of these FQHC patients have Medicaid (61%) and 17% are self-pay. The five primary care providers of the GFP offices conduct approximately 8,700 visits per year. Ninety-two percent of GFP patients are Caucasian (92%), 3% are African-American, 3.4% are Hispanic, and 2.4% are Asians. About 20% of these patients are self-paid or have Medicaid.

Practice redesign

Like most previous efforts to implement a new care model [8], ours attempted to combine a curriculum introduction with practice redesign. We collaborated with TransforMED, a proof-of-concept demonstration project through the American Academy of Family Physicians [13], which, through a series of two site visits and regular conference calls, provided a framework for the practice redesign as well as a national comparison group for patient evaluations of the practice. We conducted monthly conference calls with our consultants and semi-annual meetings with residents to redesign the practice. We made multiple changes to both the resident clinic and the faculty practices, including schedule changes, team-based care through the creation of resident provider teams, patient goal documentation, and active management of critical conditions through chart audits. Both the FQHC and GFP sites implemented a series of changes to adopt the PCMH model over the course of three years.

Lecture series

We designed and created a new lecture series in PCMH led by TransforMED and the Connecticut State Medical Society (CSMS). Topics included ‘Introduction of PCMH’, ‘Basic aspects of the PCMH at Griffin Hospital-what is missing’, ‘Team-based Care’, ‘Care coordination and clinical integration’, ‘Evidence based medicineevaluation and comparative effectiveness within the practice’, and ‘Enhanced access to medical care and practice/physician payment’.

Rotation

In their final year of training, residents participated in the four-week PCMH rotation. The rotation included four main components: selfstudy of curricular material, clinical activities, practice management/ quality improvement (QI), and self-reflection (Figure 1).

1) Self-study of curricular material: Residents were required to watch a core set of webinars hosted on TransforMED’s web exchange, DeltaNet. For health literacy, trainees completed the Health Resources and Services Administration (HRSA) web-based training program for Effective Healthcare Communication Tools for Healthcare Professionals: Addressing Health Literacy, Cultural Competency, and Limited English Proficiency. For cultural competency, residents complete the Health and Human Services (HHS) Office of Minority Health’s web-based training program called ‘A Physician’s Practical Guide to Culturally Competent Care.' Printed materials for review included articles on patient engagement from the popular literature, as well as resources on practice management [14] and quality improvement [15].

2) Ambulatory primary care: Clinical activities took place at both the FQHC and GFP sites: Residents attended their weekly continuity clinic at the FQHC as usual. Also, residents completed an ambulatory care block at the Griffin Faculty Practices, during which they spent two to three days per week at the GFP. At both sites, residents were expected to observe and reflect on any differences in the workflow and staff and patient behaviors between the FQHC and the GFP. They also observed how experienced clinicians integrate Electronic Health Record (EHR) documentation, coding, and billing in their workflow, and saw policies in action that govern same-day access and optimal use of staff and resources. The supervising physicians observed residents and provided feedback on patient engagement, workflow habits, and documentation.

3) Practice management/quality improvement: Residents learned about practice management tool through their readings (as discussed above), attending practice meetings, shadowing various office staff members, and by performing a quality improvement (QI) project. The resident chose a topic for this project at the beginning of the rotation and could modify it during the rotation. Examples of resident projects include: Analyzing and improving continuity of care at the resident clinic; Assessing the quality of care of diabetic patients; and rewriting educational material distributed at the office to make it more accessible to appropriate for patients with low health literacy. Residents presented progress reports on their QI projects at practice team meetings, and they led a meeting at the end of their rotations in which they discussed the final results of their quality improvement project.

4) Reflection: Residents were expected to reflect on their experiences at least twice a week in an email to the rotation supervisor. In these messages, residents reported on their readings or webinars completed that week and reflected on what they had learned about PCMH and how their work at the clinic compared with their understanding of the PCMH. We provided the residents a rotation handbook containing suggested topics for reflection and tip to organize reflections. The rotation supervisor replied to the reflective emails with comments. Resident output for this rotation consisted of the reflective emails, a report on the results of their QI project at the end of the rotation, and an email to their co-residents at the end of the rotation in which they summarized take-home points they felt were important to their peers. Residents had two supervisors during their rotation: their clinical supervisor, who was one of the primary care physicians in the faculty practice office; and the rotation supervisor, who was the medical director of the practice group.

Resident and rotation evaluations/statistical analyses

For the purpose of overall project evaluation, we combined resident self-assessments, patient assessments, and faculty assessments. We used SAS 9.2 (SAS Institute Inc., Cary, NC) to perform statistical analyses. Specific statistical analyses and tests are described in each subsequent section.

Evaluation of residents

The rotation supervisor and the clinical supervisor evaluated the residents at the midpoint and the end of each rotation. Supervisors based the evaluation on direct observation of the residents’ interactions with patients and staff, and on the quality of the residents’ documentation, the quality of the reflective emails, participation in practice management meetings, and an evaluation of the QI project.

Rotation evaluation of residents

The residents rated their PCMH competencies before and after they participated in the PCMH rotation in a retrospective before-and-after assessment. We used retrospective self-assessment because we reasoned that residents would have had little exposure to many facets of the PCMH before their rotation and would, therefore, have different criteria for assessing their skills and knowledge at the beginning and the end of the rotation. We used similar retrospective self-assessments in faculty development workshops, which correlate well with independent ratings [16], PCMH competencies included 13 domains, such as patient-centered care, ensuring safe care transitions, and fundamentals of practice management. Residents rated these domains on a scale of 1-9 (lowest to highest competency). We analyzed beforeand after-rotation scores using paired t-test.

Resident attitudes towards primary care and PCMH

We assessed residents’ attitudes toward primary care and PCMH annually. We used web-based and anonymous surveys to ensure confidentiality. Statements that residents rated, such as “I have the tools and resources to be a physician in a high-performance primary care practice,” were rated on a 5-point Likert scale. We used Wilcoxon rank sum test to compare variations of responses.

Residents’ communication and care management skills

In addition to requiring self-assessments, we also assessed residents’ communication and care management skills twice a year using Objective Structured Clinical Examinations (OSCEs). To evaluate residents’ skills in using the EHR for patient care management in transitions of care, we developed a medication reconciliation station. The scenario asked residents to change or discontinue medications while discharging a patient. A clinical Information Technology specialist rated the results on a scale from 0-100. A score greater than 70 was considered passing. We compared changes in the scores over the years of the program using Fisher’s Exact test.

Assessment of the practice redesign: Patient experience assessment test

The patients assessed practice accessibility and primary care support using the Patient Experience Assessment Test (PEAT), a survey developed by TransforMED that has been used by over 30,000 patients nationally. This survey lists statements that patients rate on a five-point Likert scale. The survey listed five statements, such as ‘I was able to see the clinician I requested,’ ‘My care team contacts me to remind me I need to come in for a checkup,’ and ‘I know my rights and responsibilities as a patient of this practice.’ We collected patient assessments in 2011 and again in 2012. We grouped ‘strongly agreed’ and ‘agreed’ against those that choose ‘neutral,' ‘disagreed’, and ‘strongly disagreed.' We analyzed differences between the two samples by the Fisher exact test.

Results

Twelve residents completed all components of the curriculum.

Resident rotation evaluation

All twelve residents completed the self-evaluation of PCMH skills and felt they improved significantly after the PCMH rotation in all 13 components (Figure 2).

The largest changes in before and after scores were seen in the areas of open access scheduling, coding, billing, and practicing quality improvement (Table 1).

Table 1: Resident self-evaluation of PCMH related attitudes and skills.

Resident attitudes towards primary care and PCMH

Our response rate was lower in both years [(38% (11/29) in 2010; 44% (13/29) in 2012)]. At both baseline and follow-up, residents felt positive towards both their continuity clinic and PCMH. On a fivepoint Likert scale, the average mean score was 3.71 in 2010 and 3.87 in 2012. There was no statistically significant change over the course of the curriculum in residents’ attitudes towards primary care or PCMH (Table 1).

OSCE results

We first trialed a medication reconciliation station in 2010, but it was too complicated for the residents. In 2011, ten residents participated in the medication reconciliation station, and 2/10 reached a score over 70 out of 100. In 2012, 19/21 residents scored over 70 (p<0.001).

Patients’ ratings of the practice

Forty-seven patients completed the PEAT survey in 2011 and 77 patients completed it in 2012. An overview of the patients’ ratings of the practice is given in Figure 3.

Overall, patient ratings improved significantly, most notably in patient engagement, practice accessibility, and test notification (Table 2).

Table 2: Patient assessment of practice redesign- the Patient Experience Assessment Test (PEAT).

PCMH recognition and practice redesign

The FQHC adopted a fully integrated electronic health record. The clinic also implemented a series of changes, including flexible hours, same day appointments, easy access to providers, and after-hours oncall system. The clinic received its Joint Commission PCMH designation in August 2013 and has also been designated by the National Committee for Quality Assurance (NCQA).

Discussion

Our experience shows that it is feasible to implement a PCMH curriculum and rotation, and that including this rotation can improve the residents’ experience. Such a rotation can be incorporated even if a practice is in the transformation phase of fully converting to PCMH. As in other studies, we found that building a PCMH curriculum in residency practice is labor-intensive, but rewarding [6]. PCMH provides an opportunity to improve clinic practice and residency training [17].

Contrary to our expectations, the residents’ attitudes towards their work in the clinic and towards primary care as a career choice did not improve significantly. Several possible explanations come to mind. Our response rates were relatively low which was likely due to our decision to conduct an anonymous web-based survey to obtain objective data. Previously published curricula which did demonstrate a change in resident attitudes towards clinic work [16] were based on surveys of residents immediately after the rotation and were in reference to a pain clinic that had previously been perceived negatively. In contrast, our residents’ attitudes towards primary care were already quite positive in most domains, so the potential for improvement was small. Compared to a national sample in which residents described their continuity experiences as stressful [7], our residents’ attitudes seem to be mostly positive.

We faced numerous challenges in our efforts to implement the PMCH curriculum. We made several changes as we progressed in our quest for PCMH in response to resident and staff feedback. For example, based on our experiences, we wrote an orientation manual describing the main elements of the program. Organizing and implementing the rotation was time-intensive because it required a lot of one-on-one supervision of residents, both to foster reflections and to oversee the QI project. Once residents had completed their rotations, we were not consistently successful in keeping them involved in redesigning the resident clinic. In the evaluation, we struggled to improve response rates to our web-based surveys and to obtain patient evaluations of individual residents. We had planned to survey patients about their providers, but this proved to be too difficult logistically. Another challenge was how to develop OSCE-stations that would measure behaviors important in care management and population management. The medication reconciliation station we developed depended heavily on assistance from the hospital’s Informational Technology (IT) department.

Our study is also subject to several limitations. We implemented it at a single institution and with a small sample, which may undermine the internal and external validity of the study, thus not making it generalizable to other educational programs or institutions. It was also not feasible to have a control group, given the size of our program. Essential Entrustable Professional Activities have recently been defined for PCMH [18]. Many of these have not been evaluated yet in our rotation, such as usage of EHR or use of a disease registry. Nor did we have a robust faculty development component, as in other published curricula [19], and we struggled with a lack of IT support for patient registries or patient evaluations of individual residents. Lack of IT support for higher-level EHR functions has been described previously as a barrier to implementing PCMH in resource-challenged FQHCs [20] and is not unique to our study.

Notwithstanding the above-mentioned challenges, our study has several notable strengths. In order to keep residents involved in redesigning the clinic, we instituted semi-annual meetings with all residents to review clinic procedures. Also, many resident educational activities were synergistic and overlapped with our transformation efforts. For example, residents could educate other team members about PCMH components or research best practices on the TransforMED’s web exchange, DeltaNet, to help with team practice meetings. We also used an experiential design in two practices, one at a resident continuity clinic, and one in faculty practice offices which allowed our residents to compare workflows and challenges in different settings. It also helped them distinguish which challenges and stresses are common to primary care, and which are specific to primary care in a resident continuity clinic or a private practice. Overall, our rotation was well received by our residents, and it led to improvements in their self-assessed competencies. We also used a robust evaluation structure, including external assessments and patient assessments of the practice that were compared to a national sample.

Conclusion

This curriculum may provide a useful model for other internal medicine programs to develop, implement, and evaluate PCMH curricula.

Acknowledgements

We would like to especially thank Dr. Shobha Jagadeesh, Dr. Neha Malik, and Dr. Seema DeSouza for their assistance and support throughout the duration of this project. The Griffin Hospital Institutional Review Board reviewed our application and considered this study exempt. The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. Primary Care Residency Training Grant from the Health Resources and Services Administration (HRSA grant number D5FHP20670) from 2010-2015 supported the PCMH transformation and practice redesign. HN, VC, DW, and LL were involved in the development and implementation of the PCMH curriculum. CV, PP, and CG analyzed and interpreted the data regarding resident attitudes towards primary care and PCMH, OSCE results, and patient ratings. All authors read and approved the final manuscript and declare that they have no competing interests.

References

1. Agency for Healthcare Research and Quality (2016) Defining the PCMH. PCMH resource center.
2. Peikes D, Zutshi A, Genevro J, Smith K, Parchman M, et al. (2012) Early evidence on the patient-centered medical home. Agency for Healthcare Research and Quality, USA.
3. Xin H, Kilgore ML, Sen B (2016) Is Access to and use of patient perceived patient-centered medical homes associated with reduced nonurgent emergency department use? Am J Med Qual.
4. Evans L, Whitham JA, Trotter DR, Filtz KR (2011) An evaluation of family medicine residents' attitudes before and after a PCMH innovation for patients with chronic pain. Fam Med 43: 702-711.
5. Eiff, MP, Waller E, Fogarty CT, Krasovich S, Lindbloom E, et al. (2012) Faculty development needs in residency redesign for practice in patient-centered medical homes: A P4 report. Fam Med 44: 387-395
6. Markova T, Mateo M, Roth LM (2012) Implementing teams in a patient-centered medical home residency practice: Lessons learned. J Am Board Fam Med 25: 224-231.
7. Nadkarni M, Reddy S, Bates CK, Fosburgh B, Babbott S, et al. (2011) Ambulatory-based education in internal medicine: Current organization and implications for transformation. Results of a national survey of resident continuity clinic directors. J Gen Intern Med 26: 16-20.
8. Stevens DP, Bowen JL, Johnson JK, Provost LP, Holman HR, et al. (2010) A multi-institutional quality improvement initiative to transform education for chronic illness care in resident continuity practices. J Gen Intern Med 25: 574-580.
9. Bowen JL, Provost L, Stevens DP, Johnson JK, Woods DM, et al. (2010) Assessing chronic illness care education (ACIC-E): A tool for tracking educational re-design for improving chronic care education. J Gen Intern Med 25: 593-609.
10. Muench J, Jarvis K, Boverman J, Hardman J, Hayes M, et al. (2012) Tilling the soil while sowing the seeds: Combining resident education with medical home transformation. Subst Abus 33: 282-285.
11. Leasure EL, Jones RR, Meade LB, et al. (2013) There is no “I” in teamwork in the patient-centered medical home: Defining teamwork competencies for academic practice. Acad Med 88: 585-592.
12. Barr WB, Aslam S, Levin M (2011) Evaluation of a group prenatal care-based curriculum in a family medicine residency. Fam Med 43: 712-717.
13. American Academy of Family Physicians (2016) TransforMED.
14. Halley MD (2011) Owning medical practices. (1st edn), American Heart Association Press, USA.
15. Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, et al. (2009) The improvement guide: A practical approach to enhancing organizational performance. (2nd edn), Jossey-bass business & management series, Chapters 1 and 2, pp: 13-48.
16. Evans L, Whitham JA, Trotter DRM, Filtz KR (2011) An evaluation of family medicine residents’’ attitudes before and after a PMCH innovation for patients with chronic pain. Fam Med 43: 702-711.
17. Markova T, Mateo M, Roth LM (2012) Implementing teams in a patient-centered medical residency practice: Lessons learned. J Am Board Fam Med 25: 224-231.
18. Chang A, Bowen JL, Buranosky RA, Frankel RM, Ghosh N, et al. (2012) Transforming primary care training-patient-centered medical home entrustable professional activities for internal medicine residents. J Gen Intern Med 208: 801-809.
19. Eiff MP, Waller E, Fogarty CT, Krasowich S, Lindbloom E, et al. (2012) Faculty development needs in residency redesign for practice in patient-centered medicl homes: A P4 Report. Fam Med 44: 387-395.
20. Carney PA, Eiff MP, Saultz JW, Lindbloom E, Waller E, et al. (2012) Assessing the impact of innovative training of family physicians for the patient-centered medical home. J Grad Med Educ 4: 16-22.

--