The Development and Validation of a Questionnaire to Audit Advance Care Planning

¹Department of Medicine, Kingston General Hospital, Kingston, Ontario, Canada

²Department of Community Health and Epidemiology, Queen’s University, Kingston, Ontario, Canada

³Clinical Evaluation Research Unit, Kingston General Hospital, Kingston, Ontario, Canada

⁴Center for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine, St. Paul’s Hospital and University of British Columbia, Vancouver, B.C, Canada

⁵Centre de Recherche Clinique Étienne-Le Bel, Université de Sherbrooke, Sherbrooke, Québec, Canada

⁶Departments of Medicine, and of Clinical Epidemiology & Biostatistics, McMaster University, Hamilton, Ontario, Canada

⁷Division of Palliative Care, Department of Family Practice UBC, Canada

⁸Fraser Health Authority, Surrey BC, Canada

⁹Vancouver Coastal Health and School of Nursing University of British Columbia, Canada

¹⁰Division of Palliative Medicine, Department of Oncology, University of Calgary, Alberta, Canada

¹¹Department of Internal Medicine, University of Calgary, Alberta, Canada

¹²Tom Baker Cancer Centre, Calgary, AB, Canada

Corresponding Author:

Daren K. Heyland
Angada 4, Kingston General Hospital, Kingston
Ontario, K7L 2V7, Canada
Tel: 613-549-6666/4847
Fax: 613-548-2428
E-mail: dkh2@queensu.ca

Received Date: April 13, 2012; Accepted Date: June 18, 2012; Published Date: June 20, 2012

Citation: Heyland DK, Pichora D, Dodek P, Lamontagne F, You JJ, et al. (2012) The Development and Validation of a Questionnaire to Audit Advance Care Planning. J Palliative Care Med 2:119. doi:10.4172/2165-7386.1000119

Copyright: © 2012 Heyland DK, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Palliative Care & Medicine

Abstract

Background: Advance Care Planning (ACP) is established as a standard of care in many settings. However, no validated tool exists to measure participants’ perspectives on ACP. The purpose of this study was to develop and validate a questionnaire to assess ACP from the perspectives of patients and their family members, and to pilot the use of this questionnaire in both English and French.
Methods: To develop the ACP-specific questionnaire, items were generated from focus groups with health care professionals and face-to-face interviews withhospitalized patients and their families. The items from this novel questionnaire were combined with other validated evaluation instruments and then piloted in English-speaking hospitalized patients who had advanced, life-limiting illnesses and a version for their family members. Revisions were made based on that experience and feedback from content experts and this questionnaire was then piloted in another sample of French-speaking respondents.
Results: The novel questionnaire was divided into 2 parts, the first part focusing on ACP activities before admission to hospital (6 questions) and the second part relating to ‘goals of care’ discussions that occur during hospitalization (8 questions). For the English pilot study, the average duration of the entire interview (including consent, baseline demographics and other questionnaires) was 53.1 mins (range 35-80 mins) for patients and 60.8 mins (range 33-125 mins) for family members. English-speaking patients rated the burden of participating in the interview as a mean of 2.8 (Standard Deviation [SD] 1.9, 1=no burden, 10=extreme burden) and family members as 1.9 (SD 1.9). For the French pilot study, the results were similar.
Conclusions: This is the first ACP questionnaire to be developed that has face and content validity. Despite a relatively lengthy interview process, the ACP audit process seems feasible and is not associated with undue burden.

Keywords

Advance care planning; End-of-life; Questionnaire

Introduction

Many patients will require decision-making about specific interventions and overall goals of care at the end of their lives, but this is frequently a time when they lack the capacity to make decisions [1]. By having conversations about their preferences regarding endof- life care with loved ones and health care professionals, and by documenting these preferences in advance, there is a higher likelihood that these preferences will be realized [2]. Furthermore, there is emerging evidence that communication about end-of-life care or Advance Care Planning (ACP) can have positive effects on the dying experience. In a recent randomized trial done in Australia, more than 300 patients 80 years or older were randomized to participate in ACP or usual care. Of those who received the ACP intervention, 108 (84%) expressed wishes or appointed a surrogate, or both. Of the 56 patients who died within six months, EOL wishes were much more likely to be known and followed in the intervention group (25/29, 86%) than in the control group (8/27, 30%; P<0.001). In the intervention group, family members of patients who died had significantly less stress, anxiety, and depression than those of the control patients [3].

The converse can be said for the absence of communication and planning at the end of life. In an observational study of terminal cancer patients, the absence of ACP in any of its forms was associated with poorer patient ratings of quality of life in the terminal phase of the illness, lower ratings of satisfaction by their families during the terminal illness, and increased family ratings of anxiety and depression [4]. In contrast, when physicians and patients/families engaged in ACP, there was less ‘intensification of care’ (use of intensive care units, life-sustaining technologies, or feeding tube insertions) and greater use of hospice services [4].

ACP is also associated with significantly lower health care costs during the final week of life [5]. In patients who had terminal cancers, the mean (standard error) aggregate cost of care (in 2008 US dollars) was $ 1876 ($ 177) for patients who reported EOL discussions compared with $ 2917 ($ 285) for patients who did not, a cost difference of $ 1041 (P=0.002). In the United States, based on the annual number of cancer deaths reported, this could translate into savings of over $ 75 million per year [5].

Given the initial success of the Respecting Choices program in the United States [6], ACP has become an established standard of care and many clinicians around the world have begun to implement various forms of ACP and to make the systemic changes that would embed ACP into health care. However, low rates of uptake have been reported [7-9]. There has been no evaluation in Canada of the effectiveness of these efforts from the perspective of patients and families. The purpose of this study was to develop and validate a questionnaire that could be used in both official languages (English and French) to determine the extent to which patients and their families have engaged in ACP and to determine their current satisfaction levels with end of life communication and decision making.

Methods

We conducted a multi-center prospective study to develop and validate a novel questionnaire to audit ACP practices. To generate items for the questionnaire, we considered that the essential components of ACP at the individual level include: 1) asking the patient about their personal values and wishes related to care provided at the end of life, 2) disclosing to the patient (and family) their prognosis, 3) discussing with both patient and family the various treatment options, and their associated risks, benefits, and expected outcomes, 4) deciding on future care or goals of care if the patient is not able to engage in future discussions in a manner that is consistent with the patient’s preferred role in decision-making, and 5) documenting these discussions and decisions in some way that is accessible to health care providers in various settings of care. Accordingly, we drafted a set of questions pertinent to the different steps of the ACP process and then used this questionnaire to assess whether patients and their families had engaged in each of those steps.

After generating potential items for this questionnaire, we held a focus group with 25 selected members of the Canadian Researchers at the End of Life Network (CARENET) in attendance at their biannual research protocol developmental meeting. This group includes experts in critical care medicine, palliative care medicine, nursing, research methodology, and psychometrics and they provided input on the content of the questionnaire and study procedures in a session facilitated by the Principal Investigator. Response options were “yes or no” for some items and multiple choice for other items. In several other sections of the questionnaire (such as asking about reasons for not completing an advance directive), we initially used open-ended questions. In addition to measuring ACP practices, an additional objective of this audit was to evaluate the quality of these decisions made near or at the end of life. We used validated instruments to measure 1) satisfaction with EOL communication and decision-making (the recently validated Canadian Health Care Evaluation Project (CANHELP) Questionnaire [10]), 2) decisional conflict (Traditional 16-item Decisional Conflict Scale [11]), and 3) patient and caregiver values regarding end-of-life (End-of-Life Values scale [12]). Accordingly, these 3 instruments were added to the interview after the novel questionnaire about ACP practices to assess the feasibility of combining the novel ACP evaluation questionnaire with these validated measurement tools.

At 2 sites (Kingston General Hospital, Kingston, Ontario and St. Pauls Hospital, Vancouver, British Columbia), we attempted to enroll 30 patients or family members of eligible patients to assess the clarity, sensibility, and acceptability of this questionnaire. Study research nurses conducted face to face interviews with consenting patients or family members (separately). At the conclusion of the interview, the research coordinator administered a separate questionnaire for the respondent to rate the clarity of the wording, appropriateness of the response options, comprehensiveness (were there other important aspects of ACP that should be considered?), relevance, and overall acceptability of the ACP questionnaire. Response options were: “Very Poor”, “Poor”, “Fair”, “Good”, “Very Good”, and “Excellent.” If problems were identified, we sought to identify the exact question(s) that were problematic for the respondent. In addition, all participants rated the overall burden of participating in the interview on a 10 point scale where 1=“Not a burden at all” and 10=“Extremely burdensome.”

At the conclusion of this pilot study in English, we considered feedback from patients, families, and research coordinators to make additional changes to the questionnaire. Once the English version of the questionnaire was finalized, it was translated into French, backtranslated into English by a professional translator and reviewed for inconsistencies in meaning with the Principal Investigator. Then, we piloted the questionnaire in a convenient sample of 20 French-speaking subjects at the Centre Hospitalier Universitaire de Sherbrooke, Sherbrooke, Quebec. To further minimize the overall burden of the study to the patients and to gain a different perspective on the evaluation of the questionnaire, acceptability was assessed by the study nurses after the interview and thus, the results reflect their assessment of the clarity, sensibility, and acceptability of the questionnaire in a French-speaking population.

From direct interviews with the patient and/or family member, and chart abstraction, we captured standard baseline demographics including a brief frailty scale [13], numbers of co-morbid illnesses, and other key demographic information related to preferences for care.

Study Subjects

1. For this study, we enrolled patients who were at high risk of dying and/or their family members (where available), and thus for whom ACP is highly relevant. We define ’high risk’ according to the following criteria:55 years or older with one or more of the following diagnoses:

Chronic obstructive lung disease - 2 of the 4 of: baseline PaCO2 of > 45 torr, cor pulmonale; respiratory failure episode within the preceding year; forced expiratory volume in 1 sec < 0.5 L.

Congestive heart failure - New York Heart Association class IV symptoms and left ventricular ejection fraction < 25%.

Cirrhosis - confirmed by imaging studies or documentation of esophageal varices and one of three conditions: a) hepatic coma, b) Child’s class C liver disease, or c) Child’s class B liver disease with gastrointestinal bleeding.

Cancer - metastatic cancer or stage IV lymphoma.

End-stage dementia (inability to perform all activities of daily living, mutism or minimal verbal output secondary to dementia, bedbound state prior to acute illness)

OR

2. Any patient 80 years of age or older admitted to hospital from the community because of an acute medical or surgical condition.

These clinical criteria identify a patient population that, on average, has a 50% probability of death in 6 months [14]. We have successfully used these eligibility criteria and the recruitment strategy below in our previous evaluations of quality of end-of-life care [15,16].

Potentially eligible patients were identified by a study nurse after reviewing the hospital record and in consultation with the attending physician, medical residents, or responsible nurse. Patients unable to communicate due to language limitations (English or French only) or cognitive reasons were excluded but if their family member was available, we still approached the family member. The research nurse approached eligible patients and families, explained the purpose of the study and obtained informed consent. After obtaining written informed consent from willing subjects, the research assistant conducted separate interviews with patients and family members so that their responses would remain independent.

Analysis

The analysis was largely descriptive. The focus of this paper is on the methodological development of a novel ACP questionnaire and, when combined with other validated questionnaires, the feasibility, acceptability and burden of the whole questionnaire process. To assess for differences in the demographics between English and French respondents, p values for continuous variables were calculated by Wilcoxon-Mann-Whitney test and for categorical variables, we used Fisher’s exact test. A p value of <0.05 was considered to be statistically significant.

This study was approved by local Research Ethics Boards of Queen’s University, Providence Health Care, and Sherbrooke Hospital. All participants provided written informed consent.

Results

In our initial consultation with experts, we realized that aspects of EOL communication and decision-making or ACP occur both before hospitalization and during hospitalization. Accordingly, we divided our questionnaire into 2 parts, the first part covering ACP activities before admission to hospital (6 questions) and the second part relating to health care decision-making and ’goals of care‘ discussions that occurred during the index hospitalization (8 questions). Table 1 describes the nature of the questions pertaining to ACP practices (final version).

Table 1: List of Questions Included in the Final ACP Questionnaire.

In the initial pilot in English-speaking respondents, we approached 19 eligible patients and 16 eligible family members; 15 patients and 16 family members provided informed consent. There were 2 incomplete interviews because one patient was taken for a test and discharged next day and another patient became too tired to complete the interview.

In the French pilot study, we approached 14 eligible patients and 12 eligible family members; 12 patients and 8 family members provided informed consent. The characteristics of participating patients and family members are in Table 2.

Table 2: Characteristics of Patient and Family Members.

For the English pilot study, the average duration of the interview including the consenting procedure and obtaining baseline demographics was 53.1 mins (range 35-80 mins) for patients and 60.8 mins (range 33-125 mins) for family members. Because of the long duration of some of the interviews, at the end of the piloting process, we reduced items on the questionnaire, changed the majority of the open-ended questions to closed ended, changed from the 16 item Decisional Conflict Scale to a 4 item version, and used only relevant domains of the CANHELP satisfaction questionnaire instead of the complete questionnaire. These were: Relationship with Doctors, Communication, Decision-making, and Role of the Family for the patient version and Relationship with Doctors, Communication and Decision-making, and Your Involvement for the Family member version. For the French pilot study, which used this shorter, revised questionnaire, the average duration of the interview including the consenting procedure and obtaining baseline demographics was 52.8 mins (range 38.0-75.0 mins) for patients and 53.0 mins (range 35.0- 78.0 mins) for family members.

The results of the evaluation of the questionnaire are shown in Table 3 (patients) and Table 4 (family members), for both the English and French pilot samples. In the general comments section of the form, some English-speaking respondents noted that the questionnaire was too long. Problematic concepts or words were identified and revised on an ongoing basis. In response to the question on comprehensiveness, no additional items related to ACP were identified. English-speaking patients rated the burden of participating in the interview as a mean of 2.8 (standard deviation 1.9, range 1-7) and family members as 1.9 (standard deviation 1.9, range 1.0-4.0). Using the modified version of the questionnaire, the Research Coordinators perceived the burden of participating in the interview as mean 1.5 (standard deviation 1.2, range 1.0-5.0) and family members as 1.0 (standard deviation 0.0, range 1.0-1.0).

Table 3: Assessment of the Study Questionnaire by Patients and Study Nurses.

Table 4: Assessment of the Questionnaire by Family Members and Study Nurses.

Discussion

We went through a thorough and comprehensive process to develop and validate a questionnaire to elicit patient and family member perspectives on the ACP process, both pre-hospital and during the index hospitalization. We considered that because the development process included basing the items on published literature and existing frameworks, consulting with experts, and piloting with the target population, the questionnaire has face and content validity. Based on the evaluation of the questionnaire, from the perspective of patients, families and study personnel, it would seem that the questions are clear, easily understood and therefore, responses will tell us what we want to know.

Herein, we do not report the ‘results’ of the questionnaire itself. Rather, we focus on the developmental procedures and evaluation of the questionnaire process. We considered that we had too few participants (27 patients and 24 family members) to state with any degree of confidence or reliability what the answers to our questions might be. The final version of this questionnaire is currently being used in a multicenter study, Advance Care Planning Evaluation in Elderly Patients: A multicenter, prospective study (the ACCEPT Study). For more information on the questionnaire or the ACCEPT study, we refer readers to our website, www.thecarenet.ca. We note that this questionnaire is not a tool to promote advance care planning, like so many other tools that have been developed. But rather, this questionnaire is an audit tool that will answer key questions about the quantity and quality of ACP and in-hospital medical decision making, and the barriers to these processes.

In a qualitative study of ACP in advanced cancer, over one third of participating patients suggested that it was “too soon” for them to discuss ACP [17]. In contrast, the clinical criteria we used in this study enabled us to identify a hospitalized patient population that was willing to discuss ACP. In piloting this novel questionnaire, we encountered a very low consent failure rate (17%) and all but 2 participants finished the interview process confirming the feasibility of the evaluation process. Moreover, despite taking almost one hour to complete, the majority of participants considered the experience as acceptable and rated their participation as not very burdensome. Anecdotally, the reason for the prolonged interview was that participants had many stories to tell and concerns to share with the research nurses, prompted by the questions that were asked.

We acknowledge that there are other methods to evaluate the success of ACP within institutional settings. Others have examined hospital records or administrative databases for evidences of discussions and documentations of ACP or goals of care [18,19]. However, these chart audits do not provide any account from the patient or family member’s point of view on whether in fact the conversation actually occurred or any evaluation of the quality of the interaction. We also acknowledge that patients’ and family members’ perspective are not the only views that matter in ACP, and that doctors’, nurses’, and other health care providers’ views are also important and may illuminate barriers and facilitators to ACP. Accordingly, we have developed a separate study to evaluate these health care professionals views on ACP (Decision-making about goals of care for hospitalized medical patients: A multicentre, mixed-methods study [The DECIDE Study]) [20]. The data we generate from evaluating all these perspectives (patient, family member, doctor, and nurse) will most certainly inform quality improvement activities that should result in increased quantity and quality of end of life communication and decision making.

This study has several limitations. We have already acknowledged the limited sample size and limited number of participating centers. Thus our findings of the acceptability and feasibility of engaging hospitalized patients and their family members in an evaluation of ACP practices may not be generalizable to other acute care settings. Whilst we did include both English- and French-speaking respondents in our study, we did not evaluate the questionnaire in other languages or other settings such as primary care or long-term care settings. Finally, we did not assess the reliability of the results in the context of this pilot study. Future work with this questionnaire will have to establish the acceptability and feasibility of this audit tool in these different patient populations and practice settings.

We conclude that hospitalized patients and their family members are willing to discuss ACP issues and participate in a questionnaire process that evaluates the quantity and quality of ACP, from their perspective. Notwithstanding the fact that it takes considerable time to administer, we have developed and validated a questionnaire that can be used, in English and in French, to ‘audit’ ACP experienced by patients and their family members prior to and during hospitalization in an acute care setting.

References

1. Silveira MJ, Kim SY, Langa KM (2010) Advance directives and outcomes of surrogate decision making before death. N Engl J Med 362: 1211-1218.
2. Harle I, Johnston J, Mackay J, Mayer C, Russell S, et al. (2008) Advance Care Planning with Cancer Patients: Evidentiary Base and Guideline Recommendations. Evidence-Based Series #19-1. Toronto: Program in Evidence-Based Care: A Cancer Care Ontario Program
3. Detering KM, Hancock AD, Reade MC, Silvester W (2010) The impact of advance care planning on end of life care in elderly patients: randomised controlled trial. BMJ 340: c1345.
4. Wright AA, Zhang B, Ray A, Mack JW, Trice E, et al. (2008) Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA 300: 1665-1673.
5. Zhang B, Wright AA, Huskamp HA, Nilsson ME, Maciejewski ML, et al. (2009) Health care costs in the last week of life: associations with end-of-life conversations. Arch Intern Med 169: 480-488.
6. http://respectingchoices.org/
7. Heyland DK, Allan DE, Rocker G, Dodek P, Pichora D, et al. (2009) Discussing prognosis with patients and their families near the end of life: Impact on satisfaction with end of life care. Open Med 3: 71-80.
8. Thorevska N, Tilluckdharry L, Tickoo S, Havasi A, Amoateng-Adjepong Y, et al. (2005) Patients' understanding of advance directives and cardiopulmonary resuscitation. J Crit Care 20: 26-34.
9. Ashby M, Wakefield M (1993) Attitudes to some aspects of death and dying, living wills and substituted health care decision-making in South Australia: public opinion survey for a parliamentary select committee. Palliat Med 7: 273-282.
10. Heyland DK, Cook D, Rocker G, Dodek PM, Kutsogiannis DJ, et al. (2010) The development and validation of a novel questionnaire to measure patient and family satisfaction with end of life care: The Canadian Health Care Evaluation Project (CANHELP) Questionnaire. Palliat Med 24: 582-695
11. O'Connor AM (1995) Validation of a decisional conflict scale. Med Decis Making 15: 25-30.
12. Parks SM, Winter L, Parker B, Collins L (2011) End of life values in proxy decision making.
13. Rockwood K, Song X, MacKnight C, Bergman H, Hogan DB, et al. (2005) A global clinical measure of fitness and frailty in elderly people. CMAJ 173: 489-495.
14. (1995) A controlled trial to improve care for seriously ill hospitalized patients. The study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT). The SUPPORT Principal Investigators. JAMA 274: 1591-1598.
15. Heyland DK, Dodek P, Rocker G, Groll D, Gafni A, et al. (2006) What matters most in end-of-life care: perceptions of seriously ill patients and their family members. CMAJ 174: 627-633.
16. Heyland DK, Cook DJ, Rocker GM, Dodek PM, Kutsogiannis DJ, et al. (2010) Defining priorities for improving end-of-life care in Canada. CMAJ 182: E747-E752.
17. Barnes KA, Barlow CA, Harrington J, Ornadel K, Tookman A, et al. (2011) Advance care planning discussion in advance cancer: Analysis of dialogues between patients and care planning mediators. Palliat Support Care 9: 73-79.
18. Hammes BJ, Rooney BL, Gundrum JD (2010) A comparative, retrospective, observational study of the prevalence, availability, and specificity of advance care plans in a county that implemented an advance care planning microsystem. J Am Geriatr Soc 58: 1249-1255.
19. Halpern NA, Pastores SM, Chou JF, Chawla S, Thaler HT (2011) Advance directives in an oncologic intensive care unit: a contemporary analysis of their frequency, type, and impact. J Palliat Med 14: 483-489.
20. www.thecarenet.ca