Journal of Community & Public Health Nursing
Open Access

Research on gatekeeper training, a suicide prevention strategy, has occurred at international, national, state and community levels. On an international level, community-based gatekeeper training remains part of broad suicide prevention strategies as outlined in the World Health Assembly’s first-ever Comprehensive Mental Health Action Plan [1] and in the World Health Organization’s Preventing Suicide - A Global Imperative [2]. The U.S. is among 28 countries adopting national suicide prevention plans that include gatekeeper training [2]. The National Strategy for Suicide Prevention identifies gatekeeper training as instrumental among its goals and objectives [3]. Additionally, Healthy People 2020 targets reducing the suicide rate [4]. In Montana, a rural U.S. northwestern state, Question, Persuade, Refer Suicide Prevention Gatekeeper Training (QPR) [5] was highlighted in two public health initiatives [6,7]. Aligning with state efforts and a local health improvement plan, one area in central Montana adopted QPR [8].

The aim of this study was to describe and compare cohort pre-post responses to suicide prevention gatekeeper training. The objective was to gain an understanding of the effectiveness of an evidenced-based suicide prevention training in a rural setting with one of the highest suicide rates in the U.S.

The background presents Montana suicide rates, the history of a community-based Suicide Prevention Coalition and QPR gatekeeper training.

Montana has a long-documented history of increasing suicide rates ranking among the highest in the U.S. [7]. The age-adjusted rate of suicide in Montana was 23.4 per 100,000 [7,8], significantly higher than the U.S. rate of 12.6 [9] and the 11.4 international rate [2]. Of significance was the age-adjusted rate of 22.8 for Yellowstone County which ranks among the highest in the state [8].

Rising suicide rates are anticipated to continue as reported by economists, Anne Case and Angus Deaton, recipients of the 2015 Nobel Prize who identified an alarming trend among aging babyboomers. They found a rise in suicides for those currently in midlife due to declines in physical and mental health [10]. Consistent with their findings were midlife Montanans, aged 55 to 64, who comprised the largest group of suicide deaths [7].

Local initiative - Suicide prevention coalition of yellowstone valley

In 2003, the Suicide Prevention Coalition of Yellowstone Valley (SPCYV) began as a grassroots movement in collaboration with the American Foundation for Suicide Prevention (AFSP) Montana Chapter. As of 2017, 13 volunteers were certified QPR instructors providing gatekeeper training. About 75 volunteer SPCVY members representing medical, nursing, public and mental health communities, social service agencies, secondary and higher education, local government and survivors of suicide loss meet monthly at a public health agency. Other Coalition community-based activities include support for the annual American Foundation for Suicide Prevention Out of the Darkness Walk [11].

Question, persuade, refer gatekeeper training

Question, Persuade, Refer (QPR) is used in Australia, Canada, France, Ireland, Israel, Italy, Puerto Rico, Qatar, Spain and the U.S. with more than one million persons trained [12]. It is among several suicide prevention education and training programs listed in the U.S. National Registry of Evidence-Based Programs and Practices (NREPP) [13]. The NREPP is an online registry of 350 substance abuse and mental health interventions evaluated by the U.S. Substance and Mental Health Services Administration (SAMHSA) for demonstrating positive outcomes and effectiveness in prevention or treatment.

Question, Persuade, Refer is a three-step program instructing participants how to ask whether someone is thinking about suicide, how to persuade someone to seek help and how to refer at-risk persons to appropriate resources [14]. The program included awareness of risk factors and warning signs, thereby promoting early identification and referral. Gatekeepers typically encounter suicidal persons given their occupation as healthcare professionals and lay persons such as teachers, coaches, clergy, resident hall advisors, firefighters and police officers. From a population-based perspective, the theory is that the greater percentage of those trained to recognize and refer, fewer suicides should occur. Therefore, gatekeeper training remains fundamental to a strategic public health approach [14].

Literature review

A literature search was conducted through online library databases, including Academic Search, CINAHL, Medline, PsycINFO and Psychological and Behavioral Sciences using terms: ‘suicide prevention gatekeeper training.’ Dates searched were 2000 through January 2017 yielding 728 full-text scholarly articles in English. Article selection was based on two inclusion criteria: first, research relevant to a publichealth approach and second, participants represented the adult public (i.e. teachers, clergy) and not exclusively healthcare professionals. Twenty-one articles met inclusion criteria.

Studies of gatekeeper training have been implemented in various settings in North America, Europe and Australia. With few exception, they were found to be beneficial and feasible to varying degrees. This review begins with nationwide programs and progresses to studies implemented statewide, in communities, in education and organizations.

Nationwide programs

The U.S. Air Force in 1997 began an extensive suicide prevention program which included gatekeeper training targeting military and civilian employees [15]. A cohort study examining effects on more than five million personnel between 1997 and 2002 identified a 33% relative risk reduction in suicide post-intervention compared to previous years [15]. Re-examining outcomes in 2010, authors recommended on-going education and training for all personnel asserting it would reduce stigma, raise awareness and normalize helpseeking behaviors [16].

In a pre- and post-test study design, frontline administrative and outreach staff were among a national cohort (N=602) receiving QPR and scripted practice sessions in 209 U.S. Veteran’s Administration counseling centers [17]. Based on survey data, researchers identified significant improvement from pre-post-training in knowledge and selfefficacy (p<0.0001) with greatest impact on knowing what to say and do. Authors concluded the training was a promising intervention [17].

Statewide

In the U.S., statewide suicide prevention programs studies were conducted in Kentucky, Tennessee and Virginia. In Kentucky, researchers examined pre-post QPR surveys (n=3,958) from 2004 to 2006 [18]. Participants included educational faculty and staff, corrections staff and faith-based persons. Overall, participants were satisfied with training, except for role-play which did not enhance knowledge or skill [18].

Tennessee Lives Count gatekeeper training targeted social service workers, educators and lay persons [19]. Pre-post surveys (n=416) about attitudes, knowledge and self-efficacy revealed initial improvements but gains were not maintained at six months with authors concluding long-term effects warranted further examination [19].

Results of initial training and at four months among school counselors (n=73) and teachers (n=165) in Virginia were compared to a control who did not receive training [20]. Those trained demonstrated greater knowledge of risk factors and initiated more noharm contracts. While both groups benefited, teachers relied on school counselors for referrals suggesting a need for training based on occupation [20].

Communities

Outcomes of gatekeeper training were mixed among four community-based studies in Australia, Canada and the U.S. In Tasmania, the Community Response to Eliminating Suicide (CORES) program included gatekeeper training [21]. Results of pre-post surveys revealed participants increased comfort and confidence and demonstrated greater awareness of suicide as a public health problem and knowledge of prevention resources [21].

In Manitoba, Canada, a randomized-controlled study of gatekeeper training found increased suicidal ideation occurred among First Nations tribal community members [22]. One group (n=31) received two days of Applied Suicide Intervention Skills Training (ASIST) and another group (n=24) attended a Resilience Retreat without focus on suicide awareness. Authors concluded reexamination of ASIST, used widely in Canada, was warranted given reported increase in suicidal ideation among some participants [22].

Studies in the U.S were conducted in Arizona and Oregon. On the Fort Apache Indian Reservation in Arizona, American Indians were among participants in ASIST training (n=84) [23]. Positive gains in knowledge, self-efficacy and intent to use were identified, however participants reported training did not address cultural needs [23].

In a study incorporating qualitative data, Japanese Americans in Oregon comprised a convenience sample of those completing a multimodal QPR training [24]. Controls (n=48) received only lecture, while those in the intervention group (n=85) attended QPR training with role-play and viewed a suicide documentary. Those in the intervention group showed improvement in knowledge, attitudes and self-perceived skill while the control group showed gains in only knowledge [24].

Education

Gatekeeper trainings studies have been conducted in higher and secondary education. Most have occurred in the U.S.

Higher education

Gatekeeper training studies reported in higher education have occurred in several U.S. regions and in Japan with mixed results. In five northeastern U.S. universities, a study of faculty and staff (n=50) found QPR promising from a public health perspective [25]. However, approximately half (46%) of participants failed to demonstrate ability to ask and refer. In contrast, results of a university-wide study in southeastern U.S. comprised of faculty, staff and students (n=917) revealed significant improvement at post-training in fund of knowledge about suicide, prevention, resources and asking the suicide question with gains maintained at three months [26].

In a northeastern university, similar outcomes, with a significant exception, were reported in a QPR study of support staff and parents (n=273) completing pre-post testing and follow-up at three and six months. Outcomes revealed discomfort asking the suicide question [27]. Discomfort asking about suicide thoughts was also identified in a study of 240 resident hall advisors receiving QPR across the northwest [28].

Another study in a northeastern U.S. university hospital included standardized role-play scripts with 76 administrators and secretaries receiving QPR training. Outcomes found half the role-play participants demonstrated satisfactory competence in gatekeeper skills suggesting the necessity of practice [29].

Results of a descriptive study implementing QPR yielded both quantitative and qualitative self-reported data collected from senior baccalaureate nursing students (n=150) at a western university [30]. Results showed statistically significant (p<0.000) gains in knowledge, skills and abilities. Qualitative outcomes revealed a main theme of ‘becoming capable intervening with persons at risk for suicide.’ It was recommended future studies examine retention and application of gatekeeper skills [30].

In Japan, gatekeeper training with role-play was conducted among 76 university administrative support staff with positive outcomes in competence, confidence and intent to intervene. It was concluded cultural factors be considered in the future [31].

Public education

Two randomized-controlled QPR studies conducted in the U.S. reported mixed results on gatekeeper training in secondary school settings. A study of school personnel (n=249) in 32 Georgia schools at one-year follow-up identified open communication, warmth and empathy as essential to effective training [32]. Another study of New York school personnel and parents (n=170) compared training with training plus role-play. Results showed only immediate improvement in knowledge and attitude with traditional training and role-play however both formats showed loss at follow-up [33].

Organizations

Three scholarly studies examined effects of gatekeeper training on organizational culture in France, the U.S. and U.K. In France, a study compared outcomes of gatekeeper training implemented in 12 longterm care facilities to a control group of 12 similar settings without training [34]. It was found training increased interactions among staff with those residents at-risk, improving patient care [34].

Similar findings of increased dialogue with those at-risk was an outcome of a pilot quality improvement project incorporating gatekeeper training in a long-term care facility in the western U.S. [35]. Self-evaluative pre-post survey outcomes suggested personnel (n=43) could better identify and manage at-risk persons. The setting’s leadership reported training fostered therapeutic dialog with residents illuminating those at-risk suggesting a change organizational climate [35]. Organizational climate was instrumental to effective implementation of gatekeeper training found in one qualitative study (n=58) of ASIST in Wales [36]. Gatekeeper training has been used extensively in Wales, Scotland and Northern Ireland [36].

This was a descriptive study examining pre-post responses of community-wide QPR conducted by Suicide Prevention Coalition members certified as QPR gatekeeper instructors. Data were collected from September 2013 through September 2016.

Setting

The setting was the SPCYV service area, defined as Yellowstone and adjacent counties located in central Montana. Gatekeeper trainings were conducted at various settings, including churches, social service agencies, public health departments, mental health facilities, public school, one college, one university and a suicide prevention conference.

Participants

Inclusion criteria for participants were: at least 16 years of age; English speaking; voluntarily attend the 90-minute training; and anonymously complete the self-report pre-post training survey form. Excluded were outdated survey forms and survey forms completed by participants who received training outside the service area. Each participant received the accompanying QPR booklet funded by Montana’s Public Health and Human Services.

A written two-page self-report pre-post-training survey developed by the QPR Institute was utilized for data collection. The original survey was expanded with permission from a 3-item Likert scale to a 5-item Likert scale. The pre-training survey items included demographics, knowledge, self-efficacy and intent to intervene. The post-training items included: the same knowledge, self-efficacy and intent to intervene items as the pre-survey; three items rating the instructor and training; one item recommending training; and a written comments section. The pre-post survey has acceptable internal consistency assessing self-perceived knowledge (Cronbach’s alpha 0.94) and self-efficacy (Cronbach’s alpha 0.81) and high test, re-test reliability at 0.05 level or stronger [17,32,33]. Three studies [17,32,33] utilizing the survey instrument contributed to its established use as evidence based [12].

Demographic items included age, gender, ethnicity (African- American, Asian-American, Caucasian, Latino/Hispanic, Native American and Other) and highest grade completed (junior high, high school, trade/vocational school, two years of college, four years of college and five or more years of college).

Seven pre-post items rated knowledge of suicide using the 5-item Likert scale (very high, high, medium, low and very low) in the following areas: a) facts concerning suicide prevention, b) warning signs of suicide, c) how to ask someone about suicide, d) persuading someone to get help, e) how to get help for someone, f) information about local resources for help with suicide, g) level of understanding about suicide and suicide prevention. The self-efficacy and intent to intervene items used a 3-item scale (always, sometimes, never) to rate the following: h) Do you feel asking someone about suicide is appropriate? and i) Do you feel likely to ask someone if he is thinking of suicide? Items on post-training survey rated instructors using a 5- item Likert scale (excellent, very good, good, fair and poor) on: knowledge of subject matter; presentation of material; and overall quality of training. One item focused on recommending training choosing, ‘yes’, ‘no’, or ‘undecided’. Lastly, a ‘comments’ section was provided.

Exempt status from Montana State University’s Institutional Review Board was received. Gatekeeper training was conducted upon invitation of organizations and provided by volunteer certified QPR instructors and received no compensation. Participants were invited to anonymously complete the pre-post survey.

A total of 894 pre-post-training surveys were completed by adults aged 18 years of age and older completing QPR training. Surveys were collected from QPR trainings conducted September 2013 through September 2016, from the SPCVY service area encompassing six counties.

Demographics

Most of the sample was female (73%) and Caucasian (86%) with a mean age of 40 years. The highest grade of education completed varied (Table 1).

Table 1: Ethnicity and education demographics.

Quantitative data analyses

Data were organized into nine cohorts based on settings and analyzed using Statistical Software R. Surveys were organized into the following nine cohorts: Cohorts were: 1 faith-based organizations; 2 social service clinical and non-clinical staff and volunteers (e.g. campus counselors, correctional facilities); 3 public health department staff and clinicians; 4 community at-large (e.g. funeral directors, bus drivers); 5 mental health non-clinical staff and clinicians; 6 suicide prevention conference attendees (e.g. lay persons, mental health, educational staff); 7 public school teachers and staff; 8 senior nursing students; 9 allied health (e.g. social work, counseling, physician-assistant students and medical students).

There was a 98% overall response rate to the surveys. Responses to Likert questions were coded numerically from 1 to 5 or 1 to 3 respectively with positive responses receiving higher codes.

Item analysis

The Wilcoxon Signed-Ranks test with the conservative Bonferroni correction for multiple testing was utilized to analyze responses across cohorts for each item. Outcomes were statistically significant (p<0.00001). Means for each of the nine items show positive gains. The largest pre-post gains were on items c) how to ask someone about suicide, d) how to persuade someone to get help and f) information about local resources. Two questions using a 3-item Likert response show the least pre-post gains: h) Do you feel that asking someone about suicide is appropriate? and i) Do you feel likely to ask someone if he is thinking of suicide? (Figure 1).

Figure 1: Mean response score per item.

Cohort analysis

Differences in mean scores to each of the nine items comparing preand post-outcomes by cohort were statistically significant (p<0.0001). Each of the nine cohorts demonstrated positive pre-post gains (Figure 2).

Figure 2: Mean response by cohort and item.

To compare cohorts an exploratory factor analysis was conducted on the nine items identifying similar responses. Responses to items a through f were aggregated into a combined response and used for cohort comparison. The aggregate mean response was used for conducting an Analysis of Variance (ANOVA) to compare groups using the conservative Tukey’s Honest Significant Difference post hoc analysis for pairwise comparisons between groups. In general, cohorts with the lowest pre-training mean scores showed the largest gains at post-training.

The pre-survey had four distinct tiers identified by the post-hoc analysis. In tier 1, cohort groups with the lowest scores were faithbased, community at-large and nursing students; followed by tier 2 public health, mental health and public school; followed by tier 3 allied health and social services; followed by tier 4 suicide prevention conference attendees with the highest pre-scores. Outcomes show all cohorts demonstrated self-assessed improvement in knowledge and skills with the average cohort range improving by 0.78 for the suicide prevention cohort up to 1.3 for faith-based (Figure 3).

Figure 3: Aggregate score by cohort.

Post-training scores demonstrated a leveling effect of self-reported improvement in knowledge and skills between cohorts. At the 95% family-wise confidence level, only the community at-large cohort remained behind the other groups. All other cohorts demonstrated mean scores at post-training statistically indistinguishable from each other.

Participants evaluated the training and instructor using a 5-item Likert scale. The instructor means were 4.68 for knowledge of subject matter and 4.60 for presentation of material. Overall quality of training was rated 4.54. Most (91.5%) recommended training to others.

Qualitative data analysis

Textual data analysis was guided by qualitative content analysis [37], an iterative process interwoven with disciplined researcher reflexivity and consensual interpretive data analysis. Of the 894 surveys, 280 postsurveys included textual data in the comment sections representing 31% of the sample. Textual data were analyzed independent of quantitative data.

The textual data were organized and analyzed per cohort. The analysis process proceeded with the second author reading and rereading the textual data and extracting descriptive words and key phrases. The extracted data were condensed into meaning units which were labeled with codes (n=6). The codes in order of their frequency from highest to lowest were: appreciating (n=9), learning (n=8), seeing application (n=8), being touched (n=6), encouraging (n=6) and needing (n=6). First and second authors noted similarities, differences and patterns within and among cohorts staying close to manifest meanings within condensed meaning units.

As first and second authors reflected on condensed meaning units and codes, subthemes emerged with each cohort. Repetitive patterns among subthemes were identified and refined, revealing subthemes for the entire sample. Weaving together subthemes, four themes about suicide prevention emerged: sensing heartfelt appreciation; gaining knowledge, skills and attitudes; seeking to learn more over time; and envisioning more training, education and research. Table 2 reports the extraction-condensation processes for two of the nine cohorts and is presented here as an example. Allied Health Students presented all four themes, appreciation, gaining, seeking and envisioning. The Community At-Large cohort resulted in three themes, appreciation, gaining and envisioning (Table 2).

Table 2: Example of extraction-condensation processes.

Findings from this descriptive study are consistent with some outcomes reported from literature reviewed. These include high levels of satisfaction with gatekeeper training and gains in self-reported assessments of knowledge, attitudes and perceived skill.

Satisfaction with self-reported knowledge, specifically warning signs, was identified among all cohorts. This is significant given 74% of Montanans demonstrated identifiable warning signs prior to their deaths [7]. Therefore, QPR may meet an existing knowledge deficit in the community. Noteworthy was satisfaction with fund of knowledge among nursing students, a finding consistent with a previous study examining QPR implemented in a nursing program [30]. Training may address an identified knowledge gap within nursing curriculum which may have implications for other allied health programs [30]. In a review of evidence-based gatekeeper training conducted in rural settings, QPR and ASIST were identified as more efficacious than other trainings among the 10 studies examined [38]. Therefore, gatekeeper training may meet a need in rural settings where mental health resources are lacking [38].

In this study, there were no statistically significant differences found between cohorts. However, other studies identified occupation as a variable in outcomes suggesting different trainings be offered based upon occupational role [17,20,36].

In this study, QPR occurred at the invitation of various organizations but it remains unknown whether they have on-going training. Organizational structure and support were identified as key elements in creating a culture of suicide prevention and help-seeking behavior as the norm [21,35,36]. Similarly, organizational support contributed to success of the U.S. Air Force suicide prevention program and its reduced suicide rates [15,16]. However, in the U.S. Air Force as in the Australian CORES program, gatekeeper training was one of several interventions in a broad-sweeping approach. In one systematic review, it was found the best evidence for gatekeeper training exists within broader prevention programs, however, what remains unclear is the effect gatekeeper training singularly contributes [39].

Basic QPR for the general community was utilized across all cohorts in this study. The suggestion that gatekeeper training is not a ‘one size fits all’ may have relevance to minority groups. Lower levels of satisfaction and poorer outcomes, including a concerning increase in suicidal ideation among First Nations tribal community members in Canada, raises concern about appropriateness of standardized training for minority groups [22]. The issue of culture was raised by other researchers [23,31]. It has been concluded there was insufficient evidence to recommend gatekeeper training for indigenous persons in one systematic review of suicide prevention interventions [40].

Qualitative results support, enrich and add to an understanding of responses. While this study reveals more qualitative results than others, some similarities are noted with a few studies. One study included a qualitative arm where coding schemes were equivalent to established constructs in survey items (e.g. self-efficacy) [24]. Outcomes of qualitative analysis in this study revealed an overarching theme of ‘sensing heartfelt appreciation’ suggesting satisfaction as previously reported [24]. ‘Gaining knowledge, attitudes and skill’ could be related to self-efficacy [24,30]. ‘Seeking to learn more’ may reflect acknowledging a desire for more training time [24].

Five of the nine cohorts represent professionals, para-professionals, students and support staff in healthcare or healthcare curriculums and reflected all four themes. Of interest among healthcare cohorts was a recurrent theme of ‘seeking’ not consistently endorsed by the other four cohorts comprised of the general public. One interpretation of this contrast suggests lifelong learning, as represented by ‘seeking’, is inherent among those in healthcare.

The codes ‘appreciation’, ‘learning’, ‘seeing application’, ‘being touched’, ‘encouraging’ and ‘needing’ may reflect some survey items.

The code ‘appreciation’ identified among all cohorts may reflect on survey items rating instructor training and presentation. ‘Learning’ could reflect survey items of knowledge and skill. ‘Seeing application’ may reflect on recommendation of QPR to others.

Strengths

There are several identified strengths in this study. Findings align with international, national and statewide suicide prevention initiatives. Gatekeeper training is heartily supported by local suicide prevention groups suggesting it meets a community need. Results show statistically significant outcomes based on sufficient sample size. Inclusion of qualitative data analysis, seldom reported in gatekeeper studies, support quantitative results and enriches understanding. Moreover, results support QPR as an evidence-based program. The training is feasible, cost-effective and readily administered in a 90- minute timeframe. Verification of the results occurred through reflecting on, examining and re-examining all aspects of the study while respecting the expertise of each author until consensus among all authors was achieved. All authors confirmed the final version of the manuscript.

There are limitations to this study. First, the convenience sample was not representative of the general population. The samples of voluntary participants reside in a rural state with a high suicide rate and their established interest in prevention efforts could have influenced survey outcomes. Moreover, ethnicity of the sample was very homogeneous (Caucasian 86%) further limiting generalizability. The sample size was not calculated in advance but based upon trainings provided in a given timeframe. Second, instructors were volunteers whose teaching modalities and effectiveness varied. For example, some instructors incorporated role-play while others did not. Third, sustainability of outcomes over time is unknown since follow-up was not included. Fourth, triangulation, not included in the study, could have added another perspective, strengthening results. Fifth, embedded in qualitative research is an interpretive aspect which may be one of many interpretations [41]. Please refer to Table 2 which shows evidence of an audit trail. And, lastly, this study does not contribute to our current understanding of how gatekeeper training impacts suicide rates.

We recommend follow-up studies, for example, at 6-9-12 month intervals given reported inconsistent retention over time [42]. Future research should examine effectiveness of teaching methods (lecture versus lecture with film and role-play). Furthermore, gatekeeper training is not a ‘one size fits all’, therefore culturally diverse groups should be included.

This study described responses of nine cohorts to community-wide QPR suicide prevention gatekeeper training in a rural state with a high suicide rate. Quantitative outcomes showed statistically significant gains from pre-post training among all cohorts regarding perceived knowledge, attitudes and skill with little between-group variance. Qualitative outcomes further supported and enhanced quantitative results revealing an overarching theme of ‘appreciation’ (for learning about suicide prevention). Further research is recommended focusing on inclusion of ethnically diverse samples and sustainability of gatekeeper training over time.