Routine Outcome Monitoring in Mental Health Care and Particularly in Addiction Treatment: Evidence-Based Clinical and Research Recommendations

Department of Psychiatry, Leiden University Medical Centre, Leiden, The Netherlands

*Corresponding Author:

Ingrid Carlier
Department of Psychiatry, Leiden University Medical Centre
The Netherlands
Tel: 0031(0)715265237
Fax: 0031(0)715248156
E-mail: I.V.E.Carlier@lumc.nl

Received date: June 20, 2017; Accepted date: July 07, 2017; Published date: July 14, 2017

Citation: Carlier IVE, van Eeden WA (2017) Routine Outcome Monitoring in Mental Health Care and Particularly in Addiction Treatment: Evidence-Based Clinical and Research Recommendations. J Addict Res Ther 8:332. doi:10.4172/2155-6105.1000332

Copyright: © 2017 Carlier IVE, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Addiction Research & Therapy

Abstract

Routine Outcome Monitoring (ROM) concerns the repeated measurement of the progress of a patient’s treatment during the course of therapy. ROM is receiving mounting attention as an important quality tool, and because of increased emphasis on health care accountability and cost containment. ROM seems especially relevant for complicated chronic patients that require a long-term treatment. Although addictive patients often have these clinical characteristics, ROM is relatively less investigated within addiction treatment. The present article summarizes recent state-of-the-art information concerning the use of ROM in mental health care and particularly in addiction treatment. First, some basics about ROM in general are described. Next, evidence for the effectiveness of ROM in mental health care and addiction treatment is reviewed. Finally, some clinical & research recommendations are suggested for the use of ROM in addiction treatment.

Keywords

Addictive disorders; Substance use disorders, Substance abuse; Routine outcome monitoring; Treatment progress feedback

Introduction

Routine Outcome Monitoring (ROM) refers to the periodic assessment of patient variables, such as symptom severity, functioning and well-being. Assessments take place before, during and after treatment. Outcomes of these variables, like total score against normscores and rate of change, are given to therapist and preferably also to the patient. This process is often referred to as ROM Feedback [1]. The use of ROM as a component of evidence-based practice in mental health care has increased widely during the past decade [2]. Given the large number of publications, it is evident that ROM is a hot topic [3-6]. In ROM, Patient Reported Outcome Measures (PROMs) assess the patients’ self-report experiences of symptoms, functioning, health and/or quality of life. Although PROMs are most often used in ROM, other clinician-rated instruments may be used as well [7-10].

ROM can be an important quality tool for measuring the clinical response of patients during the course of treatment and thereby for optimizing clinical decisions of the therapist. Ultimately, when a patient is identified to be responding poorly only at the end of treatment, it is too late for a shift in treatment that may have resulted in a more favourable outcome [1,11]. This is all the more important, since empirical evidence indicates that about 30% of mental health patients fail to respond during clinical trials and 65% of patients in routine care leave treatment without a measured benefit [11]. Moreover, therapists appear to overestimate positive outcomes in their patients and they are poor at identifying patients at risk for a negative outcome [11]). These findings point to the need for repeatedly measuring the patient treatment response during treatment and for giving feedback to therapists about it.

In light of this, it is suggested that ROM and feedback methods should become a standard of practice in mental healthcare [11]. This seems particularly relevant for complicated patient groups, like those with multi-morbidity and/or receiving long-term treatment. Patients suffering from addiction are often characterized with such clinical characteristics [12,13]. Research shows that co-occurrence of addiction with other mental disorders (e.g. depression, anxiety-, eating-, personality-, psychotic disorders) as well as co-occurring addictions (i.e., tobacco, alcohol, drugs, problem gambling, pathological internet use) are the rule rather than the exception. Such diagnostic comorbidity may reflect underlying genetic vulnerabilities, personality patterns, or interactions between individual symptoms, which complicate treatment progress [14-16]. Elevation of comorbid symptomatology may evoke relapse or could trigger an increase in substance abuse. In addition to presenting a more complex array of problems, patients with co-occurring or chronic disorders are at risk of poorer outcomes following treatment, which add to the cost-of-illness [17,18]. Overall, these patients have comprehensive needs that require complex and long-term treatment, including several other services (e.g. rehabilitation, detox, supported housing; [19]). In these complex treatments, ROM may be useful for monitoring the versatile psychopathology, including other relevant aspects (e.g. trauma assessment) and treatment can be adjusted in a more timely manner. This is in agreement with recent studies regarding integrated treatments for co-occurring addictive and psychiatric disorders [20-22].

Despite the interest in, and potential benefits of ROM, its growth across the various psychiatric disorders has been rather uneven [3]. For instance, ROM is relatively less investigated in chronic psychiatric patients, like those with addictive disorders and possible co-occurring psychiatric disorders [6,11]. Therefore, the present article summarizes recent state-of-the-art information concerning the use of and evidence for ROM in mental health treatment. First, some basics about ROM in general are described. Next, evidence for the effectiveness of ROM in mental health care and specifically in addiction treatment is reviewed. Finally, some clinical and research recommendations are suggested for the use of ROM in addiction treatment.

Method

For the section on the effectiveness of ROM (including ROM feedback) with adult psychiatric patients, a literature search was carried out up to May 2017 according to guidelines [23,24]. Consequently, the most relevant databases/literature sources were used: e.g. Medline; Embase; PubMed; Cochrane Central Register of Controlled Trials; CINAHL; Web of Knowledge; Ovid PsycINFO; Web of Science; International trial registries; Google Scholar. Electronic searches were supplemented with manual scanning of reference lists of retrieved articles and reviews. Retrieved references were then screened for relevance. To be comprehensive, non-randomized but relevant studies (e.g. ROM in addiction treatment) were included. Main search terms were: routine outcome monitoring, (progress) feedback, treatment outcome, mental health/psychiatry, addiction, addictive disorders, substance use disorders, substance abuse, effectiveness, efficacy, cost-effectiveness. Excluded were publications concerning routine outcome management (merely benchmarking), children/ adolescents and non-mental healthcare.

Routine outcome monitoring

The routine measurement of outcomes has risen in prominence over the past years. ROM started originally in health care and somatic medicine (e.g. oncology, neurology), later it was also utilized in mental health care [1]. Implementation of ROM in mental health care was somewhat delayed, mainly because it proved to be more difficult than implementation in other areas of health care [25]. Several terms besides (routine) outcome monitoring are used in the context of ROM: e.g. (routine) outcome measurement, treatment progress monitoring, patient-reported outcomes monitoring, patient monitoring, personfocused outcome measurement, measurement-based care, practicebased evidence, measurement feedback system.

There now exist multiple ROM instruments/systems that can be used to measure the progress of patients during mental health treatment [11,26]. Some instruments or questionnaires are designed specifically for ROM, while others are not, but can be used nevertheless. Examples of generic mental health questionnaires designed specifically for ROM are: the Outcome Rating Scale [ORS; 27; see also paragraph "Practical use of ROM in addiction treatment"] and the Session Rating Scale [SRS; 27]; the Outcome Questionniare-45 (OQ-45) and OQ-System [28; see also paragraph "Practical use of ROM in addiction treatment"]; and the licence-free Symptom Questionnaire-48 (SQ-48 [29,30]; for its web-based ROM system [7]). Mental health questionnaires that are not specifically designed for ROM, but often are used in ROM include for instance: the Brief Symptom Inventory [BSI; 31]; the Symptom Checklist-90-Revised26 [SCL-90-R; 32]; and the Mood & Anxiety Symptom Questionnaire [MASQ; 33]. Some relevant overviews and comparison of ROM questionnaires/systems have been published [26,34,35]. Which ROM instrument(s)/system to choose are less important than the fact that ROM basically implies multiple measurements during treatment, and not solely a pre- and a post-treatment measurement. If necessary, follow-up measurements after treatment can also apply, especially with chronic psychiatric disorders such as addiction. Additionally, ROM implies providing ROM-feedback on patient progress to the therapist and preferably also to the patient. In fact, without these basic conditions, there is no ROM but merely assessment.

In this context, at least two conceptualizations can be considered in which links between processes and outcome of ROM feedback are theorized: Feedback Intervention Theory and Therapeutic Assessment [1,10]. Feedback Intervention Theory (FIT; [11,36-38]) implies that the provision of feedback to a therapist is a critical factor in the enhancement of self-regulation and knowledge for a therapist. Feedback to therapists works best with characteristics such as: e.g. immediate, frequent, simple, unambiguous, data-driven, novel information [36,37]. ROM feedback may initiate important management decisions for the clinician such as changing treatment, giving extra advice or education to the patient, or referring to other professionals [10]. Feedback may also help clinicians engage in thinking more deeply about patients and it provides them with a greater sense of professional identity [39]. Therapeutic Assessment (TA; [40,41]) on the other hand is focused on the therapeutic effects of providing test feedback of questionnaires to patients. In this context, discussing test-derived inferences with patients is part of a therapeutic strategy rather than diagnostics. TA usually involves three sessions and it is a mini-treatment in its own right, because it functions as a mean of patient self-verification, self-enhancement, self-efficacy, self-discovery [1]. Repeated ROM measurement might also increase patients’ symptom awareness, their ability to report relevant symptoms, their self-confidence and treatment compliance [39]. This could help the patient to improve more quickly [10]. Additionally, feeding ROM results back to both therapist and patient could enable shared decisionmaking and shared goals between therapist and patient, which may be beneficial for patient satisfaction and possibly also treatment outcome [42,43].

Potential benefits of ROM are multiple, e.g.: a) improving diagnostics, timely adjustment of treatment and treatment management; b) shared decision making and patient participation; c) better patient outcomes (e.g. symptoms, functioning, quality of life); d) better cost-effectiveness of care; e) better satisfaction with provided care; f) better patient-therapist communication and therapeutic alliance; g) better adherence to clinical directives by therapists; h) scientific research; i) benchmarking (internal/external). There may also be potential obstacles for ROM, e.g.: a) financial burden; b) time burden for patient and therapist; c) implementation challenges; d) applicability in daily clinical practice (i.e., the actual use of ROM feedback by therapists); e) therapist reservations against ROM (e.g. external control, internal benchmarking, confidentiality). In this context, there is also debate on whether ROM is valid for the use of group-level ROM data for external benchmarking [44]. ROM use for benchmarking pertains to health care accountability and cost containment, and thereby growing influence of insurance companies who emphasize more transparency regarding cost-effectiveness and patient satisfaction. When ROM data are used for benchmarking, outcomes of mental health institutions are compared, but this may be invalid because institutions differ considerably from each other (“comparing apples to oranges”; [43]). The discussion on external benchmarking with ROM is further complicated by concerns about the privacy of patient data and the restriction that only pre-post treatment ROM data are used. For more discussion on external benchmarking with ROM, we refer to some relevant publications [44-49].

Effectiveness of ROM in mental health care

As described above, there might be many potential benefits of ROM in mental health care. However, what evidence for these benefits is available so far? In summary, the empirical evidence on the effectiveness of ROM or ROM feedback in mental health treatment of adult psychiatric patients is as follows [1,10,11,39,50-59].

In general, giving clinicians ROM feedback on the progress of their patients may improve certain outcomes of mental health therapy [6,11]. Research showed that ROM feedback to therapists was especially helpful for off-track patients who are at risk for treatment failure (i.e., signal cases). That is to say, ROM feedback made a marked difference for complicated patients who went significantly off-track during treatment (about 20-40% of treated patients). In accordance with the aforementioned theory of Therapeutic Assessment, it was found that giving ROM feedback to both therapist and patient was more effective than to therapist alone [11]. Research also suggests that providing feedback from multiple sources (both wellbeing and affective psychological distress) enhances patient outcomes more, compared to single source feedback ([only wellbeing; [43]). Furthermore, ROM may improve the patient-therapist communication and the process quality of treatment (i.e., presence of treatment plan) and ROM feedback to therapists can lead to fewer treatment sessions [42]. Noteworthy was the finding that giving ROM feedback to both therapist and his/her supervisor (including joint discussion about it) seemed not more effective than giving ROM feedback to therapists alone [6]. Mixed results were found regarding the cost-effectiveness of ROM and the positive impact of ROM on satisfaction of patients with provided care [10,42,60].

Despite these mostly positive results of ROM progress feedback in mental health treatment, there are also some remarks. First, studies regarding the effectiveness of ROM feedback differed quite a bit in terms of overall effect sizes, which ranged from small (0.1) [10] to larger (0.3-0.7) [11,61]. With more severe mental health problems, compared to common or mild problems, effect sizes seem to be reduced [6]. Second, it is hard to draw general conclusions on the effectiveness of ROM feedback, because ROM studies are heterogeneous and methodologically not always optimal (e.g. blinding of assessors, attrition; [10]). Third, ROM feedback may also have adverse effects in very vulnerable and emotionally deregulated patients, especially when they are not yet experiencing any symptomatic relief. Such adverse effects of ROM feedback were found in for instance (partial) inpatients with certain personality disorders [62] and in emergency psychiatry patients [63]. Especially in crisis situations, immediate feedback to psychiatric patients could be counterproductive, because patients are not (yet) able to reflect on their treatment processes, and confronting them repeatedly with their low level of functioning may demoralise them. So, perhaps ROM feedback does not work as well with more disturbed patients in comparison with the less disturbed [63].

Effectiveness of ROM in addiction treatment

What about the specific effectiveness of ROM feedback within substance abuse treatment? On the whole, the above-mentioned effectiveness of ROM feedback in mental health treatment was found across clinical samples and treatment settings [6,11]. That is, although the effectiveness of ROM feedback research largely concerned patients with common psychiatric disorders (depression, anxiety), patients with more complex and chronic psychiatric disorders were also studied (e.g. substance abuse, eating disorders, somatoform disorders, and psychiatric inpatients [6,11,64-67]). The following is a summary of the most relevant studies regarding ROM feedback in a substance abuse treatment context.

One of the first studies evaluating feedback in substance abuse treatment was the study of McCaul and Svikis [68], which examined the impact of providing feedback reports on patient’s attendance data to clinicians in a substance abuse treatment clinic. The results showed improvements in attendance. Forman et al. [69] reported on the feasibility of implementing a web-based feedback system for monitoring patient ratings in outpatient substance abuse treatment clinics. Although the results confirmed the feasibility of the feedback system for therapeutic alliance and treatment satisfaction, further modifications of the system were needed to enhance its potential clinical usefulness (e.g. drug/alcohol use).

Crits-Christoph et al. [70] reported on the results of a randomized trial that examined the efficacy of providing caseload feedback (averaged patient outcomes on therapeutic alliance, treatment satisfaction, drug/alcohol use) to therapists who treat substance abuse. No effects of feedback were found, because therapists were relatively unmotivated by feedback reports that were oriented towards helping them with patients as a group rather than feedback reports for specific patients. Therefore, these results led to a modified version of the feedback system, consisting of weekly feedback to therapists on individual patient outcomes (including functioning, symptoms and treatment satisfaction), with special attention to patients who were not progressing in treatment. The modified feedback system was tested in a subsequent study, which showed that a ROM feedback system adapted to the treatment of substance use problems was promising [64]. In a related study, Crits-Christoph et al. [71] examined predictors of outcome of substance use treatment, in order to inform the development of recovery curves for feedback reports to therapists. The results showed, among other things, that baseline severity of symptoms/functioning, employment, and craving were positively associated with the rate of change in symptoms/functioning, although further research is recommended [71].

Schuman et al. [72] broadened the usefulness of ROM feedback for substance abuse from individual treatment to group treatment. More specifically, they showed the efficacy of client feedback in group psychotherapy with soldiers referred for substance abuse treatment. When patients received feedback about their symptom course, the overall treatment effect was more favourable. This result is important, because many substance use treatment facilities use a group format to provide treatment. Noteworthy was the fact that these feedback effects were present for all patients in the feedback condition, not just those at risk for negative treatment outcome. This seems to be in contrast with ROM feedback in individual treatment, which is mostly effective in at risk patients [11]. Finally, Johnston et al. [73] evaluated the applicability and acceptability of ROM feedback in an addiction medicine setting. Their results generally showed that both therapists and patients agreed that the ROM feedback helped them to identify areas of greatest concern and was useful for treatment planning.

Practical use of ROM in addiction treatment

What about the practical use of ROM feedback in substance abuse treatment by therapists? In this context, particularly the previously mentioned publications of Crits-Christoph et al. [64] and Schuman et al. [72] are inspiring examples. Both publications concern the application of ROM among addicted patients according to an evidence based and fairly easy implementable method. Moreover, these publications include both psychotherapeutic treatment modalities: individual treatment [64] as well as group treatment [72].

In the study of Crits-Christoph et al. [64], patient progress across individual addiction treatment was tracked using the well-established Outcome Questionnaire-45 (OQ-45) [28], adapted to include drug and alcohol use (Modified OQ-45). The original OQ-45, a validated instrument across a broad range of normal and client populations, is a 45-item self-report scale (5-point scale) developed specifically for tracking and assessing patient outcomes in a therapeutic setting. The original OQ-45 has three subscales that measure quality of interpersonal relations, social role functioning, and symptom distress. High scores on the OQ-45 indicate greater levels of symptoms and/or poorer functioning [28,64]. The Modified OQ-45 included two extra items measuring the number of days in the past week that the patients used (a) alcohol and (b) drugs. Crits-Christoph et al. [64] only used the total OQ-45 score, which provides a global assessment of functioning, and the individual alcohol and drug use items. The Modified OQ-45 was administered immediately prior to every treatment session, up to a maximum of 12 treatment sessions. All OQ-45 data were collected electronically by clinic receptionists, research assistants, or therapists. Data were captured electronically and stored in a database. First, immediate OQ-45 feedback reports were viewed electronically, so that increasing or decreasing drug/alcohol use of the patient would be evident to the therapist [64]. Second, when patients were identified as being off-track (not progressing as expected) based upon their OQ-45 scores, therapists were given the opportunity to use clinical support tools (CST) that provided suggestions about ways to improve treatment [64]. Upon being identified as off-track, patients were asked to complete the Assessment for Signal Clients (ASC) [11]; a 40-item self-report measure (5-point scale) designed to assess the severity of problems that may be impeding treatment progress within 4 evidence based domains: problems with therapeutic alliance, motivation, social support, and stressful life events [64]. Therapists were trained to examine the ASC feedback report, noting the domains and the specific items of concern. After the patient completed the ASC, the ASC feedback report is made available to the therapist. This web-based ASC feedback report contains a link to a list of interventions that target each problem domain identified by the ASC [64]. Some specific examples of treatment interventions for the “motivation domain” are the following: assess how important it is for the client to make target changes, having him or her rate the desire to make specific changes on a scale of 1-10; discuss the client's important values, goals, and aspirations, and ask open-ended questions to see if the client is ready to make a commitment to change [64]. If it appeared that difficulty within the four domains did not adequately address the patient's treatment failure (i.e., going off-track), therapists were trained to reassess the patient's diagnostic formulation and to consider whether an alternate kind of treatment could be more effective and/or whether a referral for a medication consultation is an appropriate avenue to pursue [11,64]. In conclusion, the study of Crits-Christoph et al. [64] showed that patient progress feedback to therapists appeared to successfully bring the offtrack patients back on-track. A next step in this regard may be to also inform patients about their treatment progress via their therapist, which could facilitate the mutual communication and shared decision making about treatment.

In the study of Schuman et al. [72], patient progress across group addiction treatment was tracked using another well-known and validated instrument, the Outcome Rating Scale (ORS; [27]). This is a self-report instrument designed to measure patient progress repeatedly throughout the course of therapy. Adapted from the OQ [28], the ORS assesses four dimensions: (i) Individual-personal or symptomatic distress or well-being. (ii) Interpersonal-relational distress or how well the patient is getting along in intimate relationships. (iii) Social-the patient’s view of satisfaction with work/school and relationships outside of the home. (iv) Overall-general sense of well-being [27]. The ORS translates these four dimensions into a visual analogue format of four 10 cm lines, with instructions to place a mark on each line, with low estimates to the left and high to the right. The four 10-cm lines add to a total score of 40. The score is the summation of the marks made by the patient to the nearest millimetre (mm) on each of the four lines, measured by a centimeter (cm) ruler or template. A software program was used to collect data and provide the basis for feedback to the therapists [27,72]. The program used algorithms derived from previous ORS research and normative samples to predict the expected treatment response (ETR) for individual patients entering therapy with the same intake score. ORS scores from subsequent sessions are then compared against the ETR, which allows clinicians to identify patients who are making progress as expected as well as those at risk for premature termination or a negative outcome [27,72]. Therapists were provided with a graph that used four different colours (Green/Blue/Yellow/Red) to indicate one of four different stages of treatment progress corresponding to the patient’s ORS score compared to the ETR: “Patient’s functioning is in the normal range. Consider successful termination.” (Green); “Patient’s functioning is in the adequate range. No change in the treatment plan is recommended.” (Blue); “Patient’s functioning is less than adequate. Consider altering the treatment plan by intensifying treatment, shifting intervention strategies, and monitoring progress especially carefully.” (Yellow); and “Patient is not making the expected level of progress. Steps should be taken to carefully review this case and decide upon a new course of action such as referral or a higher level of care. The patient’s readiness for change may need to be re-assessed.” (Red) [27,72]. Five sessions of group treatment (10 patients per group) were offered, because it served the primary function to identify at-risk clients. If patients continued to exhibit concerns that warranted further intervention, additional group sessions were offered [72]. Before each group session, patients completed the computerized version of the ORS in a private room. Once patients placed their marks on the appropriate lines, they would click the “done” button and proceed to the group therapy room [72]. The program automatically scored the instrument. Patients did not see the results generated by the computer program. Any results they may have seen would have been revealed by their therapist if he or she chose to share the results. At the conclusion of each group session, therapists were given progress graphs of patients [72]. In sum, Schuman et al. [72] showed that the benefits of patient feedback in individual psychotherapy appear to extend to group psychotherapy. However, they also point to a downside of the ORS: although psychometrically acceptable, the ORS does not directly assess substance use which was a primary focus of their group treatment [73].

Clinical and research recommendations for use of ROM in treatment

As with any chronic disease, the objectives for successful long-term management of substance use disorders feature regular screening, additional care as needed, and especially long-term monitoring [60]. Long-term monitoring implies the monitoring of treatment progress of patients both during and after treatment, which is particularly relevant for chronic mental diseases such as addiction. Therefore, several studies in the context of ROM and addiction were focused on the feasibility and validity of low-budget telephonic follow-up interviews in routine outcome monitoring of substance abuse treatment [74-77]. It appears that telephone case monitoring is a cost-effective strategy that increased the days of abstinence after treatment [60]. However, in this context, research also showed that tracking problems make it difficult to interview substance abuse patients at the intended followup time [76].

An important concern generally remains the implementation and the integration of ROM in clinical practice [26,78-80]. ROM feedback can be considered as a communication tool, and research showed that ROM feedback discussions have a positive effect on treatment cooperation [81]. However, research showed that ROM feedback reports were often not used by therapists [11,66,82]. So, more attention may be needed for recurrent training of therapists on the use of ROM as an opportunity to discuss treatment progress, stagnation, or decline with the patient and to set common treatment goals together [4,79,83]. Moreover, ROM results could be integrated in the treatment plan of a patient [25]. This integration, however, is far from evident which was demonstrated in a recent study of Tasma et al. [2]. This study investigated whether, with ROM identified (mental) health problems (i.e., symptoms, psychosocial problems, cardiovascular risk factors), were reflected in the treatment plans (Electronic Patient Files) of psychotic patients. They found a substantial discrepancy between the ROM measurements and the treatment plan: low rates of detection of problems in the treatment plans, even though these problems were identified with ROM and the other way around was also the case. Thus, ROM and clinical practice appear to be two separate processes, whereas ideally they should be integrated. Strong efforts should be made to integrate ROM and consequent treatment activities. Such integration may help to provide patients with adequate and customized care and simultaneously minimize under- and over-treatment [2].

A major limitation of many feedback researches so far is the absence of information on the specific types of clinical actions that were taken by clinicians in response to feedback reports about their client. Future research will need to address this topic, because it relates to the possible mechanisms of feedback effects [64]. Additionally, it may be of interest to examine the effects of feedback within subgroups defined by certain clinically relevant baseline variables. It may be, for example, that feedback has little impact among patients with long term substance use and comorbidity and more impact on less complex patients. So, further research regarding possible moderators or mediators of feedback effects in (addiction) treatment is required [71].

It is further recommended to supplement ROM treatment progress feedback with earlier mentioned Clinical Support Tools (CST). CST refers to an instrument assessment for signal cases, with detailed information to therapists about alarming scores of signal cases on specific subscales or items (e.g. therapeutic alliance, social support, readiness to change, diagnostic formulation, life events, need for medication referral, suicidality [11]). Research showed that, when feedback with CST is used for off-track patients, outcome improvement is greater than feedback without CST [11]. Future research is necessary to develop and compare different ROM clinical support tools. Such studies may also investigate what might be the optimal alert system for therapists regarding clinical deterioration of patients, thereby increasing therapeutic impact and efficiency [6].

Some final suggestions for future research on ROM are the following [6,10,11,59]. First, research is required regarding the positive impact of ROM on a broad range of all relevant patients’ outcomes (e.g. symptoms, social functioning, quality of life). In addition, further research is needed concerning the positive impact of ROM on costs, treatment management (e.g. number and frequency of treatment sessions) and shared decision making, because these aspects are still insufficiently investigated to draw final conclusions. Second, possible differential effectiveness of ROM in different treatments (like psychotherapy versus medication) has not been investigated sufficiently [10]. Third, most studies of ROM feedback in mental health treatment showed its short term effectiveness, and less researched are the long-time effects of ROM. Therefore, more research regarding the beneficial effect of ROM feedback in the longer term [beyond six months; 10] is recommended. Fourth, additional research is needed on possible harmful effects of ROM feedback in very vulnerable psychiatric patients. Fifth, further research is necessary on the effectiveness of ROM feedback in group psychotherapy, since studies in this context showed inconsistent results for different patient groups [71,84].

Conclusion

The use of Routine Outcome Monitoring (ROM) as a quality tool for measuring outcome of treatment in mental healthcare is evidencebased. In general, promising results were found regarding the effectiveness of ROM feedback in the treatment of both common mental disorders and more chronic mental disorders like substance abuse. Despite the large amount of research on ROM feedback so far, better quality trials are needed that are more decisive regarding when and for whom ROM feedback is effective in both individual and group treatment. This is particularly the case with addictive disorders in which ROM feedback is relatively less investigated. It is also recommended to examine how therapists actually use the ROM feedback for the benefit of their clinical decisions and in their communication with the patient. Only in this way, ROM can be an integral and valuable part of routine practice.

References

1. Carlier IV, Meuldijk D, Van Vliet IM, Van Fenema E, Van der Wee NJ, et al. (2012) Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. J Eval Clin Pract 18: 104-110.
2. Tasma M, Swart M, Wolters G, Liemburg E, Bruggeman R, et al. (2016) Do routine outcome monitoring results translate to clinical practice? A cross-sectional study in patients with a psychotic disorder. BMC Psychiatry 16: 107.
3. Roe D, Drake RE, Slade M (2015) Routine outcome monitoring: An international endeavour. Int Rev Psychiatry 27: 257-260.
4. Boswell JF, Constantino MJ, Kraus DR, Bugatti M, Oswald JM (2016) The expanding relevance of routinely collected outcome data for mental health care decision making. Adm Policy Ment Health 43: 482-491.
5. Schiepek G, Aichhorn W, Gruber M, Strunk G, Bachler E, et al. (2016) Real-time monitoring of psychotherapeutic processes: Concept and compliance. Front Psychol 7: 604.
6. Davidson KM, Rankin ML, Begley A, Lloyd S, Barry SJ, et al. (2017) Assessing patient progress in psychological therapy through feedback in supervision: The MeMOS* randomized controlled trial (*Measuring and monitoring clinical outcomes in supervision: MeMOS). Behav Cogn Psychother 45: 209-224.
7. de Beurs E, den Hollander-Gijsman ME, van Rood YR, Van der Wee NJ, Giltay EJ, et al. (2011) Routine outcome monitoring in the Netherlands: Practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clin Psychol Psychother 18: 1-12.
8. Carlier IV, Colijn S, van Rood YR, Streevelaar MF, Van Vliet IM, et al. (2014) A comparative analysis of personality pathology profiles among patients with pure depressive, pure anxiety and pure somatoform disorders. J Affect Disord 168: 322-330.
9. Conijn JM, Emons WH, Page BF, Sijtsma K, Van der Does W, et al. (2016) Response inconsistency of patient-reported symptoms as a predictor of discrepancy between patient and clinician-reported depression severity. Assessment: 1073191116666949.
10. Kendrick T, El-Gohary M, Stuart B, Gilbody S, Churchill R, et al. (2016) Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults. Cochrane Database Syst Rev 7: CD011119.
11. Lambert MJ (2017) Maximizing psychotherapy outcome beyond evidence-based medicine. Psychother Psychosom 86: 80-89.
12. Hasin DS, Stinson FS, Ogburn E, Grant BF (2007) Prevalence, correlates, disability and comorbidity of DSM-IV alcohol abuse and dependence in the United States: Results from the national epidemiologic survey on alcohol and related conditions. Arch Gen Psychiatry 64: 830-842.
13. Compton WM, Thomas YF, Stinson FS, Grant BF (2007) Prevalence, correlates, disability and comorbidity of DSM-IV drug abuse and dependence in the United States: Results from the national epidemiologic survey on alcohol and related conditions. Arch Gen Psychiatry 64: 566-576.
14. Keeley JW, Chmielewski MS, Bagby RM (2015) Interaction effects in comorbid psychopathology. Compr Psychiatry 60: 35-39.
15. Delgadillo J, Böhnke JR, Hughes E, Gilbody S (2016) Disentangling psychopathology, substance use and dependence: A factor analysis. BMC Psychiatry 16: 281.
16. Hodgson K, Almasy L, Knowles EE, Kent JW Jr, Curran JE, et al. (2017) The genetic basis of the comorbidity between cannabis use and major depression. Addiction 112: 113-123.
17. Urbanoski K, Kenaszchuk C, Veldhuizen S, Rush B (2015) The clustering of psychopathology among adults seeking treatment for alcohol and drug addiction. J Subst Abuse Treat 49: 21-26.
18. Odlaug BL, Gual A, DeCourcy J, Perry R, Pike J, et al. (2016) Alcohol dependence, co-occurring conditions and attributable burden. Alcohol Alcohol 51: 201-209.
19. Fleury MJ, Grenier G, Bamvita JM, Perreault M, Caron J (2014) Predictors of alcohol and drug dependence. Can J Psychiatry 59: 203-212.
20. Morley K, Baillie A, Leung S, Sannibale C, Teesson M, et al. (2016) Is specialized integrated treatment for comorbid anxiety, depression and alcohol dependence better than treatment as usual in a public hospital setting? Alcohol Alcohol 51: 402-409.
21. Johansen AB, Tavakoli S, Bjelland I, Lumley M (2017) Constructivist simultaneous treatment of borderline personality disorder, trauma, and addiction comorbidity: A qualitative case study. Qual Health Res 27: 236-248.
22. Milosevic I, Chudzik SM, Boyd S, McCabe RE (2017) Evaluation of an integrated group cognitive-behavioral treatment for comorbid mood, anxiety and substance use disorders: A pilot study. J Anxiety Disord 46: 85-100.
23. Systematic reviews (2009) Centre for review and dissemination.
24. Higgins JP, Green S (2011) Cochrane handbook for systematic reviews of interventions. John Wiley & Sons.
25. Siebum W, Pijl YJ, Wolf GS (2015) Routine outcome monitoring and process quality in mental health care: A descriptive study in daily practice. J Eval Clin Pract 21: 620-625.
26. Wampold BE (2015) Routine outcome monitoring: Coming of age-With the usual developmental challenges. Psychotherapy (Chic) 52: 458-462.
27. Miller SD, Duncan BL, Sorrell R, Brown GS (2005) The partners for change outcome management system. J Clin Psychol 61: 199-208.
28. Lambert MJ (2015) Progress feedback and the OQ-system: The past and the future. Psychotherapy (Chic) 52: 381-390.
29. Carlier I, Schulte-Van Maaren Y, Wardenaar K, Giltay E, Van Noorden M, et al. (2012) Development and validation of the 48-item Symptom Questionnaire (SQ-48) in patients with depressive, anxiety and somatoform disorders. Psychiatry Res 200: 904-910.
30. Carlier IV, Kovács V, van Noorden MS, van der Feltz-Cornelis C, Mooij N, et al. (2017) Evaluating the responsiveness to therapeutic change with routine outcome monitoring: A comparison of the symptom questionnaire-48 (SQ-48) with the brief symptom inventory (BSI) and the outcome questionnaire-45 (OQ-45). Clin Psychol Psychother 24: 61-71.
31. Derogatis LR, Melisaratos N (1983) The brief symptom inventory: An introductory report. Psychol Med 13: 595-605.
32. Derogatis LR, Cleary PA (1977) Confirmation of the dimensional structure of the SCL-90: A study in construct validation. J Clin Psychol 33: 981-989.
33. Clark LA, Watson D (1991) Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. J Abnorm Psychol 100: 316-336.
34. Drapeau M (2012) 10 tools for progress monitoring in psychotherapy. Integr Sci Pract 2: 5-43.
35. Lyon AR, Lewis CC, Boyd MR, Hendrix E, Liu F (2016) Capabilities and characteristics of digital measurement feedback systems: Results from a comprehensive review. Adm Policy Ment Health 43: 441-466.
36. Kluger AN, DeNisi A (1996) The effects of feedback interventions on performance: A historical review, a meta-analysis and a preliminary feedback intervention theory. Psychol Bull 119: 254-284.
37. Riemer M, Rosof-Williams J, Bickman L (2005) Theories related to changing clinician practice. Child Adolesc Psychiatr Clin N Am 14: 241-254.
38. Bickman L, Douglas SR, De Andrade ARV, Tomlinson M, Gleacher A, et al. (2016) Implementing a measurement feedback system: A tale of two sites. Adm Policy Ment Health 43: 410-425.
39. Gondek D, Edbrooke-Childs J, Fink E, Deighton J, Wolpert M (2016) Feedback from outcome measures and treatment effectiveness, treatment efficiency and collaborative practice: A systematic review. Adm Policy Ment Health 43: 325-343.
40. Finn SE, Tonsager ME (1992) Therapeutic effects of providing MMPI-2 test feedback to college students awaiting therapy. Psychol Assess 4: 278-287.
41. Poston JM, Hanson WE (2010) Meta-analysis of psychological assessment as a therapeutic intervention. Psychol Assess 22: 203-212.
42. Janse PD, De Jong K, Van Dijk MK, Hutschemaekers GJ, Verbraak MJ (2016) Improving the efficiency of cognitive-behavioural therapy by using formal client feedback. Psychother Res, pp: 1-14.
43. Dyer K, Hooke GR, Page AC (2016) Effects of providing domain specific progress monitoring and feedback to therapists and patients on outcome. Psychother Res 26: 297-306.
44. Hoenders RH, Bos EH, Bartels-Velthuis AA, Vollbehr NK, Van der Ploeg K, et al. (2014) Pitfalls in the assessment, analysis, and interpretation of routine outcome monitoring (ROM) data: Results from an outpatient clinic for integrative mental health. Adm Policy Ment Health 41: 647-659.
45. Klazinga N, Fischer C, Asbroek A (2011) Health services research related to performance indicators and benchmarking in Europe. J Health Serv Res Policy 16: 38-47.
46. Delgadillo J, McMillan D, Leach C, Lucock M, Gilbody S, et al. (2014) Benchmarking routine psychological services: A discussion of challenges and methods. Behav Cogn Psychother 42: 16-30.
47. McKay R, Coombs T, and Duerden D (2014) The art and science of using routine outcome measurement in mental health benchmarking. Australas Psychiatry 22: 13-18.
48. Delespaul PA (2015) Routine outcome measurement in the Netherlands - A focus on benchmarking. Int Rev Psychiatry 27: 320-328.
49. Beurs E, Barendregt M, Heer A, Duijn E, Goeree B, et al. (2016) Comparing methods to denote treatment outcome in clinical research and benchmarking mental health care. Clin Psychol Psychother 23: 308-318.
50. Lambert MJ, Harmon C, Slade K, Whipple JL, Hawkins EJ (2005) Providing feedback to psychotherapists on their patients' progress: Clinical results and practice suggestions. J Clin Psychol 61: 165-174.
51. Marshall M, Lockwood A, Green G, Zajac-Roles G, Roberts C, et al. (2004) Systematic assessments of need and care planning in severe mental illness. Br J Psychiatry 185: 163-168.
52. Knaup C, Koesters M, Schoefer D, Becker T, Puschner B (2009) Effect of feedback of treatment outcome in specialist mental healthcare: Meta-analysis. Br J Psychiatry 195: 15-22.
53. Shimokawa K, Lambert MJ, Smart DW (2010) Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. J Consult Clin Psychol 78: 298-311.
54. Lambert MJ, Cattani K (2012) Practice-friendly research review: Collaboration in routine care. J Clin Psychol 68: 209-220.
55. Shaw EJ, Sutcliffe D, Lacey T, Stokes T (2013) Assessing depression severity using the UK quality and outcomes framework depression indicators: A systematic review. Br J Gen Pract 63: e309-e317.
56. Boswell JF, Kraus DR, Miller SD, Lambert MJ (2015) Implementing routine outcome monitoring in clinical practice: Benefits, challenges and solutions. Psychother Res 25: 6-19.
57. Gibbons MBC, Kurtz JE, Thompson DL, Mack RA, Lee JK, et al. (2015) The effectiveness of clinician feedback in the treatment of depression in the community mental health system. J Consult Clin Psychol 83: 748-759.
58. Krägeloh CU, Czuba KJ, Billington DR, Kersten P, Siegert RJ (2015) Using feedback from patient-reported outcome measures in mental health services: A scoping study and typology. Psychiatr Serv 66: 224-241.
59. Kendrick T, Stuart B, Leydon GM, Geraghty AW, Yao L, et al. (2017) Patient-reported outcome measures for monitoring primary care patients with depression: PROMDEP feasibility randomised trial. BMJ Open 7: e015266.
60. McCollister K, Yang X, McKay JR (2016) Cost-effectiveness analysis of a continuing care intervention for cocaine-dependent adults. Drug Alcohol Depend 158: 38-44.
61. Whipple JL, Lambert MJ, Vermeersch DA, Smart DW, Nielsen SL, et al. (2003) Improving the effects of psychotherapy: The use of early identification of treatment and problem-solving strategies in routine practice. J Counsel Psychol 50: 59-68.
62. De Jong K SJ, Ingenhoven T, Van Busschbach J, Timman R (2017) Adverse effects of outcome monitoring feedback in patients with personality disorders: A randomized controlled trial in day treatment and in patient settings. J Pers Disord 8: 1-21.
63. van Oenen FJ, Schipper S, Van R, Schoevers R, Visch I, et al. (2016) Feedback-informed treatment in emergency psychiatry; a randomised controlled trial. BMC Psychiatry 16: 110.
64. Crits-Christoph P, Ring-Kurtz S, Hamilton JL, Lambert MJ, Gallop R, et al. (2012) A preliminary study of the effects of individual patient-level feedback in outpatient substance abuse treatment programs. J Subst Abuse Treat 42: 301-309.
65. Probst T, Lambert MJ, Loew TH, Dahlbender RW, Göllner R, et al. (2013) Feedback on patient progress and clinical support tools for therapists: Improved outcome for patients at risk of treatment failure in psychosomatic in-patient therapy under the conditions of routine practice. J Psychosom Res 75: 255-261.
66. Simon W, Lambert MJ, Harris MW, Busath G, Vazquez A (2012) Providing patient progress information and clinical support tools to therapists: Effects on patients at risk of treatment failure. Psychother Res 22: 638-647.
67. Simon W, Lambert MJ, Busath G, Vazquez A, Berkeljon A, et al. (2013) Effects of providing patient progress feedback and clinical support tools to psychotherapists in an inpatient eating disorders treatment program: A randomized controlled study. Psychother Res 23: 287-300.
68. McCaul ME, Svikis DS (1991) Improving client compliance in outpatient treatment: Counselor-targeted interventions. NIDA research monograph 106: 204-215.
69. Forman R, Crits-Christoph P, Kaynak ÃÂ?, Worley M, Hantula DA, et al. (2007) A feasibility study of a web-based performance improvement system for substance abuse treatment providers. J Subst Abuse Treat 33: 363-371.
70. Crits-Christoph P, Ring-Kurtz S, McClure B, Temes C, Kulaga A, et al. (2010) A randomized controlled study of a web-based performance improvement system for substance abuse treatment providers. J Subst Abuse Treat 38: 251-262.
71. Crits-Christoph P, Markell HM, Gallop R, Gibbons MB, McClure B, et al. (2015) Predicting outcome of substance abuse treatment in a feedback study: Can recovery curves be improved upon? Psychother Res 25: 694-704.
72. Schuman DL, Slone NC, Reese RJ, Duncan B (2015) Efficacy of client feedback in group psychotherapy with soldiers referred for substance abuse treatment. Psychother Res 25: 396-407.
73. Johnston KL, Lawrence SM, Dodds NE, Yu L, Daley DC, et al. (2016) Evaluating PROMIS® instruments and methods for patient-centered outcomes research: Patient and provider voices in a substance use treatment setting. Qual Life Res 25: 615-624.
74. McLellan AT, McKay JR, Forman R, Cacciola J, Kemp J (2005) Reconsidering the evaluation of addiction treatment: From retrospective follow-up to concurrent recovery monitoring. Addiction 100: 447-458.
75. Hawkins EJ, Baer JS, Kivlahan DR (2008) Concurrent monitoring of psychological distress and satisfaction measures as predictors of addiction treatment retention. J Subst Abuse Treat 35: 207-216.
76. Oudejans SC, Schippers GM, Merkx MJ, Schramade MH, Koeter MW, et al. (2009) Feasibility and validity of low-budget telephonic follow-up interviews in routine outcome monitoring of substance abuse treatment. Addiction 104: 1138-1146.
77. Lennox RD, Sternquist MA, Paredes A (2013) A simplified method for routine outcome monitoring after drug abuse treatment. Subst Abuse 7: 155-169.
78. Lucock M, Halstead J, Leach C, Barkham M, Tucker S, et al. (2015) A mixed-method investigation of patient monitoring and enhanced feedback in routine practice: Barriers and facilitators. Psychother Res 25: 633-646.
79. Ross DF, Ionita G, Stirman SW (2016) System-wide implementation of routine outcome monitoring and measurement feedback system in a national network of operational stress injury clinics. Adm Policy Ment Health 43: 927-944.
80. Moltu C, Veseth M, Stefansen J, Nøtnes JC, Skjølberg ÃÂ?, et al. (2016) This is what I need a clinical feedback system to do for me: A qualitative inquiry into therapistsâÂ?Â? and patientsâÂ?Â? perspectives. Psychother Res: 1-14.
81. van der Veeken FC, Lucieer J, Bogaerts S (2016) Routine outcome monitoring and clinical decision-making in forensic psychiatry based on the Instrument for Forensic Treatment Evaluation. PLoS ONE 11: e0160787.
82. de Jong K, van Sluis P, Nugter MA, Heiser WJ, Spinhoven P (2012) Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychother Res 22: 464-474.
83. Santana MJ, Haverman L, Absolom K, Takeuchi E, Feeny D, et al. (2015) Training clinicians in how to use patient-reported outcome measures in routine clinical practice. Qual Life Res 24: 1707-1718.
84. Davidsen AH, Poulsen S, Lindschou J, Winkel P, Tróndarson MF, et al. (2017) Feedback in group psychotherapy for eating disorders: A randomized clinical trial. J Consult Clin Psychol 85: 484-494.