Evaluation of a Mental Health Research Database for Stakeholders

David Nicholas^1* Angus H Thompson² Orrin Lyseng³ Tom Shand^3,4 Carmela Hutchinson^3,5 Andrew Carr⁵ Sharon Springer⁶ Sheba Skariah¹ Dino Skariah¹

¹Faculty of Social Work, University of Calgary, Edmonton, AB, Canada

²Institute of Health Economics, Edmonton, AB, Canada

³Alberta Alliance on Mental Illness and Mental Health, Edmonton, AB, Canada

⁴Canadian Mental Health Association, Edmonton, AB, Canada

⁵Alberta Network for Mental Health, Irricana, AB, Canada

⁶University of Alberta, Edmonton, AB, Canada

*Corresponding Author:

David Nicholas
#444, 11044 82 Avenue, Edmonton, Alberta, T6G 0T2, CANADA
Tel: 780-492-8094
Fax: 780-492-5774
E-mail:nicholas@ucalgary.ca

Received date: November 20, 2013; Accepted date: December 20, 2013; Published date: December 23, 2013

Citation: Nicholas D, Thompson AH, Lyseng O, Shand T, Sutherland S, et al. (2013) Evaluation of a Mental Health Research Database for Stakeholders. J Community Med Health Educ 3: 257. doi: 10.4172/2161-0711.1000257

Copyright: © 2013 Nicholas D, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Community Medicine & Health Education

Abstract

The Database Project website is a systematized repository of peer-reviewed findings in the field of mental illness, currently focusing on schizophrenia. It offers accessible summaries of research findings largely for non-profit groups in the mental health sector. It is intended to assist these organizations in ascertaining trends and providing data to support grant preparation and other project development. This study evaluates the content and usability of the Database. The strengths and drawbacks of the Database are explored along with the Database’s utility for stakeholders. The Database was reported to have an easily accessible and helpful selection of content. Reviewers concluded that the Database is of substantial value in terms of content and usability, offering accessible knowledge of peer-reviewed research to users. This ultimately serves in building capacity at a community and organizational level. Recommendations and implications for practice and resource development are discussed.

Keywords

Mental health; Schizophrenia; Epidemiology; Database; Knowledge translation

Introduction

The Database Project (the Database) is a newly developed repository of studies addressing mental illness outcomes. Located at http://aamimh.ca/Research.aspx, it indexes key study information for community and lay users, including methodological design and outcomes. To date, the Database includes epidemiological studies in schizophrenia. This initiative was recently developed by the Alberta Alliance for Mental Illness and Mental Health (AAMIMH), and its development has been guided by the Executive Director of AAMIMH and an advisory committee of interdisciplinary mental health service, advocacy and research team members, including individuals with extensive professional and in some cases, family experience related to mental health conditions. This paper reports on a recent evaluation of the Database, focusing on its content and navigational capabilities.

Background

Needs associated with mental illness invite the translation of new knowledge for evidence based community and clinical practice [1]. As a burgeoning field in the health sciences, knowledge translation comprises, “a dynamic and iterative process that includes synthesis, dissemination, exchange and ethically-sound application of knowledge to improve the health of Canadians, provide more effective health services and products and strengthen the health care system” [2]. The transfer of knowledge in mental health settings is fraught with potential barriers and impediments to sharing research results with knowledge users such as decision makers, clinicians and families. For instance, critical questions emerge such as how can community-based mental health and advocacy organizations gain cost-neutral, timely and comprehensive access to a continually expanding and broad base of research literature. Knowledge translation priorities are at the heart of current priorities in rendering knowledge accessible in ultimately working toward best practice.

Common models of knowledge transfer have traditionally implemented a top down approach whereby emerging research is transferred uni-directionally from the academic and clinician to the community. A key weakness of this approach lies in its temporal delay and inherent failure to welcome the end user ‘as equal’ in knowledge access and research engagement [3]. It further ignores the fact that knowledge is situated and political; that individuals, including stakeholders, will actively interpret received information within their own set of values, experiences and organizational or other constraints that ultimately shape interpretation. Finding engaging models for lateral knowledge reach in a timely manner to all relevant knowledge users is a critical aim and priority.

To that end, Broner et al. [3] posit a consensus building model as a more inclusive approach that takes into account the diversity of the mental health system and considers the role of knowledge users in transferring and mobilizing knowledge toward the development of best practice. The model embraces the integration of multiple holders of knowledge that come together to create a new understanding and/ or that build a program or intervention. This model empowers the community to engage in knowledge exchange and mobilization [3].

The value of engaging communities in knowledge access and transfer is highlighted by McGrath et al. [4]. They suggest that integrated knowledge transfer, where researchers, stakeholders and users are mutually engaged in the research process, offers advantages of facilitating uptake to clinical practice. The Canadian Senate Report on the Status of Mental Health states that without effective knowledge translation within the mental health system, ineffective and possibly harmful treatments could continue and, conversely, evidence-based interventions may not be used [5]. Additionally, the Mental Health Commission of Canada created the Knowledge Exchange Centre with the goal of improving the lives of Canadians with mental illness by encouraging citizens to share knowledge and information and collaborate with innovative ideas [6].

Boydell et al. [7] conducted focus groups exploring knowledge translation in rural and remote regions of Canada. They found that strategies to improve uptake comprised two-way exchanges of knowledge between researchers and service providers, and an alignment of resources to create change and foster the sustainability of that change. Accessing knowledge necessary to guide practice, policy and program development, emerges as central in ensuring that capacity is built and ultimately best practices can be truly sought. Ironically, less than 10% of researchers’ time is devoted to sharing research results beyond their academic community [8]. This highlights a startling gap in knowledge access for community groups that are outside of the postsecondary academy. Much work is needed in bridging and mobilizing knowledge such that health outcome benefits are more broadly realized [8].

In redressing these gaps within the mental health community, the Database project was implemented. As a follow-up to this initiative, a simultaneous evaluation was conducted. Areas of focus in this evaluation comprised an examination of database content, navigation and user friendliness for relevant stakeholders (including administrators, clinicians, service recipients, and families, with a focus on use by NGO and advocacy groups). Guiding research questions for this evaluation were as follows:

1. What do stakeholders perceive as the benefits and drawbacks of the Database?

2. How would stakeholders reportedly use the Database?

Methods

The Database provides a summary of epidemiology-based studies in the peer-reviewed literature examining schizophrenia (Figure 1). To date, 76 articles are available within the Database, with an initial focus on schizophrenia and epidemiology in largely Canadian-based studies. These articles were independently reviewed and entered into the Database by a graduate student. Information collected for each study included subject, author, topic, source and date of publication.

Figure 1: screenshot of the Database homepage.

The graduate student was hired following receipt of a federal Student Temporary Employment Program (STEP) grant plus AAMIMH funds to defray the costs of employing a research candidate for website development. The doctoral student brought extensive experience and expertise in this substantive area. She reviewed, coded and inputted summaries of peer-reviewed studies addressing epidemiology and schizophrenia.

The Database was designed by a database developer working in the mental health sector. His time and expertise were donated by the Alberta Network for Mental Health, a member organization of the AAMIMH. Aims of the Database included offering support to the mental health and not-for-profit sector through accessible outcome data. It was anticipated that this resource would assist in the development of literature reviews for grant applications, policy statements and other purposes that reflect a need for data capture, analysis approach, and presentation of findings.

Evaluation of the Database comprised the assessment of site content and navigation. Upon evaluation training and orientation to the Database, a team of five reviewers (one methodology-based university professor and four research graduate students) independently reviewed the site and applied specific data searches and navigational tests to assess usability. These searches included hypothetical, simulated topics that an administrator or clinician would reasonably initiate. Upon completion of the independent reviews by all evaluation team members, intra-team assessment reports were submitted, discussion was facilitated, and consensus among reviewers was achieved.

Rigor of assessment findings was verified through several wellestablished criteria for trustworthiness in qualitative analysis [9]. This included the achievement of inter-rater reliability, demonstrated by consensus agreement following blinded reviews, and peer debriefing among key experts through careful review among mental health advocates, service providers and researchers.

Results

Findings address the substantive content of the Database as well as processes for navigation. In terms of content, the majority of cited research in the Database was published after 1991. Of the total of 76 studies, substantive content addresses the following areas related to co-morbid or associated concerns: Substance Use/Abuse (27 articles), Major Depression/Depression (18 articles), Age (16 articles), Suicide (15), Mental Disorders (not specified)/Mental Illness (13 articles), Psychosis (12 articles), Bipolar (affective) disorder (10 articles), and Alcohol abuse (8 articles).

The Background page of the Database indicates that the reviewed research within the Database is largely geared for a regional Canadian context with just a few exceptions of non-Canadian based seminal studies. The national focus of the Database and its substantive focus (to date, schizophrenia) offers greater focus and depth for the resources and agencies within its catchment area.

On the other hand, reviewers consistently recommended expansion of the scope, range and focus of the Database, both in quantity and substantive foci of new studies being entered. In terms of quantity, increasing the volume of research and including a wider range of geographic locales and relevant topics in mental health, were recommended. Careful attention to the needs of Database users was also suggested in future planning, including potentially more options for targeted yet user-friendly searches. As an example, more versatile search engine capacity was recommended, with sophisticated search abilities akin to other well-known platforms (e.g., EBSCOhost, SAGE). However, it is important to contextualize such recommendations for heightened functionality within the financial requisite of additional funding for such increased searching capacity.

Reviewers highlighted the need for increased attention to website branding and potential Database advertisement more broadly. Heightened awareness of the Database was thought to likely result in greater utility by the mental health community. To that end, it was thought that access to the Database through a password, as is currently required, may not be necessary and actually could restrict legitimate access, hence should be relaxed.

Database usability

Participants generally suggested that the Database could be navigated with relative ease by a novice user. To further enhance navigation, enhanced clarity of the aims of the Database was recommended. Clearer descriptions of expected benefits to users potentially could be communicated on the site. Reviewers suggested that this page could showcase Database components such as Background Information, Vision, Mission Statement, and Resource Information. Of note, since the initial review of the Database the Introduction and Background Information sections have been updated and improved.

Adding separate headings for Vision and Mission Statement was expected to enhance clarity, definition and context for users. Such additional clarity was determined to improve access for information visibility.

As part of the About Us page, reviewers suggested that Database users should be informed about how and how often the Database was being reviewed and updated. A short summary of updated information as well as the number and nature of studies reviewed in the Database, were thought to potentially offer user benefit in clarifying and publicizing the Database.

Content of the database

Article information summarized within each citation in the Database, was perceived to reflect a high level of accuracy relative to the originating primary study. Of the 76 articles that indicated geographic location of data collection or the origin of the researcher, 47 were based in Canada. This speaks to a relatively active role in Canada in this substantive area, and a predominant focus of Canadian scholarship within the Database.

It was suggested by reviewers that clarification was needed to inform users that the current focus of reviewed articles in the Database was on schizophrenia, although this substantive focus is expected to expand over time. One reviewer suggested an introductory statement such as the following: “Currently, the Database contains 76 articles, of which X articles primarily deal with the epidemiology of schizophrenia. Y are peer reviewed articles, and Z have been published within the last 10 years.” As more articles and substantive foci are added, this topical breakdown and the corresponding numbers could be updated. Reviewers noted and applauded the use of a scholarly and inter-professional advisory committee in overseeing the project. They recommended that the names and affiliations of committee members be published on the site, as an indication to users of quality assurance and site credibility. In addition, examples of ways to use the data and potential benefits of the site were recommended, within a creativelyand well-written narrative section on the Database.

Navigating beyond the About Us page currently takes users to the search screen to begin a search. To that end, an enhanced breakdown of subject type and research methodology was thought to potentially better assist the user in the search process. An outline of possible search terms, focusing on schizophrenia as an example, was suggested as follows:

1. Schizophrenia and substance use/abuse

2. Schizophrenia and depression

3. Schizophrenia and suicide

4. Schizophrenia and mental illness, and

5. Schizophrenia and psychosocial practices.

Moving beyond the Log In and the About Us page, reviewers noted that site users are presented with a screen that allows search options. Currently, the first tab offers a “Quick Search” that scans the entire Database to match the search term anywhere in the abstract. A more advanced search would permit narrowing the search term according to the fields the user has selected. The user then can select “option” boxes or “click on” options from a drop-down menu (e.g., “Peer-Reviewed only” or “Full-Text” articles). The “Advanced Search” could further facilitate advanced searching by providing additional options such as, “All” articles, “Published Works”, and “Scholars” from the list of selected records.

As exemplified here, increased flexibility is advised in optimally facilitating a focused search with navigational ease. This may warrant additional options such as “Subject area”, “Date”, “Author”, and “Type of resource”. Reviewers noted the potential benefit for users if fields were to offer the option of being separated by “AND” and “OR” commands, and the “NOT” command could further assist in narrowing the search. Adding the * symbol anywhere in the search term algorithm could allow for uptake due to a potentially broader search at the missing letter(s) where the symbol was entered.

It was noted in the review that limiting or narrowing the search can be further facilitated by selecting the “change” button on the screen. One can limit the articles by publication year, author, and type of publication such as journal article, book chapter, dissertation, etc. Moreover, it was suggested that another set of tabs or refining fields could be triggered when the list of records appears. Through such additional options, targeted information could be more quickly and narrowly refined, for instance, through a system of tab clicks. These features likely would enhance user experience, navigation and search output; but again, each has budgetary implications for web development.

Navigating the database

The Database currently has only one field that has a clickable “search” button with a subject and author search option. However, if the Database were to be upgraded in terms of enhanced search capacity domains (e.g. year of publication, region, etc.), a more specific search could be facilitated.

Upon entering search terms into the current Database, bibliographic citations or ‘records’ are listed. Each record indicates the Author(s), Title, Publication, Year, Journal Citation and Abstract. The Database indicates this information with additional lines for Finding(s) of the Article, Geographic Location, and Data Location.

For ease of user access, the title of the citation could be hyperlinked to the full electronic text; however, this may not be possible due to copyright restrictions. If not possible, indicating that the Database only contains study abstracts, would add clarity for users. Alternatively, access to PDF, HTML, or Word electronic text could be facilitated via providing a link to the text; however, compliance with publication requirements must be upheld. Some library databases indicate where users can access the microfiche or archives if the article is not electronically available. Some even offer support services to order an online version of the full-text article.

In trialing the search function, it appeared that the same citations and abstracts in the same order were generated for term searches on both “mental” and “schizophrenia”. More specificity in search terms thus appears needed. Putting the onus on the user to weed through a broad list of items may decrease efficiency and user tolerance for searching. This invites consideration of mechanisms such as term boxes or search fields which would sort citations that match with the search term(s) based on the rank of relevancy. For instance, if the search terms are “schizophrenia”, “addiction” and “bipolar”, the first citation should be the best match that encompasses all terms rather than only broadly listing records that include the search term. Accordingly, the rank of relevancy could be determined by how many times the search term(s) appears and where it appears within the entry.

The list of records per page ideally could be determined by the user, with a reasonable default determined by the provider. If the user is determining the number of records, an option box could offer logical options. Options to open the relevant records and click on the box beside the record number to “Mark”, “Save”, “Print” or “Email” the record (citation), are recommended.

Some databases including Google books, indicate “Cited Reference” which shows how many other users have cited a particular author or article. This would highlight commonly used resources. A “Help and Support” function would invite users to submit any questions via email. A greater sense of trust in the Database provider and accessible support were thought to likely result from such augmentation of the Database. Toward the bottom of the page, an “Exit” button was suggested to allow users to easily exit the page and potentially navigate to the search screen for a new inquiry.

It was suggested that Database functionality could be enhanced through a “Display Records” function that could display a full or truncated format, depending on user preference. Reviewers further suggested capability to retain and store citations and download them into Word or PDF formats.

Discussion

This evaluation highlights the Database as a useful and promising resource in proactively redressing barriers imposed on mental health organizations and communities relative to their current lack of easy access to salient peer-reviewed literature. The Database initiative thus appears to have particular importance for knowledge access, translation and mobilization in service delivery and advocacy communities. It appears to offer a valuable and helpful resource, particularly if expanded to also include other mental health conditions. To our knowledge, this project is unique in Canada in offering an accessible review of the literature. In our search of the literature, we found reports on the development of databases; however, these were focused on patient record access and based in the United Kingdom and Australia. The focus of this project is unique in that it addresses a gap in mental health research accessibility for service providers.

Enhancing the Database through heightened visibility, functionality and navigation would enhance what likely could become an important contribution to the mental health field in Canada. It is anticipated that recommended enhancements would render the Database even more viable to the novice user, especially if a compelling Database branding approach enhanced access, and a simple training package for new users was available.

Of note here, such an aim of innovation in knowledge translation is a pressing priority for health services more broadly [3]. Addressing this need, the Database already offers a useful model both for professional and lay users. Its inclusive approach importantly invites broad information access to the mental health and NGO community.

Finally, the Database importantly offers a leveling to the traditionally hierarchical and inequitable pattern of knowledge access and research distribution. Through an integrated ‘community’ model of knowledge access and mobilization implicit in the Database, scholarly information can be obtained ‘on the ground’. This outcome seems consistent with aims of empowering communities to advance practice and policy within a reciprocal and participatory approach that both yields and effectively uses knowledge [3].

Accordingly the Database seems well-positioned to inform and foster evidence-based interventions and programs [5]. It emerges as an important initiative heightening access to research findings, thus fostering potential for capacity building for a range of mental health stakeholders. It offers a tool for knowledge mobilization at a level that is uniquely and beneficially targeted to key users and developers of mental health services and policy. To that end, the Database is an endeavor worthy of continued development and investment.

Acknowledgements

The authors report no conflict of interest. Funding was received through a federal Student Temporary Employment Program and the Alberta Alliance for Mental Illness and Mental Health.

References

1. Hoagwood K, Johnson J (2003) School psychology: a public health framework: I. From evidence-based practices to evidence-based policies. Journal of School Psychology 41: 3-21
2. Canadian Institutes of Health Research (2013) About knowledge translation and commercialization.
3. Broner N, Franczak M, Dye C, McAllister W (2001) Knowledge transfer, policy making and community empowerment: A consensus model approach for providing mental health and substance abuse services. Psychiatric Quarterly 72: 79-102
4. McGrath P, Lingley-Pottie P, Emberly D, Thurston C, McLean C (2009) Integrated knowledge translation in mental health: Family help as an example. Journal of the Canadian Academy of Child and Adolescent Psychiatry 18: 30-37
5. Kirby MJL, Keon WJ (2006) Out of the shadows at last: Transforming mental health, mental illness and addiction services in Canada.
6. Mental Health Commission of Canada (2013) Knowledge exchange centre
7. Boydell K, Stasiulis E, Barwick M, Greenberg N, Pong R (2008) Challenges of knowledge translation in rural communities: The case of rural children’s mental health. Canadian Journal of Community Mental Health 27: 49-63.
8. Barwick MA, Boydell KM, Omrin C (2002) A knowledge transfer infrastructure for children’s mental health in Ontario: Building capacity for research and practice. The Hospital for Sick Children, Toronto.
9. Lincoln YS, Guba E (1985) Naturalistic inquiry. Sage, Thousand Oaks.