Journal of Community Medicine & Health Education
Open Access

Quality of care is an important determinant of health services utilization ultimately influencing health outcomes of public health importance [1]. It is known that the perception of the users about the quality of service offered in a health facility is a determinant of choice of provider and willingness to use or pay for the services [2]. There are gaps in the literature addressing assessment of perceived quality of child health care in Africa. For instance, studies on quality of care in Nigeria have mostly not reported the processes of validation of the tools used as such there are varying degrees to which their reports are reproducible [3-10]. Also, review of African health literature yielded only one locally developed quantitative tool for assessing users’ perspectives of quality of care in health facilities that had been subjected to systematic, quantitative content validation [11]. The tool was however not specifically for child health care services.

We used a three phase approach to create a validated tool for the assessment of caregiver’s perceived quality of child health care. In the first phase we conducted an explorative study using focus group discussions (FGD) sessions to collect qualitative data on perception about the quality of child health care services in PHC facilities. Participants were men and women selected from some communities in South West Nigeria. Analytical framework and codes that focused on identifying recurrent, dominant and divergent opinions were developed. The exchanges in the FGD sessions relating to perception of quality of child health care were organized into 39 quality related items. The 39 items fell into three major areas corresponding to: Perception about health workers; perception about the health services; and perception about the health facility. The report of this first phase is documented elsewhere.

In the second phase we used a modified Delphi technique to identify the critical issues amongst the 39 quality of care related items earlier identified based on expert opinion. The Delphi method is a structured communication technique, originally developed as a systematic, interactive forecasting method which relies on a panel of experts [12]. It is based on the principle that forecasts (or decisions) from a structured group of individuals are more accurate than those from unstructured groups [13,14]. Typically the panelists are requested to contribute their opinions on the subject of interest in a listing fashion or otherwise provide answers to a questionnaire. They are also required to score the opinion based on their priority to the issue of interest. Descriptive statistics are used to analyze scores by calculating means, medians, and standard deviations for each item data for each iteration. Consensus on opinions are determined by computing the inter-quartile range (middle 50% of response) for each of the identified items or priority statement [14].

Though originally formulated by military strategists, it has been widely used in such fields as public policy, education and health usually in an adapted form for instance: the modified Delphi method [14]. While the original version starts with the expert panel contributing items, modified method usually starts with the researcher developing a list of items from literature review or analysis of qualitative data on the subject of interest. The panelists are then requested to rank the items during several iterations. The method is useful for providing evidence for content validity in quantitative scale design [15], Expert opinions provide strong basis for including or excluding items in the scale. The third phase of the study tested the scale amongst caregivers of under-five children in selected PHC facilities. This paper reports findings of the second phase.

This study was carried out between October and November 2014. The study population consisted of public health physicians, anthropologist, sociologists and pediatricians practicing in Nigeria. Some of the experts were engaged by peer recommendation and others through their association mailing list, in particular, Association of Public Health Physicians of Nigeria (APHPN). Thirty-three experts practicing in 11 states of the Federal Republic of Nigeria responded to the survey. Of the thirty-three, 27 had post graduate qualification in Public Health, two in Pediatrics, two in Sociology and two others were medically qualified public health practitioners.

An online survey platform was used via a commercial online survey website: www.createsurvey.com. The online method was adopted because it allowed for reaching a significantly large number of potential respondents without requiring their physical presence. The 39 ‘quality’ items that were identified from the FGD were modified to be read as questions and uploaded on the platform. Each expert respondent received an email which contained a link to the online questionnaire. The experts were requested to rate each item on the questionnaire on the appropriateness of the question for soliciting users’ perception of quality of child health care services from caregivers. It was five-point rating scale where 1 represented ‘not appropriate’ and 5 represented ‘very appropriate’.

The expert review was done in three iterative rounds. In each round the experts rated the questionnaire items, gave suggestions for rephrasing any of the questions where they deemed fit and gave suggestion of any other item/question that they believed should be included in the list. The inter-quartile range (IQR) of the scores for each question item from each round were calculated. For each question item, the lower the IQR the higher consensus amongst the reviewers. This means that the items with lower IQR were the more usefulness items for assessing perception of quality of care based on expert consensus. After each of rounds, the questionnaire was modified to include comments and suggestions of reviewers in previous rounds.

The 1^st round of the modified Delphi process started with 39 quality related question items derived from FGD sessions. The responses of the reviewers showed that about only 41% of the question items had IQR greater than 1.5 (Tables 1-3).

Table 1: Round one items and summary statistics (health worker related items).

Table 2: Round one items and summary statistics (health services related items).

Table 3: Round one items and summary statistics (Health facility related and unclassified items).

Based on some of the comments of the reviewers, three question items were merged into one because of their similarity. These are: “The drugs given in this health facility are effective”; “The drugs given in this health facility are of good quality”; and “Your children recover from their illness when they are treated in this health facility”. These were merged into the following single question item: “Children recover from their illnesses when they use the drugs provided/prescribed in this health facility”.

Of the remaining items, three that had IQR greater than 1.5 were completely excluded from the subsequent round. These are: “The number of health facilities in this community is adequate” [IQR=2.0]; “This health facility has sufficient beds for patients” [IQR=2.5]; and “In your opinion, this health facility has adequate equipment for care of the patients” [IQR=2.5]. Furthermore, the reviewers suggested the addition of three new question items as follows; “The health workers do follow up after treating children in this health facility”; “You are likely to use the child health services of this health facility again or recommend it to others (friends, relatives etc)”; and “Health workers allow for participation of users in monitoring and improving the quality of child health services in the health facility”. In all, round one started with 39 items, three items were merged into one, three items were excluded and three new items were added. Thus, thirty-seven question items went into the second round. Some of the reviewers’ comments from round one are:

“There are some uncomfortable questions for the clients especially if the study is facility based. The clients may not be entirely truthful in answering some questions bothering on the health workers’ attitude especially on issues like favoritism, lateness, and corrupt practices”.

“The question to ask is: is the patient in a good position to be knowledgeable about the issue he/she has been asked to comment on? In many of the questions, the client is not likely to have good knowledge of the issue. e.g. drugs quality, equipment availability etc”.

“Some of the questions assume that the users will have the competency to ascertain the claims e.g. clients may not be able to determine whether health workers are competent or not. They may also not be able to determine whether the lab request are appropriate or not”.

“There is a need to be more specific with some of these questions and avoid having two or more ideas in one question”.

What is the meaning of “sufficiently available”, I think asking the patient whether drugs prescribed are usually available in the health facility is more appropriate; “sufficient immunization service” is also an ambiguous question, “are the immunization needed for the child always available” is more appropriate.

In the 2^nd round, higher consensus was achieved as 87% of the question items had IQR lower than 1.5 (Table 4). Following this high level of consensus on several of the items, the cut off point for good consensus was reduced to IQR to 1. There were nine items with IQR greater than 1 and only these were presented for the third round. Furthermore, amongst the items with IQR less than or equal 1, specific comments from the reviewers indicated that a number of them were redundant or not suitable for the targeted respondents to be able to assess and give appropriate responses (Table 5). Based on the comments of the reviewers, these items were excluded from the final tool. Thus after analysis of responses to the second round, 21 items were considered to already have sufficient consensus and needed no further deliberations upon, 7 items were completely excluded and 9 items were introduced into the third round. Specific comments from the reviewer in the second round include the following:

Table 4: Round two items and summary statistics (health worker related items).

Table 5: Round two items and summary statistics (health services related item).

“Health workers are competent in their work and diligent in their work may mean different things but to the patients they may not be able to differentiate the questions”

“What makes a health education appropriate? Is it when it addresses my issue or not? Adequate time during consultation looks subjective to me, why not takes time to attend to patient's problems? What about questions that addresses empathy? What about answers to questions asked by patients? Do they allow them to ask questions?”

“The health service has satisfactory immunization services'- I would have preferred that what is meant by satisfactory is explained if possible- like if the vaccines are available or the staff nice, etc?”

“What do you mean by satisfactory immunization service please explain and possibly rephrase. The issue of service being ‘properly organized’ and ‘preferable to services in other facilities’ need to be rephrased in such a way that the caregiver can give their opinion without prejudice”.

“The follow up after treatment is good but it is important to consider possibility of the home visit as part of health workers' activities. Also consider: Direct community service in form of clearing the surrounding bushes and protecting facility properties against robbers”.

In the third round two approaches were used to seek consensus amongst the reviewers. Of the nine items in round three, six were displayed in the five-point scale format as in the previous rounds. The remaining three were put up for a vote of whether to exclude or include them. The reason stems from comments of some reviewers suggesting that these three items should be removed while others suggesting they should be retained. All the six items displayed in the five-point scale format had IQR more than 1.5 (Table 6). They were all excluded from the final set of items. Of the three items that were voted upon two were voted to be retained but after deliberations amongst the authors, only one of them was retained. Also one of the reviewers suggested a new item which had not featured all along during the Modified Delphi process. Considering the value of the question it was added to the final tool. The question item is: “In your opinion, the health workers in this health facility take time to explain to the caregivers about the illness of their child/ward”.

Table 6: Round three items and summary statistics.

At the end of the three rounds of expert review therefore, the total number of question items that had shown consensus considerably amongst the expert reviewers were 23 question items.

For these 23 items, a three point Likert scale response format was framed appropriately as a quantitative scale for assessing perceived quality of child health services in PHC facilities.

The tool development which started with 39 quality related items ended with 23 in the third phase. The 23 items fitted into the three broad categories of perception on health workers, health services and the health facilities. It is important that the number of question in surveys is limited to the barest minimum without compromising on the internal structure and reliability of the tool during the development process. Scales composed of large numbers of items that are irrelevant to the experiences of a specific patient are more susceptible to receiving a biased rating [16]. Furthermore, the shorter the tool the more cooperative the respondents are likely to be and the more likely that correct responses are provided. This is even more important if such tool will be administered as an exit interview in a busy clinic setting. Although the description of the specific expressions given to the items of the new tool were different, they did overlap with the items of the pre-existing tool on perceived quality of care in primary health care facilities created by [11]. The item of the new scale however specifically focused on child health care. Table 4 contains the 23 final items.

Modified Delphi technique is a very useful method of seeking expert opinion through consensus on technical issues. The method helped to create a 23-item tool for assessment of user’s perception of child health care services in PHC facilities in Nigeria from an initial list of 39. Methods such as this help to provide evidence for validity of content when creating quantitative scales from the scratch.