Cross-cultural Adaptation, Internal Consistency and Validity of the Handwriting Proficiency Screening Questionnaire (HPSQ) for Spanish Primary School-age Children

Handwriting is a complex human activity and an essential fine motor skill in school-age children that requires the maturity and integration of cognition, visual perception and fine motor skills [1]. Handwriting and spelling skills together with reading and maths skills are part of the repertoire needed to succeed in school. Research shows that approximately 3–21% of children have difficulty with this essential task of writing [2]. Poor handwriting performance, namely reduced speed and legibility, has been linked to decreased self-esteem and lower academic achievement. In order to be assisted, these children are frequently referred to school-care occupational therapists [3]. However, prior to treatment, handwriting capacities should be evaluated as a means to determinate the severity of the problem and the goals of our intervention. With this aim in mind some researchers elaborated several handwriting evaluation scales capable of producing quantitative scores for handwriting quality [4,5]. Nowadays, appropriate tools for handwriting performance evaluation in school settings are scarce. The Handwriting Proficiency Screening Questionnaire (HPSQ), developed by Rosenblum [6], tries to provide a solution to this problem. The HPSQ is inexpensive and technically simple to implement in environmentally friendly settings such as in the child’s classroom. In her work, Rosenblum showed that, despite the fact that the HPSQ is a subjective handwriting evaluation based on the teachers’ view of pupils’ handwriting proficiency; the questionnaire successfully reflects the constellation of handwriting problems in children. Moreover, her analyses of the psychometric properties of the questionnaire have revealed a high internal consistency (.90) and good test–retest reliability (.84, p < .01) [6]. According to the author, the HPSQ has been translated into several languages, but not into Spanish. Additionally, to the best of our knowledge, observational questionnaires for handwriting evaluation in Spanish have not been actually developed, and no previous studies evaluating handwriting in children with an observational questionnaire have been reported in Spanish population.

The purpose of this study was designed to translate and culturally adapt the HPSQ into Spanish and to obtain a reliable and viable version of HPSQ for Spanish school-age children. To this aim we followed a sequential procedure (forward translation – back translation) similar to that followed in the IQOLA (International Quality of Life Assessment) project [7] developed to obtain the different versions of the SF-36 [8], the DASH and other instruments as the CTS [9,10]. Finally, the internal consistency, construct validity and content validity of the new Spanish version were also assessed.

2.1 HPSQ original version

The original HPSQ [6] is composed by 10 items which evaluate legibility (items 1, 2, 10), performance time (items 3, 4, 9) and physical and emotional well-being (items 5, 6, 7, 8). The items are worded so as to be directly answerable by teachers from their observations of children as they are writing in the classroom. For example, “Does the child often erase while writing?” The items are scored on a 5-point Likert scale, ranging from 0 (never) to 4 (always), with higher scores indicating poorer performance. The final score is computed by summing the scores of all the 10 test items.

2.2 Translation and adaptation process

Before the adaptation process, permission was obtained from the author of the original instrument to translate and validate it into Spanish. The theoretical framework for cross-cultural adaptation used in this study is one of the most extensively described proposals for instrument adaptation, first advocated by Guillemin et al. [11] in the 90s and secondarily revised and updated by Beaton [12]. The sequence of this process was as follows: forward translation, synthesis, back-translation, expert committee review and pilot study (Figure 1).

Figure 1: Translation and adaptation process.

Step 1: Forward translation

The original questionnaire was translated into Spanish by two bilingual translators with clinical experience whose native language was Spanish. Each translator prepared a separate translation. Both of them were provided with a sheet containing item definitions and a paragraph for each item to explain potential item–specific translatability problems.

Step 2: Synthesis of the translation

No special difficulties to obtain conceptually equivalent expressions in Spanish were found. Both Spanish versions were discussed simultaneously between the translators and the main researcher to get a consensus and produce the final first adaptation (version 1.0).

Step 3: Back Translation

To assess the conceptual equivalence, the Spanish version 1.0 was translated back into English by two bilingual translators living in Spain whose native language was American English. Back translators were not informed about the concepts underlying the items content and were not medically schooled or trained. They were simply instructed to translate the Spanish items into American English.

Step 4: Expert Committee

An expert meeting was organised to consolidate all the versions. The expert committee was composed of a methodologist, an occupational therapist, a school teacher, as well as the translators that had participated in the previous steps of the process (both forward and backward translators).

During the expert meeting two back translations were compared with the original version to identify those items or words that were not exactly equivalent. Discrepancies between the translations or potential cross-cultural adaptation were discussed. Final decisions were made to generate a pre-final version of the HPSQ–SP, a version ready to be tested by school teachers (version 2.0).

Step 5: Pilot Study

Before field testing, the authors evaluated content validity of the HPSQ-SP by recruiting a team of 15 teachers with at least 10 years of experience each, as potential users of this instrument. Teachers rated individual items on the degree to which they do or do not measure specific objectives listed by the test developer. The aim of this stage was to evaluate the questionnaire's applicability, and the comprehension of the new Spanish version. The scoring system for each questionnaire item was as follows: +1 = clearly measuring; 0 = degree to which it measures the content area is unclear; −1 = clearly not measuring. The item-objective congruence (IOC) value for each item was calculated using the summation of scores from each expert divided by the number of experts [13].

With the aim of studying construct validity and internal consistency 165 children (aged between 6 and 9, M=6.95 SD=.75) from five different private and public schools of Malaga and Granada (Spain) were evaluated. Students were excluded if they had problems with communication, upper extremity disorder, psychological disorders or a history of cerebral disorder. The team of 15 school-teachers that had evaluated questionnaire objective congruence rated the handwriting skills of the children with the 2.0 version of the questionnaire. All teachers were instructed to answer the questionnaire independently, without any assistance from any researcher. This study was approved by the Research Ethics Committee of the University of Malaga.

Data analysis

Content validity was evaluated by using item-objective congruence (IOC) value. An IOC value of 0.5 or more is considered satisfactory [12].

A principal-components factor analysis was employed to analyse the structure of the questionnaire (construct validity). The internal consistency of the Spanish version of HPSQ was measured by calculating Cronbach’s alpha coefficient whit the acceptable value of > 0.7. All data analyses were performed using SPSS 22.0.

Step 1 to 3: Forward translation, synthesis and back translation

In general, no mayor difficulties were found during the whole process. There was no need to modify any item by the expert committee. Items were easy to translate and no relevant problems with cross cultural adaptation were referred by translators.

There were little differences between both forward translation versions (version T1 and version T2). Discrepancies were found in item 7 and 8 about the word “while” and the translation into Spanish language without losing the original meaning. After a careful discussion, an agreement was reached and the word “mientras” was used. Then, back translation was carried out, resulting in two versions (BT1 and BT2). Both still retained the questionnaire's original meaning but differences were found in terminology. In the translation from the Spanish version to the English version the word “mientras” in item 7 and 8 was rewritten in English language as “while”.

Step 4: Expert Committee

During the expert committee meeting no cultural issues were identified. About the word “while” in item 7 and 8 it was decided that “mientras” was appropriate translation for the final version. There were no inconsistencies between the back translations and the original items, this warranted the initial forward translations.

Step 5: Pilot Study

There were no missing items. School-teachers reported that items were clearly written and very practical. None recommended changing the language used in the questionnaire either, however, two different teachers referred that they perceived item 10 to be difficult to quantify because in certain cases, children could be satisfied or no with their own writing, and that does not mean they have a suitable graphics.

Content validity

It was established by quantitatively measuring the item-objective congruence (IOC) for each item in the questionnaire [13]. Table 1 presents the score provided by each expert to each item and the mean score for each item. As it is observed the IOC value of every item ranged from 0.73 to 1.00, indicating good content validity [14]. The highest IOC (1.00) was observed in most of the items, except for items 1 (IOC 0.80) and 10 (IOC 0.73).

Table 1: Teachers’ evaluation of content validity of the HPSQ-SP. Note: Rows indicate the number of the questionnaire item and columns the punctuation provided by each teacher (T) to each item. IOC indicates Item-objective Congruence mean score for each item.

HPSQ scores and construct validity

The mean HPSQ questionnaire score for the group of 165 children was 5.69 (SD= 4.18). Scores in the test ranged between 0 and 19. No differences were found regarding gender, boys performed similarly to girls (respectively, Girls: M= 6.02, SD= 4,57; Boys: M= 5.27, SD= 3.62; t(163)=1,163; p=.25). Factor analysis yielded similar results to those obtained by Rosenblum. Two factors were extracted using orthogonal rotation that accounted for 35% and 14% of the variance respectively (with a similar structure, the original questionnaire explained 65% of the variance [6]). As in Rosenblum study the first factor comprises items 3 through 9 (performance and well-being) and the second factor (legibility) includes items 1, 2 and 10. The factor structure is given in Table 2. The results and the parallelism between versions, suggest the Spanish version had acceptable construct validity.

Table 2: Orthogonal Factor loading matrix for the ítems in the HSPQ questionnaire.

Internal consistency

The internal consistency of the Spanish version of HPSQ was moderately high as reflected by a Cronbach alpha value of 0.78.

The main aim of the study was to verify the conceptual equivalence and validity of the Spanish version of the HPSQ in order to provide Spanish teachers and occupational therapists with an easy, fast and reliable instrument to evaluate handwriting difficulties in primary school children. The cross-cultural adaptation was carried out following the recommendations of Beaton [12].

No major discrepancy arose during the adaptation process. Some minor language discrepancies occurred during the forward and backward translations; they were discussed and easily resolved by the expert committee. No inconsistencies were found between both forward and back translations and the original items, indicating that the translated version conveys the same item content as the original version. In fact, the back translation from the English native back translator was almost perfectly equivalent to the original, suggesting that items did not require cross-cultural adaptation. Only in item 10 two participants referred an appreciation in the pilot study because they perceive this item to be difficult to quantify. Teachers referred that, in some cases, children could be satisfied with their own writing and that does not mean they have a suitable phonographic knowledge. It is possible that children’s psychological characteristics, like self-esteem, may modulate responses to this question, and although this should be considered in a new version of the questionnaire, we decided not to modify this item to respect to the original questionnaire. Anyhow, either our work or Rosemblum´s [6] presented no major problems about this item.

Teachers’ evaluation of the items questionnaire indicated good content validity, with values ranging from 0.7 to 1. Items’ objective congruence (IOC) values were higher than 0.5, (range 0.7-1). The highest IOC (1) was observed in 8 out of 10 items. The Spanish version of the HPSQ presented also a good internal consistency (Cronbach’s alpha= 0.78) for the 10 items, although lower than the one shown by the original version (Cronbach’s alpha= 0.9). Anyway, this indicates a high consistency between the questionnaire items. Principal-component analysis identified two factors that explained 50% of the variance. Importantly, the factorial structure of the Spanish version paralleled that of the original version identifying one factor associated to performance and well-being and another to legibility. All in all, the psychometric properties of the Spanish version of the HPSQ made it a valid tool for assessing handwriting skills. It should be commented that both the internal consistency and the proportion of the variance explained by the factorial analysis are lower than in the original questionnaire. A reason for this could rely on two factors: i) the different number of participants (165 in our study versus 230 in Rosenblum’s) and ii) in the different characteristics of the samples, whereas the age of our participants ranged between 6 and 9, it ranged between 7 and 14 in Rosenblum’s study, this makes room for more variability in their study and hence the chance to get more precise estimates.

In future, it could be useful for occupational therapy intervention to look for correlation between poor handwriting and manual dexterity in primary school children. Previous studies find a correlation between legibility and hand-writing disorders in children with different pathologies [15,16]. But deep and more specific studies should be done in order to determinate this correlation in healthy primary school children.

Some limitations have been found in this study. Certain authors, as Rosales et al. [9] and Alonso et al. [17], recommended quantifying the conceptual equivalence and the difficulty of the translation process. The lack of inter-rater agreement in the reliability analysis can also be considered a limitation that should be taken into consideration in future investigations. In the present study other forms of validity as convergent validity have not been contemplated because of the absence of tests in Spanish to assess the quality of handwriting. In any case, due to the large parallelism between both versions, it seems reasonable to consider that the validity of the Spanish version and the original one are not very different. Finally, an avenue for future research is the exploring of the discriminant validity of the Spanish version of the questionnaire.

For occupational therapists, it is necessary to have reliable instruments to evaluate handwriting capacities as a means to determinate the severity of the handwriting problems, the goals of interventions and the progression of therapies. In spite of the fact that different evaluation scales have been developed in the last years in order to standardized handwriting quality evaluations, handwriting evaluation in occupational therapy environment are scarce. The area demands a clear evaluation instrument, concise and reliable in all languages, in order to compare studies and therapies in different countries and cultures.

This paper provides an easy and quick screening tool for handwriting proficiency that can be used for Spanish language children. HPSQ Spanish version can assist teachers, occupational therapists, doctors, physiotherapists and other professionals in identifying handwriting problems in primary school children. It also provides the preliminary basis for further studies that can be conducted in other Spanish-speaking children populations and in future studies aimed to analyse the relationships between manual dexterity, hand skills developing and writing.

Additionally, it should be considered the relevance of developing versions of the questionnaire for different school (age) levels to provide teachers an easy and fast questionnaire to quantify the quality of handwriting in children at different ages and hence, to increase our capability of detecting handwriting pathologies and their evolution with more confidence.

To conclude, although there are still ergonomic, biomechanical or unspecified aspects of handwriting which may not be determined whit HPSQ-SP, it can definitely assist teachers and occupational therapists in identifying handwriting problems in primary school children.

Authors would like to acknowledge the institutions “Madre Asunción School” from Málaga (Spain) for their voluntaries participation in this study.

The authors report no declarations of interest. No actual or potential conflict of interest exists and no external financial support was sought or received for the conduction of this review.