Validation of a specific measure to assess health-related quality of life in patients with schizophrenia and bipolar disorder: the 'Tolerability and quality of life' (TOOL) questionnaire
© Montejo et al; licensee BioMed Central Ltd. 2011
Received: 24 December 2010
Accepted: 11 March 2011
Published: 11 March 2011
Perception of quality of life may differ depending on the perspective. The aim of the study was to assess the psychometric properties of the Spanish version of the 'TOlerability and quality Of Life' (TOOL) questionnaire, a specific self-rated instrument to evaluate the impact of side effects of antipsychotic drugs on health-related quality of life (HRQoL). The questionnaire consists of eight items answered on a four-point Likert scale.
A psychometric study was conducted with clinically stable outpatients with schizophrenia and bipolar disorder under antipsychotic treatment. The translation and cultural adaptation of the questionnaire was performed according to international standards. Internal consistency using the Cronbach α coefficient and test-retest reliability using the intraclass correlation coefficient (ICC) was used to assess the reliability of the instrument. Patients completed generic and specific measures of quality of life and clinical severity.
A total of 238 patients were analysed, with a mean age of 42 years (SD 10.9). The mean completion time was 4.9 min (SD 4.4). Internal consistency and intraclass correlation coefficient were adequate (Cronbach α = 0.757 and ICC = 0.90). Factorial analysis showed a unidimensional structure (a single eigenvalue >1, accounting for 39.1% of variance). Significant Spearman's rank correlations between the TOOL and both generic and specific measures were found. The questionnaire was able to discriminate among the Clinical Global Impression - Severity scores (Mann-Whitney U test, P < 0.001).
The TOOL questionnaire shows appropriate feasibility, reliability, and discriminative performance as a patient-reported outcome. TOOL constitutes a valuable addition to measure the impact of adverse events of antipsychotic drugs from the patient perspective.
Schizophrenia and bipolar disorder are worldwide prevalent and severe mental diseases [1, 2]. Newer antipsychotic treatments have proven useful in reducing both relapses and the severity of symptoms . However, weight gain, extrapyramidal symptoms, sexual dysfunction, or sedation are quite common side effects among patients under antipsychotic treatment . The occurrence of these symptoms may affect patient adherence to medication, leading to discontinuation, limiting treatment effectiveness, and increasing both personal and social costs [5, 6]. Therefore, when comparing alternative therapies, side effects and their impact on patient health-related quality of life (HRQoL) could be of great importance in order to define the most efficient antipsychotic drug treatment [3, 7].
The systematic assessment of patient perspective may provide valuable information that could be lost when relying only on clinical evaluation. In chronic conditions such as schizophrenia and bipolar disorder, there are advantages in using patient-reported measures to understand complex needs and improve alliances between patients and clinicians . The importance of involving patients in their own healthcare, and of patient-reported assessments, is increasingly recognised .
Many efforts have been made to develop or validate specific instruments to assess patient affectation within different domains [10–13]. Regarding side effects, the most widely used and specific measure is the Udvalg for Kliniske Undersøgelser (UKU) side effects scale . This questionnaire, filled out by clinicians or patients, comprises 56 items that refer to psychic, neurological, autonomical and others effects. Despite this scale, more specific tools in terms of HRQoL are still needed to afford greater insight in describing and grading the impact of side effects associated to antipsychotic drugs [15, 16]. A brief instrument is preferred for use in clinical practice and in investigations such as clinical management studies or clinical trials comparing the effectiveness of treatments.
A specific measure to assess HRQoL impairment related to adverse events of antipsychotic drugs has been previously developed and validated in Sweden: the 'TOlerability and quality Of Life' (TOOL) questionnaire . This self-rated measure reflects the subjective perception of side effects in patients treated with antipsychotic medication. It comprises eight attributes and has four levels per domain (Likert scale: 1 = minimum to 4 = maximum impact). These domains are mood (worry-upset), function capabilities, and several adverse events frequently associated with antipsychotic treatment (fatigue-weakness, weight gain, stiffness-tremor, physical restlessness, sexual dysfunction, and dizziness-nausea). In contrast to the Drug Attitude Inventory  or the Subjective Well-being under Neuroleptic scale , the TOOL questionnaire was specifically designed to identify from the patient perspective the most common adverse events of both typical and atypical antipsychotic drugs [4, 18].
The aim of the present study was to evaluate the linguistic adaptation and psychometric validation into Spanish of the TOOL questionnaire for the assessment of side effects in patients treated with antipsychotic medication and their impact on health-related quality of life.
Linguistic adaptation of the TOOL questionnaire
Forward/backward translations of the original TOOL questionnaire were completed by expert translators. Firstly, three independent Spanish experts translated the original version in Swedish into Spanish. Experts examined and compared these three different versions in order to reach a single one by consensus (intermediate version 1.0). Secondly, one Swedish expert translated this intermediate Spanish version again into Swedish (backward translation) and compared his results with the original version to ensure conceptual equivalence (intermediate version 2.0). Finally, all the expert translators participated in the proof reading test of the intermediate version 2.0, and the final Spanish version was thus established.
Subsequently, the Spanish version was reviewed by a panel of experts (five psychiatrists and one general practitioner specialised in HRQoL). According to expert criterion, three items were modified to facilitate patient comprehension: mood, physical restlessness, and dizziness-nausea. Next, 40 clinically stable patients (20 with schizophrenia and 20 with bipolar disorder) filled out the Spanish version of the TOOL. They were also asked to review this version in terms of comprehension (C) and importance (I) using a scale ranged from 0 (lowest level of C/I) to 4 (highest level of C/I). All the items scored higher than two points (threshold) in both scales. Consequently, the Spanish version of the TOOL questionnaire was ready for validation (Additional file 1).
Psychometric validation of the TOOL questionnaire
A multicentre, non-interventional psychometric study was conducted. The study was carried out at 60 psychiatric centres throughout Spain, and was approved by the institutional review board of the University Hospital of Salamanca (NCT00692133).
Participants were outpatients treated in community healthcare centres. Eligibility criteria included being at least 18 years old, having a Diagnostic and Statistical Manual of Mental Disorders, 4th Edition, Text Revision (DSM-IV-TR) diagnosis of schizophrenia or bipolar disorder as established by the Structured Clinical Interview for DSM-IV, being clinically stable (defined as having had no changes in severity or new treatments initiated in the last month), and taking a single oral antipsychotic medication. After complete description of the study to the participants, written informed consent was obtained.
Investigators completed the following scales: Positive and Negative Syndrome Scale (PANSS)  (only to patients with schizophrenia), Young Mania Rating Scale (YMRS)  (only to patients with bipolar disorder), Montgomery-Asberg Depression Scale (MADRS) , Clinical Global Impression - Severity scale (CGI-S) , and UKU Side Effect Rating scale (UKU) .
Retesting was carried out 2 to 3 weeks after the first visit. In order to analyse the psychometric properties of the TOOL questionnaire, feasibility, reliability and validity were studied.
The item response rate of the TOOL questionnaire was described and completion time was registered as well. Finally, both floor-ceiling effects were evaluated in an exploratory analysis of the answers given to the TOOL distribution and by using percentiles 25, 50 and 75 (Tukey's Hinges).
Regarding construct validity, item-total correlation (ITC) was checked following the criterion of removing items with a score lower than 0.20 in the discrimination rate. To assess the dimensionality of the questionnaire, an exploratory factorial analysis (extraction criterion of eigenvalue >1) was also conducted. The Spearman rank correlation coefficient (rs) between the total score of the TOOL and those of the EQ-5 D (unweighted) and the SF-6 D (unweighted) was used to test convergent validity. Correlation was considered to be high for rs ≥ 0.5, moderate for rs values between 0.3 to 0.5, and low for rs < 0.3. This statistic was also applied to determine the relationship between the TOOL and the MADRS, YMRS, and UKU scales. These analyses were performed differentially according to the two groups resulting from clinical diagnosis (patients with schizophrenia or bipolar disorder). In addition, rs was used to study the association between each of the TOOL items and SF-6 D and EQ-5 D measurements.
In order to evaluate criterion validity, patients were categorised taking into account the CGI-S to compare TOOL scores. Accordingly, the collected sample was divided into two groups: patients with no to mild involvement versus patients with moderate to severe involvement. It was hypothesised that patients with lesser involvement would obtain lower scores in the TOOL (that is, lesser side effects and a better HRQoL) than those patients with worse health status according to expert criterion. Again, a non-parametric statistic (Mann-Whitney U test) was used in these comparisons considering the skewed distribution of data.
The internal consistency of the questionnaire was analysed using Cronbach's α. Test-retest reliability of the scores was also examined by computing the intraclass correlation coefficient (ICC).
The sample size was calculated taking into account a specific objective of calculating multiattribute utility function reflecting patient experience of side effects of antipsychotic therapy . Thus, the determination was guided by the estimation of the utility function according to which a large number of patients is needed to maximise the precision and reduce the risk of measurement error. To this end, in the present study a sample size of about 250 patients was considered. Consequently, the number of patients required for the validation process (5 to 15 patients per item) was ensured, and hypothetical comparisons between the 2 independent groups could be established (that is, schizophrenic and bipolar patients) with 80% power and a 5% significance level (2-sided tests), involving 68 patients per group.
Data tabulation, database validation and the statistical analyses were carried out using the statistical packages SPSS (version 14.0; SPSS, Chicago, IL, USA) and Stata (version 10.0; Stata, College Station, TX, USA). For all statistical tests, a level of 0.05 was considered significant.
This trial was registered under no. NCT00692133.
Sociodemographic and clinical characteristics of the sample
Total n = 238
Age, mean (SD)
Median time of treatment, months (SD)
Marital status, n (%):
In a couple
Working full time
Working part time
Retired (result of disease)
Clinical Global Impression Severity (median score = 3.00):
Borderline mentally ill
Treatment with atypical antipsychotic drugs
Clinical characteristics of the sample
EQ-5 D (unweighted)
SF-6 D (unweighted)
The percentage of missing values in each of the items contained in the questionnaire was less than 5%. The mean completion time was 4.91 min (SD 4.48). A ceiling effect was found for the following items: stiffness-tremor and dizziness-nausea (more than 50% of all patients reported not being bothered at all by any of these symptoms).
Spanish 'TOlerability and quality Of Life' (TOOL) questionnaire construct validity: principal component analysis and component matrix
Extraction sums of squared loadings
Percentage of variance
Percentage of variance
Component matrix: Spanish TOOL domains
Spearman rank correlations (rs) between Spanish 'TOlerability and quality Of Life' (TOOL) and generic measures of health-related quality of life (HRQoL)
EQ-5 D (unweighted)
SF-6 D (unweighted)
Spanish TOOL (schizophrenia)
Spearman rank correlations (rs) between generic and specific measures of health-related quality of life (HRQoL)
EQ-5 D (unweighted)
SF-6 D (unweighted)
PANSS: general psychopathology
EQ-5 D (unweighted)
SF-6 D (unweighted)
PANSS: general psychopathology
Spearman rank correlations (rs) between Spanish 'TOlerability and quality Of Life' (TOOL) items and UKU subscales.
Finally, with the aim of testing criterion validity and discriminative validity of the TOOL, patients were classified depending on their CGI-S scores, yielding two groups: patients with no or only mild involvement, and patients with moderate or severe involvement. Differences between these groups of CGI-S in the TOOL and generic HRQoL scores (Mann-Whitney U test) were significant in all cases (P < 0.001). The results highlighted that patients with no or only mild involvement had lower TOOL scores and higher EQ-5 D and SF-6 D scores, indicating a better HRQoL (mild CGI-S: TOOL = 12.168; EQ-5 D = 0.821; SF-6 D = 0.832). In contrast, those patients with moderate-severe involvement showed higher TOOL scores and lower EQ-5 D and SF-6 D outcomes (moderate-severe CGI-S: TOOL = 14.825; EQ-5 D = 0.663; SF-6 D = 0.739).
Agents involved in health technology research have incorporated other important aspects to the basic aims of granting new treatments efficacy and safety, such as those related to patient subjective perception [9, 26]. There is published evidence on the appropriateness and accuracy of self-assessments or self-report evaluations in patients suffering from mental chronic illness [27–29]. Despite this, however, recently published studies have focused on the side effects of antipsychotic drugs and their relationship with HRQoL by following mainly clinician criterion . Few studies have tried to develop specific instruments to comprehensively quantify the impact of side effects on HRQoL based on the patient perspective. This objective should be of primary importance, taking into account the high prevalence of side effects in patients undergoing antipsychotic treatments and their relationship with adherence to treatment .
Our results show that the TOOL questionnaire presents adequate psychometric characteristics for use in patients with schizophrenia and bipolar disorder. The completion time of the questionnaire was low (<5 min) compared with longer questionnaires such as the UKU scale. Moderate-high correlations were found in both bipolar and schizophrenic patients between the TOOL and specific clinician-rated measures as the MADRS, PANSS, and YMRS, as well as a negative relationship with generic quality of life instruments (EQ-5 D and SF-6D), thus highlighting the convergent validity of this instrument. These correlations were also similar to those found in other studies analysing the association between generic and specific measures . Criterion and discriminative validity were proven by the significant differences found in the TOOL between those patients with mild involvement and patients with moderate-severe symptoms. Thus, patients suffering a worse health status according to the CGI-S obtained higher scores in the TOOL, which indicates a more severe impact on their HRQoL, as has been previously underscored [32, 33].
The one-dimensional construct of the TOOL, along with its internal consistency and test-retest reliability, have been proven, thus allowing clinicians to obtain a simple and global score (ranging from 8 to 32) with the same interpretation in terms of severity as the generalised UKU scale.
Despite these results, this study presents some limitations that should be considered. A ceiling effect was observed in two items: stiffness-tremor and dizziness-nausea. This may be due to the idiosyncrasy of the population sample collected in the present study: clinically stable patients under drug treatment, based mainly on atypical antipsychotic agents. Although similar ceiling effects have been reported in other studies [31, 32], these items should be tested in further investigations with schizophrenic and bipolar patients showing a worse health status in order to test discriminative power. The characteristics of the sample may limit the scope of use of this measure. Therefore, it may be arguable whether patients with greater involvement (that is, patients with severe negative symptoms or patients suffering a maniac episode) could provide reliable reports when completing the questionnaire. Although there is evidence supporting the accuracy of such measurement, as has been commented above, the questionnaire should be applied within such populations to test its psychometrical properties. Secondly, it is necessary to analyse the sensitivity of the TOOL in detecting changes in longitudinal studies.
The Spanish validation of the TOOL questionnaire shows appropriate feasibility, reliability, and discriminative performance as a patient-reported outcome to be used for the assessment of the impact of side effects on patient health-related quality of life. This information could be very important to improve therapeutic alliances and treatment adherence among patients with schizophrenia and bipolar disorder.
This study was funded by AstraZeneca Medical Department Spain. The authors would like to thank Aurelio García, Juan José Uriarte, and Fermín Mayoral for their contribution in the linguistic validation.
- McGrath J, Saha S, Chant D, Welham J: Schizophrenia: a concise overview of incidence, prevalence, and mortality. Epidemiol Rev. 2008, 30: 67-76. 10.1093/epirev/mxn001.View ArticlePubMedGoogle Scholar
- El-Mallakh RS, Elmaadawi AZ, Loganathan M, Lohano K, Gao Y: Bipolar disorder: an update. Postgrad Med. 2010, 122: 24-31. 10.3810/pgm.2010.07.2172.View ArticlePubMedGoogle Scholar
- Buckley PF: Update on the etiology and treatment of schizophrenia and bipolar disorder. CNS Spectr. 2008, 13 (Suppl 1): 3-10.Google Scholar
- Edwards SJ, Smith CJ: Tolerability of atypical antipsychotics in the treatment of adults with schizophrenia or bipolar disorder: a mixed treatment comparison of randomized controlled trials. Clin Ther. 2009, 31: 1345-1359. 10.1016/j.clinthera.2009.07.004.View ArticlePubMedGoogle Scholar
- Perkins DO, Gu H, Weiden PJ, McEvoy JP, Hamer RM, Lieberman JA: Predictors of treatment discontinuation and medication non-adherence in patients recovering from a first episode of schizophrenia, schizophreniform disorder, or schizoaffective disorder: a randomized, double-blind, flexible-dose, multicenter study. J Clin Psychiatry. 2008, 69: 106-113. 10.4088/JCP.v69n0114.View ArticlePubMedGoogle Scholar
- Velligan DI, Weiden PJ, Sajatovic M, Scott J, Carpenter D, Ross R, Docherty JP: The expert consensus guideline series: Adherence problems in patients with serious and persistent mental illness. J Clin Psychiatry. 2009, 70 (Suppl 4): 1-46.PubMedGoogle Scholar
- Bridle C, Palmer S, Bagnall A-M, Darba J, Duffy S, Sculpher M, Riemsma R: A rapid and systematic review and economic evaluation of the clinical and cost-effectiveness of newer drugs for treatment of mania associated with bipolar affective disorder. Health Technol Assess. 2004, 8: 1-187.View ArticleGoogle Scholar
- Fonagy P, Mattheus R, Pilling S: The Mental Health Outcome Measurement Initiative: Best practice guidance for local implementation adapted from the report from the Chair of the Outcomes Reference Group. 2005, Leeds, UK: National Institute of Mental Health in EnglandGoogle Scholar
- Thornicroft G, Tansella M: Growing recognition of the importance of service user involvement in mental health service planning and evaluation. Epidemiol Psichiatr Soc. 2005, 14: 1-3.View ArticlePubMedGoogle Scholar
- Hogan T, Awad A, Eastwood R: A self-report scale predictive of drug compliance in schizophrenics: reliability and discriminative validity. Psychol Med. 1983, 13: 177-183. 10.1017/S0033291700050182.View ArticlePubMedGoogle Scholar
- Naber D, Moritz S, Lambert M, Pajonk FG, Holzbach R, Mass R, Andresen B: Improvement of schizophrenic patients' subjective well-being under atypical antipsychotic drugs. Schizophr Res. 2001, 50: 79-88. 10.1016/S0920-9964(00)00166-3.View ArticlePubMedGoogle Scholar
- Voruganti L, Awad A: Personal evaluation of transitions in treatment (PETiT): a scale to measure subjective aspects of antipsychotic drug therapy in schizophrenia. Schizophr Res. 2002, 56: 37-42. 10.1016/S0920-9964(01)00161-X.View ArticlePubMedGoogle Scholar
- Patrick DL, Burns T, Morosini P, Rothman M, Gagnon DD, Wild D, Adriaenssen I: Reliability, validity and ability to detect change of the clinician-rated Personal and Social Performance scale in patients with acute symptoms of schizophrenia. Curr Med Res Opin. 2009, 25: 325-338. 10.1185/03007990802611919.View ArticlePubMedGoogle Scholar
- Lingjaerde O, Ahlfors UG, Bech P, Dencker SJ, Elgen K: The UKU side effect rating scale. A new comprehensive rating scale for psychotropic drugs and a cross-sectional study of side effects in neuroleptic-treated patients. Acta Psychiatr Scand. 1987, 334: 1-100. 10.1111/j.1600-0447.1987.tb10566.x.View ArticleGoogle Scholar
- Revicki DA, Matza LS, Flood E, Lloyd A: Bipolar disorder and health-related quality of life: review of burden of disease and clinical trials. Pharmacoeconomics. 2005, 23: 583-94. 10.2165/00019053-200523060-00005.View ArticlePubMedGoogle Scholar
- Alonso J, Croudace T, Brown J, Gasquet I, Knapp MR, Novick D: Health-related quality of life (HRQL) and continuous antipsychotic treatment: 3-year results from Schizophrenia Health Outcomes (SOHO) study. Value Health. 2009, 12: 536-543. 10.1111/j.1524-4733.2008.00495.x.View ArticlePubMedGoogle Scholar
- Lindström E, Jönsson L, Berntsson A: A patient perspective on side effects of antipsychotic therapy: the TOOL instrument. Value Health. 2009, 12: A361-View ArticleGoogle Scholar
- Montejo AL: Prolactin awareness: an essential consideration for physical health in schizophrenia. Eur Neuropsychopharmacol. 2008, 18 (Suppl 2): S108-S114. 10.1016/j.euroneuro.2008.02.004.View ArticlePubMedGoogle Scholar
- Kay SR, Fizbein A, Vital-Herne M, Fuentes LS: The Positive and Negative Syndrome Scale-Spanish adaptation. J Nerv Ment Dis. 1990, 178: 510-517.View ArticlePubMedGoogle Scholar
- Colom F, Vieta E, Martínez-Arán A, Garcia-Garcia M, Reinares M, Torrent C, Goikolea JM, Banús S, Salamero M: Spanish version of a scale for the assessment of mania: validity and reliability of the Young Mania Rating Scale [in Spanish]. Med Clin (Barc). 2002, 119: 366-371.View ArticleGoogle Scholar
- Lobo A, Chamorro L, Luque A, Dal-Re R, Badía X, Baro E: Validation of the Spanish versions of the Montgomery-Asberg Depression and Hamilton anxiety Rating Scales. Med Clin (Barc). 2002, 118: 493-499.View ArticleGoogle Scholar
- Guy W: Early Clinical Drug Evaluation (ECDEU) Assessment Manual. 1976, Rockville, MD: U.S. National Institute of Health, Psychopharmacology Research BranchGoogle Scholar
- EuroQol Group: EuroQol - a new facility for the measurement of health-related quality of life. Health Policy. 1990, 16: 199-208. 10.1016/0168-8510(90)90421-9.View ArticleGoogle Scholar
- Brazier J, Roberts J, Deverill M: The estimation of a preference-based measure of health from the SF-36. J Health Econ. 2002, 21: 271-292. 10.1016/S0167-6296(01)00130-8.View ArticlePubMedGoogle Scholar
- Jönsson L, Lang A, Lindström E: TOOL: Multi-attribute utility function reflecting patient experience of side effects to antipsychotic therapy. Value Health. 2009, 12: A361-View ArticleGoogle Scholar
- Willke RJ, Burke LB, Erickson P: Measuring treatment impact: a review of patient-reported outcomes and other efficacy endpoints in approved product labels. Control Clin Trials. 2009, 25: 535-552. 10.1016/j.cct.2004.09.003.View ArticleGoogle Scholar
- Barbera M, Sanjuan J, Munarriz M, Barbeŕa M, Sanjuán J, Munárriz M, Novella E, Santiago C, Simó M: Subjective experience with antipsychotics: quantitative evaluation [in Spanish]. Actas Esp Psiquiatr. 2006, 34: 287-294.PubMedGoogle Scholar
- Bobes J, Garcia-Portilla MP, Bascaran MT, Saiz PA, Bousono M: Quality of life in schizophrenic patients. Dialogues Clin Neurosci. 2007, 9: 215-226.PubMed CentralPubMedGoogle Scholar
- McCabe R, Saidi M, Priebe S: Patient-reported outcome in schizophrenia. Br J Psychiatry. 2007, 50: S21-S28. 10.1192/bjp.191.50.s21.View ArticleGoogle Scholar
- Taksh U: A critical review of rating scales in the assessment of movement disorders in schizophrenia. Curr Drug Targets. 2006, 7: 1225-1229. 10.2174/138945006778226589.View ArticlePubMedGoogle Scholar
- Konig HH, Roick C, Angermeyer MC: Validity of the EQ-5 D in assessing and valuing health status in patients with schizophrenic, schizotypal or delusional disorders. Eur Psychiatry. 2007, 22: 177-187. 10.1016/j.eurpsy.2006.08.004.View ArticlePubMedGoogle Scholar
- Hayhurst H, Palmer S, Abbott R, Johnson T, Scott J: Measuring health-related quality of life in bipolar disorder: relationship of the EuroQol (EQ-5D) to condition-specific measures. Qual Life Res. 2006, 15: 1271-1280. 10.1007/s11136-006-0059-z.View ArticlePubMedGoogle Scholar
- Briggs A, Wild D, Lees M, Reaney M, Dursun S, Parry D, Mukherjee J: Impact of schizophrenia and schizophrenia treatment-related adverse events on quality of life: direct utility elicitation. Health Qual Life Outcomes. 2008, 6: 105-10.1186/1477-7525-6-105.PubMed CentralView ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.