ISSN Print 2500–1094
ISSN Online 2542–1204
Bulletin of RSMU
BIOMEDICAL JOURNAL OF PIROGOV UNIVERSITY (MOSCOW, RUSSIA)
ORIGINAL RESEARCH
MFTS and AQSA scales validation in patients with multiple and concomitant foot fractures
About authors
Department of Traumatology, Orthopedics and Field Surgery, Faculty of Pediatrics,
Pirogov Russian National Research Medical University, Moscow
Correspondence should be addressed: Maksim Korolev
Leninskiy prospect, d. 10, korp. 7, Moscow, Russia, 117049; ur.liam@ffelorokrotcod
Received: 2016-03-30
Accepted: 2016-04-07
Published online: 2017-01-05
|
Modern requirements on provision of medical care to patients with severe injuries are now compelling physicians to focus not just on early diagnosis, active treatment and subsequent rehabilitation of the patient, but also on the cost-effectiveness of procedures, reduction of treatment and rehabilitation duration and reduction in disability. In addition to objective difficulties, such as short duration of stay by the patient at the hospital, lack of medical history data and general serious condition, diagnosis and treatment of concomitant and multiple injuries carry considerable financial and material costs for clinics and health insurance funds. This underlines the importance of objective evaluation of efficacy of treatment using rating scales.
There are so many scales and assessment questionnaires. There are non-specific scales (general assessment: VAS, NRS, SF-36, etc.) and specific scales (for a certain anomaly: AOFAS, FFI, DASH, etc.), which can be characterized by validity, compliance (friendliness), reliability, reproducibility of results in subsequent studies, and sensitivity to objective changes in indicators. A particular assessment tool for a specific group of patients is chosen based on these parameters [1, 2, 3].
However, the existing scales are imperfect. There is need to create new questionnaires for assessment of treatment results and correctly interpret the results; validation of these questionnaires is also required [4, 5, 6, 7]. We developed two new scales at the Department of Traumatology, Orthopedics and Field Surgery, Faculty of Pediatrics, Pirogov Russian National Research Medical University.
Moscow Foot Trauma Scale (MFTS) is a specific scale for evaluation of treatment outcome after foot injury. It consists of subjective and objective parts, each of which includes 3 multiple-choice questions (fig. 1). At the end of the scale are keys indicating the number of points for each answer. Possible score ranges from 0 to 90. Possible treatment effectiveness assessment results: 90–61 p — excellent; 60–41 p — good; 40–21 p — satisfactory; 20–11 p — bad; 10–0 p — very bad.
Abbreviated Questionnaire of Subjective Assessment (AQSA) is a non-specific scale that can be used to assess whether a patient has limited activity, whether he needs special orthopedic shoes or additional support, whether his pain syndrome needs to be relieved using analgesics, and to assess the change in the nature of work performed. The Questionnaire includes 6 questions (fig. 2). At the end of the scale are keys that indicate the number of points for each answer. The possible score ranges from 0 to 30. The higher the score, the worse the result: 0–10 p — good; 11–20 p — satisfactory, above 20 p — bad.
Before using the newly developed scale, it is necessary to confirm its theoretical and pragmatic validity in the conditions of use. Confirmation of theoretical validity enables to establish whether this scale indeed assesses the indicator needed by us, while confirmation of pragmatic validity allows to determine how well does the scale perform its function in practice when dealing with patients. Validity is interpreted in different ways, depending on the task. Validity usually refers to the degree of confidence at which a test, measurement or experiment actually performs the function for which it is intended [8].
Checking the validity of the new assessment tool is quite a challenging task. In traumatology, the SF-36 (Short Form 36) and AOFAS (American Orthopaedic Foot and Ankle Society Score) scales are most commonly used to verify theoretical validity. These scales have confirmed their stability in population-based studies on large and relatively homogeneous samples [9, 10, 11, 12, 13].
During analysis, correlation between the attributes assessed by the scale and similar attributes of the reference scale should be revealed, and there should be no correlation with symptoms that have other theoretical grounds. Meeting these conditions means that the scope of the scale has been chosen correctly.
Pragmatic validity was evaluated by an external attribute, which should be relevant (i.e. correspond to the test attribute by meaning), free from interference (this is usually ensured by formation of a fairly homogeneous sample) and reliable [14].
Before validity checking, it is required to establish the level of system reliability. Reliability is a relative constancy, stability, consistency of test results in initial and repeated use on the same group of patients. Gurevich recommends interpreting reliability as: 1) reliability of the measuring instrument itself; 2) stability of the test attribute; 3) constancy, i.e., relative independence of results from the experimenter’s identity. He proposes calculating three corresponding coefficients: reliability, stability and constancy [15].
Reliability coefficient was calculated by split-half reliability method, in which the test was divided into equal parts and the correlation between their values calculated. The technique is considered reliable if the correlation coefficient is above 0.75.
The stability coefficient of the attribute under study is determined through test-retest. Its meaning lies in the re- testing of the group being investigated using the methodology under study. The correlation coefficient between the initial and repeated test characterizes the stability of the attribute.
Constancy is checked by the testing of one test group based on one technique but by different researchers. The correlation coefficient should be greater than 0.80 provided the same conditions apply.
Apart from calculation of these coefficients, it is possible to assess the reliability of the system by equivalent-form technique, which requires creation of similar forms of one test and testing that test on a large group of patients with correlation calculated. We didn’t use it due to large labor input and relatively small sample involved.
In addition to calculating the above indicators, the coefficient of internal consistency (Cronbach’s alpha, αK), split- half correlation coefficient (Guttman’s lambda, ʎG) and Kuder– Richardson coefficient (KR20) should be calculated [16, 17, 18].
The aim of the study was to test the validity of the MFTS and AQSA scales for patients with multiple and concomitant foot fractures.
METHODS
The study included 79 patients (59 men and 20 women; mean age — 42 years) at the trauma unit of Pirogov City Clinical Hospital No 1. The patients were treated for multiple and concomitant foot fractures in 2007–2016. Surgical treatment was carried out in 32 patients, conservative treatment in 47. The study was approved by the Ethics Committee of Pirogov Russian National Research Medical University (Minutes No 139 of 10 November 2014). All the patients gave informed consent to participate in the study.
By localization, right foot fractures occurred more frequently (n = 39) than left foot fractures (n = 27) and were bilateral in 13 patients. Initial examination revealed 69 fractures, which is 54.3 % of the total number of fractures. Subsequent examinations by traumatologists and other specialists detected other 24 fractures (26 %). Another 28 fractures were late-diagnosed and 12 of these cases needed surgical treatment. The more severe a patient’s condition was, the higher the likelihood of diagnosing the fractures late. In severe condition assessed on the ISS scale (Injury Severity Score), 11 and 17 fractures were undiagnosed for scores less than 16 p and above 16 p respectively. Before injury, 55 people were able to work fully. After treatment, 37 patients retained their working ability.
After multiple-stage or single-stage treatment, the patients were discharged from the hospital and observed at an injury care center near where they lived. When necessary, the patients were sent to major hospitals for consultations.
The patients were divided into two groups: group of patients with retrospective observation (n = 36) and group with prospective observation (n = 43). The treatment results were evaluated 1, 3, 6 and 12 months after injury or the treatment was repeated using the developed scales MFTS and AQSA and scales SF-36 and AOFAS. The coefficients of reliability, stability, constancy, αK, ʎG and the intraclass correlation coefficient were calculated for all the scales. The Kuder–Richardson coefficient was not calculated due to the non-dichotomous nature of all the scales. While using the test-retest approach, repeated testing was performed after 11 ± 3.2 days.
These numerical values of treatment results for patients with foot fractures showed the importance of careful attention to diagnosis and treatment of foot bones, as well as timely and thorough examination of the patient with further treatment tactics.
The Statistica 10.0 software (StatSoft, USA) was used for statistical data analysis. With a relatively small sample, a significance level of p ≤ 0.05 was taken. For data analysis, non- parametric statistics methods were used due to the presence of data distribution in most cases that is different from normal distribution.
RESULTS
Tab. 1 presents the treatment effectiveness assessment results after 12 months using reference scales and the studied scales as an example of the use of scales. Like assessment using the SF-36 and AOFAS scales, the MFTS and AQSA assessments confirmed a pattern: the later the fractures are diagnosed, the lower the treatment effectiveness.
The MFTS scale recorded the highest Cronbach’s coefficient that indicates the internal consistency of a scale, while the physical component summary of the SF-36 scale yielded the biggest Guttman coefficient (PCS) (tab. 2). The ʎG value was also high in MFTS.
For the MFTS scale, in different periods of testing, the values of the constancy coefficient ranged from 0.81 to 0.93, for the AQSA scale — from 0.57 to 0.69.
The intraclass correlation coefficient defined through the test-retest approach was equal to 0.85–0.96 for MFTS and 0.76–0.93 for AQSA. For both scales, p ≤ 0.05. This result indicates there is high dependency of indicators within the MFTS and AQSA scales.
Tab. 3 shows convergent correlations between the reference scales and studied scales for both groups of patients. As can be seen, they have low level of significance. Correlation between MFTS and AOFAS and correlation between MFTS and the physical component summary of the SF-36 scale were identified, which is logical, given the one-way specialization of these questionnaires. However, all the values had a low level of significance. Therefore, the existence of a real relationship between the scales can only be assumed.
In view of its low specificity, AQSA correlated with both physical and mental components of the SF-36 scale. It correlated negatively with AOFAS.
DISCUSSION
The high Cronbach’s coefficient recorded in the MFTS scale indicates there is optimal construction of questions in the scale. With the help of the Statistica 10.0 software, the need to exclude similar questions was assessed. For the MFTS scale, it was recommended to remove 2 questions (in order to reduce the value of αK). For the AOFAS and AQSA scales, it was recommended to add from 1 to 3 questions to improve the internal consistency of the evaluating tool.
Guttman coefficient confirmed the effectiveness of assessment with the use of physical component summary of the SF-36 scale (0.986). The high values of the test-retest indicator — 0.85–0.96 for MFTS and up to 0.93 for AQSA — reflected the good reproducibility of these scales.
The data obtained indicate there is a fairly high level of individual validity of the MFTS and AQSA scales. However, the convergent validity values were lower than those of the reference scales. One cannot tell exactly what will be the correlation between the indicators under the conditions of another experiment, but it is important that a relationship is still there. Perhaps in future studies involving different groups of patients, we will be able to confirm that there is a relationship.
CONCLUSIONS
Statistical data analysis confirmed that the MFTS scale has a high validity and compliance for the doctor, and high sensitivity and reliability. The disadvantages are average reproducibility and low compliance for the patient. The AQSA scale showed high reliability, reproducibility and compliance for the doctor and patient, but low validity and sensitivity. The convergent validity values of these scales with the SF-36 and AOFAS scales showed there is a weak correlation between the scales.
The MFTS and AQSA scales can be used to assess the effectiveness of treatment of patients with multiple and concomitant foot fractures. In this case, the peculiarities indicated for them should be taken into account.