Conundrum of Kappa and Why Some Musculoskeletal Tests Appear Unreliable Despite High Agreement: A Comparison of Cohen Kappa and Gwet AC to Assess Observer Agreement When Using Nominal and Ordinal Data.

PHYSICAL therapy assessment; STATISTICS; RESEARCH evaluation; CONFIDENCE intervals; INTER-observer reliability; COMPARATIVE studies; DESCRIPTIVE statistics; DIAGNOSIS of musculoskeletal system diseases; STATISTICAL models; DATA analysis software; EVALUATION

In clinical practice, physical therapists often use different kinds of tests and measures in the assessment of their patients. For therapists to have confidence when using their tests and measures, an important attribute is having intratester and intertester reliability. Studies that assess reliability are cases of observer agreement. Many studies have been performed assessing observer agreement in the physical therapy literature. The most commonly used method to assess observer agreement studies that use nominal or ordinal data is the statistical method suggested by Cohen and the corresponding reliability coefficient, Cohen kappa. Recently, Cohen kappa has undergone scrutiny because of what is called kappa paradox, which occurs when observer agreement is high but the resulting kappa value is low. Another paradox also occurs when asymmetries exist between raters on their disagreements, resulting in a higher kappa value. In the physical therapy literature, there are numerous examples of this problem, which can often lead to misunderstanding the meaning of the data. This Perspective examines how and why these problems occur and suggests an alternative method for assessing observer agreement. [ABSTRACT FROM AUTHOR]

Copyright of PTJ: Physical Therapy & Rehabilitation Journal is the property of Oxford University Press / USA and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)