Scales, measures and tests are commonly used in the social and health sciences as research and clinical tools. It is important that these are well designed and are reliable and valid for the purpose intended. This course provides an overview of the scale development process and the statistical techniques used including Reliability analysis, Factor Analysis and Rasch Analysis. Rasch analysis is one of the modern psychometric techniques that form part of Item Response Theory. Using specially designed software (RUMM2030) this technique allows a formal assessment of the measurement properties of scales and tests. It provides detailed feedback on all aspects of the scale including the response format, the fit of the items, item bias, unidimensionality and the spread of items across the trait being measured. Rasch analysis also allows a linear transformation of ordinal scales to interval level measures, permitting the use of parametric statistical techniques and the calculation of change scores.
Topics
• Introduction to measurement and testing
• Theoretical frameworks – Classical Test Theory (CTT), Item Response Theory (IRT)
• Overview of the scale development process
• Development of a test plan, item generation, selection of response format
• Assessing reliability – internal consistency (Cronbach alpha coefficient)
• Assessing dimensionality - Exploratory Factor Analysis
• Rasch Analysis, including:
o Preparing the data file for Rasch analysis
o Evaluating overall model fit
o Checking the response format (thresholds) and rescoring items
o Detecting misfitting items and persons
o Identifying item bias (Differential Item Functioning)
o Assessing the dimensionality and targeting of a scale
o Exporting Rasch scores to SPSS for other analyses
The course takes a practical applied approach (rather than mathematical) and participants are provided with detailed, hands-on experience. SPSS will be used for practical demonstrations, examples and exercises relating to reliability and factor analysis. RUMM2030 is used for the Rasch analysis training and participants will be provided with a limited licence version of the software.
An understanding of elementary statistics, equivalent to the syllabus of ‘Fundamentals of Statistics’ is required. This course is designed for people with little or no experience of scale development.
There is no textbook for this course but participants will be provided with a detailed set of notes. Course participants are encouraged to bring existing scales, questionnaires and/or data sets to class for discussion.