The application of item response theory to employee attitude survey data using Samejima's graded response model

Date of Completion

January 1997

Keywords

Psychology, Experimental|Psychology, Industrial

Degree

Ph.D.

Abstract

This study illustrated the application of IRT methods to employee attitude survey data. Classical test theory results were compared to IRT results to demonstrate the gain in information through IRT methods. Exploratory factor analysis was used to establish the factor structure of the data. Confirmatory factor analysis was then used to examine the factors for violations of unidimensionality. Six unidimensional scales related to employee attitudes were identified for analysis by IRT methods. Samejima's (1969) graded response model was selected for the IRT analysis of the Likert scale items using the software program Multilog (Thissen, 1991). Differential item functioning (DIF) analysis was conducted for the management and non-management subgroups of respondents, following the procedure proposed by Thissen, Steinberg, & Wainer (1993). Results showed the information gained by IRT methods. Illustrations included: variations in precision of measurement across items and scales, high standard errors for the middle response category for some scales, ranking of items according to the information each provided, information gained by an entire scale across the attitude continuum, and differential item functioning results.^ Improvement opportunities were identified in IRT statistical methods: (a) models to fit multidimensional scales with graded responses, (b) practical model fit indices, and (c) guidelines pertaining to: number of items, sample size requirements, and acceptable degree of unidimensional violation. Improvements are also necessary to the IRT tools, especially software programs and documentation, to make these procedures accessible to a wider audience and to avoid errors in the evaluation of results. While widespread use of IRT for employee attitude surveys will await such improvements, it is recommended that these techniques be used, in their current state, for theory building and in practice, where feasible.^ IRT analysis can be applied to employee attitude data with benefits accruing in instrument design, administration, analysis, and reporting. It is important that survey instruments are psychometrically sound, given their widespread use and the trend to integrate such data into the process of continually improving organizations. IRT methods can be a powerful tool toward this end. IRT can also be an aid in theory building for employee attitude constructs. ^

Link to Full Text

Share

COinS