THE EFFECTS OF COMBINED SCALING TECHNIQUES ON THE PSYCHOMETRIC CHARACTERISTICS OF A RATING SCALE
Date of Completion
January 1987
Keywords
Education, Tests and Measurements|Psychology, Psychometrics
Degree
Ph.D.
Abstract
This study examined the psychometric characteristics of several variations of a rating scale that was developed as a part of a selection and promotion project for the Police Department of Peoria, Illinois. The rating scale was specifically for the purpose of rating the incumbent sergeants in the department. The sergeants were to be rated by their supervisors.^ Supervisory ratings are among the most common indices of performance in settings such as this. Since this data is often critical to promotion decisions, psychometrically sound rating instruments are extremely important.^ A combination of rating scale development techniques was used. Characteristics important to the job of sergeant were solicited from the incumbent sergeants and the administrative staff of the department. These characteristics were edited, assigned to a rating instrument and sent back to the sergeants and administrators. They were then asked to rate each characteristic on the continua of both importance to the job and how often the characteristic is needed in the job. The items were arranged by rank order of importance and used as a pool to create two separate rating scales.^ The first scale consisted of the simple assignment of the items to a 52 item Likert type scale. The second scale was created by selecting 23 items that evenly spanned the importance continuum. These scales were then used by supervisors to rate the incumbent sergeants. Both of these scales were scored using a simple summated procedure and using a procedure that attributed weights to each item equal to their median importance ratings.^ Cronbach's Alpha, the skewness index, and validity coefficients were calculated for each variation of the rating scale. (Ratings were correlated with a paper and pencil test developed to measure job knowledge to determine the validity coefficient).^ Results indicate that the short weighted versions of the rating scale provide ratings with significantly superior validity data. ^
Recommended Citation
THOMAS, RICHARD DONALD, "THE EFFECTS OF COMBINED SCALING TECHNIQUES ON THE PSYCHOMETRIC CHARACTERISTICS OF A RATING SCALE" (1987). Doctoral Dissertations. AAI8800246.
https://digitalcommons.lib.uconn.edu/dissertations/AAI8800246