The construct validity of the revised job diagnostic survey

The Job Characteristics Model, the accompanying Job Diagnostic Survey (JDS) and the proposed steps for improving motivation, satisfaction and performance have been functionally utilized in South Africa as a revised job redesign practice to address some critical human resources problems currently facing managers and human resources practitioners. The aim of the study was to determine the construct validity of the Revised Job Diagnostic Survey (JDS) in the South African context. The Revised JDS was evaluated using item analysis and confirmatory factor analysis. The finding of this study is in favour of the Revised JDS as a reliable and factorially valid instrument for South African use. The promising results of this study should pave the way for further research and the search for more conclusive evidence on the construct validity of the Revised JDS in the South African context.


Introduction
Job characteristics play an important role in organizational theory.It makes it possible to identify differences and similarities between jobs and to determine internal work motivation in designing and redesigning jobs (Idaszak & Drasgow, 1987).Hackman and Oldham (1976) have identified and defined the following five job characteristics:

•
Skill Variety: the degree to which the job requires a variety of different activities; • Task Identity: the degree to which the job requires the completion of a whole and identifiable piece of workdoing a job from beginning to end with a visible outcome; • Task Significance: the degree to which the job has a substantial impact on the lives or work of other people; • Autonomy: the degree to which the job provides substantial freedom and discretion to the individual in scheduling the work and in determining the procedures to be used in carrying it out; and • Feedback: the degree to which carrying out the work activities required by a job results in the individual's obtaining direct and clear information about the effectiveness of his or her performance.Hackman and Oldham (1975) developed the Job Diagnostic Survey (JDS) to measure these five core job characteristics.
According to Boonzaaier, Ficker and Rust (2001), the JDS can be used to • diagnose jobs considered for redesign in order to establish the current potential of a job for enhancing motivation and satisfaction; • identify those specific job characteristics that are most in need of enrichment; and • assess the 'readiness' of employees to respond positively to improved jobs.
The model, the JDS and the proffered action steps facilitate a process by means of which managers can practically achieve an optimal fit between workers and their jobs.They can do so by addressing motivation, satisfaction and performance problems caused primarily by shortcomings in the nature of jobs (Boonzaaier et al., 2001).
However, several inconsistencies have been identified in the JDS's factor structure. Lee and Klein (1982) reported that employees at different job levels understand the complex format of some of the JDS items differently.Lee and Klein (1982) systematically compared the factor structure across occupational categories and found that the JDS structures were not invariant across occupational groups.Fried and Ferries (1986) also reported inconsistencies, and they suggested that the respondents' age, level of education and position level can influence the factor structure.Hackman and Oldham (1975) deliberately included a few reverse scored items to minimize response bias.However, Harvey, Billings and Nilan (1985) recommended that the reversed items should be rewritten and reversed.Idaszak and Drasgow (1987) therefore rewrote the reverse score items.This resulted in a five-factor solution that closely reflected Hackman and Oldham's (1975;1976) theory.Idaszak and Drasgow (1987) concluded that the new JDS items had substantially improved measurement properties and that these new scales should be used in future research.Renn and Swiercz (1993) examined the Revised JDS.They found that the revised scales provided adequate measures for the five core job characteristics, but that more construct valid items should be added.The expected result would be a more stable factor structure in the JDS.They concluded that the Revised JDS could now be used with more confidence.Kulik, Oldham and Langer (1988) compared the original JDS with the Revised JDS.They believed that Idaszak and Drasgow's (1987) recommendations were somewhat premature, due to the following limitations: the two JDS's were not compared with each other and it was unknown what impact the revision had on the criterion-related validity of the Revised JDS.The results of the Kulik et al. (1988) study suggested that the revised items provided purer measures of variety, significance and identity than the original items, but did not substantially improve the measurement of autonomy and feedback.Their results were generally consistent with those of Harvey et al. (1985) in showing that the negatively worded items might have been the cause of inconsistency in the JDS.They suggested, moreover, that further research should focus on developing new items to improve the JDS factor structure, rather than on using the Revised JDS.
Due to the inconsistent results regarding the Revised JDS, it needed to be assessed with various samples from several groups to enable its measurement properties to be properly understood.
Idaszak, Bottom and Drasgow (1987) addressed the measurement equivalence issue and the methodological problem by means of factor analysis of the original JDS and the Revised JDS.They found that the methodological problems were the primary cause of the inconsistencies obtained in the factor analysis of the JDS.Several points that needed to be looked at were the following: the data sampled across a representative set of jobs should have at least four to six items per scale; sampling fluctuations seem to play an unacceptably important role in samples of several hundred individuals when only three items are used to assess each scale.The JDS's inconsistent factor structures may also be due to the fact that the JDS is sometimes used to assess individual differences in perceptions within a single type of job and, at other times, it is used to assess perceptions of job characteristics among different types of jobs.Although the instrument might show different psychometric properties in these different situations, it has proven to be useful in job design research (Taber & Taylor, 1990).Boonzaaier and Boonzaaier (1993) researched the JDS in a South African context.They obtained results that show that the JDS is applicable in South African organizations and that it satisfies the basic requirements of reliability and validity for use.However, they did only study the relationship between job characteristics, motivation and satisfaction on the job.
The purpose of this study is to examine the construct validity of the Revised JDS on a South African sample.

Sample
A convenience sample of 677 respondents from different organisations completed the Revised JDS.The biographical information of the respondents is set out in Table 1.The sample consisted of Black (n=180) and White (n=486) respondents.
Approximately 49% of the respondents indicated that Afrikaans was their first language and 23% indicated that English was their first language.Of the respondents, 27% indicated that an African language was their first language, and 1% of the respondents spoke European languages other than English as their first language.

Measuring instrument
The Revised JDS was used to measure the five core job dimensions (Skill Variety, Task Identity, Task Significance, Autonomy, and Feedback).Three seven-point Likert-type items measured each core dimension.Table 2 sets out these variables per factor.In previous studies the reliability of the sub-scales has often been found acceptable (Cook, Hepworth, Wall & Warr, 1981).

Statistical analysis
Descriptive statistics in respect of the Revised JDS scale items were calculated.The initial analysis involved the generation of item statistics.Means, standard deviations, skewness, kurtosis, item-total correlations and coefficient alphas (if the item would be deleted) were calculated to provide an indication of the item quality.These statistics gave an initial indication of the appropriateness of the subsequent analysis procedures.Harvey et al. (1985) proposed that using confirmatory factor analysis allows researchers to formulate and directly test competing hypotheses regarding the underlying factor structure.They based their arguments on the following limitations of exploratory studies -if the predicted factor solution is not found, two interpretations are possible: first, there are true differences between the actual and predicted JDS factor structure, or, second, the exploratory factor analysis was unable to uncover the hypothesized factor structure.The most troublesome difficulty with exploratory factor analysis is selecting the appropriate underlying dimensionality.After considering Harvey et al. (1985)'s arguments, the authors decided to perform only a confirmatory factor analysis for the purposes of this study.Maximum likelihood estimation was used, employing EQS 6.1 for Windows.The Bentler-Bonnett non-normed fit index (NNFI), the Comparative Fit Index (CFI), the Bollen Non-normed Fit index (IFI) were included as these indices are less sensitive to the effect of sample size In addition the Root Mean Squared Error of Approximation (RMSEA), and Sattorra-Bentler Scaled Chi-Square were used as model fit indices (Kelloway, 1998;Medskar, Williams & Holahan, 1994).

Results
The descriptive statistics for the Revised JDS scales are set out in Table 3.The means, standard deviation, skewness and kurtosis reflect how the participants responded to the different scales.The results indicate that the data that were collected and analysed have small but noticeable deviations from the normal distribution.Marda's coefficient for multivariate kurtosis was significant and therefore the normality assumption that applies for multivariate analyses had been violated.
Item analyses for the five subscales of the Revised JDS are set out in Table 4. Kline (1986) noted that items should ideally correlate beyond 0,2 with the total score.In this case, all items correlate above 0,2, with a lowest correlation of 0,3.All items appear to contribute positively to scale reliability except for Item 3 that impact negatively on the reliability of the Skill Variety scale.The alpha coefficients for the sub-scales ranged between 0,67 and 0,79.Nunnaly (1978) has indicated 0,7 to be and acceptable reliability coefficient, but according to Santos (1999) and Morgan and Griego (1998) lower thresholds are sometimes used in the literature, especially if there are only a small number of items in a scale.A confirmatory factor analysis was done on the group to determine whether the Five-factor model of the JDS fitted the data.The structural equation model for the five domains underlying the Revised JDS for the group is set out in Figure 2. The latent variables have been allowed to correlate with one another.As indicated earlier, the assumption of multivariate normality had been violated and consequently the distribution free Robust Method of parameter estimation was used.As indicated in Table 6, the CFI, NNFI and IFI values were 0,919, 0,894 and 0,920 respectively.A value of 0,90 is considered to be a good fit for all the above fit indices (Bentler, 1990;Bentler & Bonnett, 1980;Steiger, 1995).In this study, the RMSEA had a value of 0,059 (90% confidence interval 0,051-0,067.Steiger (1995) considers RMSEA values less than 0,10 as an acceptable model fit.
The chi-square was 266,076 on 80 degrees of freedom (p = 0,000) for the total group.The chi-square/degrees of freedom ratio was 3,326.Carmines and McIver (1981) claim that the relative chi-square should be in the 2:1 to 3:1 range for an acceptable model.However, ratios between two and five have also been accepted.Given the large sample size, it would be incorrect to conclude a poor fit based on the significance of the chi-square indices.Thus the chi-square ratio of 3,326 can be interpreted as indicating a good fit (Kelloway, 1998).The correlations between the latent variables suggest that the five-factors are strongly interrelated.The covariance structure suggests that V3 (Item 3) has a relatively low correlation with the latent variable 'Skill Variety' compared to the correlations with latent variables obtained on the remaining items.

Discussion
The Job Characteristics Model and the JDS had been used in South Africa for years without any sound evidence on its construct validity in the South African context.Construct validity refers to the degree to which an instrument measures what it purports to be measuring.
The confirmatory factor analysis results confirmed the factor validity of the five-factor measurement model of the JDS for the sample group.Fit indices of the sample confirmed that the JDS measurement model does fit the data to an acceptable degree.
According to the results of the reliability analysis, the reliability of the instrument is acceptable (Byrne, 2001).The alpha coefficients for the sub-scales ranged between 0,67 and 0,79.The reliability results correlate with previous research (Johns, Xie & Fang, 1992) conducted with this instrument, thus confirming that the measure of the instrument is reasonably reliable.
These results are important to construct validation, as an instrument cannot be deemed valid if it is not reliable.Any threat to an instrument's reliability poses a threat to its construct validity (Brown, 2000).
All items on the JDS appear to function at an acceptable level except for Item 3 that form part of the Skill Variety scale.There is reason for concern that this item may not be suitable for the South African context and should possibly be revised.
The study provided preliminary evidence that the Revised JDS may be of use to managers in South Africa as an instrument for job redesign and to address critical human resources problems facing managers and human resources practitioners.It should be emphasized that additional studies need to be conducted before final conclusions can be reached on the construct validity of the JDS in the South African context.Taber and Taylor (1990) indicated that research studies conducted thus far on the JDS were inconsistent in terms of factor structures obtained and therefore further research on the instrument is required.Not withstanding the problems that have been experienced in respect of the factor stability of the JDS, Taber and Taylor (1990) indicated that the JDS has proven to be useful in job design research and should be explored further.Some limitations and recommendations must be kept in mind concerning the findings of the study:

•
From a survey research perspective, it should be noted that the researcher was not present when the respondents completed the questionnaires.Hence respondents would not have been able to request clarification of the questions from the researcher and this could have had an impact on the results.

•
The Revised JDS is a relatively short survey, with only 15 items for five underlying constructs.According to Idaszak et al. (1987) four to six items per factor would reflect the true underlying structure more clearly.

•
The alpha coefficients for the sub-scales ranged between 0,67 and 0,79, which is generally good.According to Byrne (2001), indicators should have a Cronbach alpha of 0,7 for the set to be judged reliable.The alpha may be low because of a lack of homogeneity of variances among the items; and it is also lower when there are fewer items in the factor.

•
The chi-square fit index is very sensitive to sample size.The larger the sample size, the more likely a Type II error (rejecting something true) is.Given the large sample size (n=677), it would be incorrect to conclude a poor fit based on the significance of the chi-square indices (Kelloway, 1998).
Item 3 should possibly be revised depending on the results of further research.