Towards an instrument for surveying knowledge management practices

The research was conducted to develop and stabilize a data collection and analysis instrument for an annual survey of knowledge management practices in the South African business sector. From a literature study it was deduced that six factors could be identified with necessary and sufficient reason to be used as main parameters for the assessment instrument. Synthesized a priori and posteriori judgmental knowledge was used to construct a number of aggregational indicators for each factor. Each indicator posited as a statement being a recognized knowledge management practice. A modified six point Likert scale was created to score/indicate gradual progression towards full implementation of a specific practice. Regression factor analysis and one-way analysis of variance was used on the collected data to evaluate the instrument. These analyses indicated that the instrument could detect the expected differences on the demographics and on the KM practices of the survey and could therefore be declared sound and verified. The analyses also revealed several modifications that could be used to improve the instrument.


Introduction
This paper outlines the research methodology used for the 'Development of a SA Knowledge Management Audit Instrument' and includes important statistical outputs from the analyses.The purpose of the paper is to provide interested readers with more background on how the research for the development of the instrument was conducted.It is organized in the following subsections: (1) research objective; (2) literature review and model development; (3) survey instrument development; (4) data collection and sample description; (5) survey data analysis; (6) instrument evaluation/verification; (7) conclusion.

Research objective
The objective of the study was to develop and stabilize a methodology, including a data collection and analysis instrument for an annual audit of knowledge management (KM) practices in the SA business sector.The aim of this paper is to report on the development and evaluation of the audit instrument.

Literature review and model development
By careful examination of the literature on topics related to measures of knowledge management and knowledge management practices, a conceptual framework for the measurement model was construed (Botha & Fouche, 2002).Primary factors representing key organizational variables were identified by using deductive/inductive reasoning, judgement based on grounded management theory, frequency of discussion by notable academics and practitioners, focus of research topics, and qualitative and quantitative emphasis placed on the main factors -the dependent variables.The factors finally selected and judged to be representative of an organisational knowledge ecology were knowledge leadership, organizational culture, structures, processes and routines, information and communication technology and measures.A comparative analysis of similar assessment models, found during the literature examination, was done.The most important in this category was the work done by David Skyrme Associates (Skyrme, 1999) and a joint development by Arthur Andersen (AA) and The American Productivity & Quality Center (RSA, 1996).Skyrme used ten key dimensions of which five, namely, leadership, culture/structure, processes, technology and measures corresponded with the selections for this study.The AA model on the other hand is based on four knowledge management enablers, leadership, culture, technology and measurements, which act in a dynamic relationship with the so called KM processes, namely, create, identify, collect, adapt, organize, apply, and share.
The emphasis in the model, called the knowledge management reference model (KMRM) used for this study is however, on the interrelationship between the four primary factors, culture, structure, processes and technology, which are in continual alignment with leadership and monitored by measures.The description and validation of this model -of which the data model is depicted in Figure 1 below -was presented as a paper at an international conference in Pretoria and subsequently published in the relevant proceedings of the conference (Botha & Fouche, 2002).The rationale for using this model as basis for the development of the instrument and the comparative and definitive discussion of the six constructs (primary factors) are covered in this paper.For each of the six main factors discussed above a number of knowledge management practices deemed to be aggregated indicators -the independent variables -of these factors were selected.Criteria used for this selection were the quantitative and qualitative mention and discussion of these practices by notable practitioners and researchers as contributors and indicators of sustainable KM practices (Allee, 1997;Badaracco, 1991;Skyrme, 2000;Cortada & Woods, 2001;Davenport & Prusak, 1998;Marchand, Davenport & Dickson, 2000;McDermott, 1999;Myers, 1996;Sveiby, 1997;Zack, 1999).Although it was suggested by the literature, the supposition that a causal relationships between the main factors exist was accepted.Similarly a supposition that a specific knowledge management practice can contribute to more than one of the main factors was also postulated.This reasoning was then used to construct the Knowledge Management practices survey instrument.
The set of knowledge management practices for each of the six factors was formulated in the form of statements.Each statement attempts to describe a knowledge management practice, employed by a world class organization in pursuit of sustainable competitive advantage -deemed to be a best practice (Allee, 1997;Badaracco, 1991;Cortada & Woods, 2001;Davenport & Prusak, 1998;Marchand et al., 2000;Myers, 1996;Skyrme, 1999Skyrme, & 2000;;Sveiby, 1997;Zack, 1999).The contents of the statements are analyzedsynthesized a priori judgements made on information from the extant literature and put forward as the most plausible (Kant, 1929).Twenty-four statements -best practiceswere formulated in this way.
A modified Likert -a composite measure with an internal intensity order -ordinal scoring scale was designed to indicate progressive degrees in the state of the implementation of knowledge management practices.This scale guides individuals to assess the present status of a specific KM practice in their organization.The modified Likert scale conforms to the criteria as proposed by Sedlack and Stanley (1992).
The twenty-four statements constituting the knowledge management practices and the selected Likert scoring scale were composed into an audit instrument.The data-model, therefore consist of the six main factors -the dependent variables, each dependent on three or more observable variables -the independent variables, counting as 24 indicators in total.Respondents could score these 24 indicators over a range of six intervals counting from zero to five, where a score of five indicates the highest state of implementation of a KM practice.
Pilot-testing of the model and the audit instrument were conducted at several South African based companies that were reputedly well advanced in the practice of knowledge management, some with considerable global experience.

Data collection and sample description
Data was collected by doing several presentations on the knowledge management reference model (KMRM) to groups of South African company representatives at venues in Cape Town, Johannesburg and Pretoria.After the presentations the attendees were requested to complete the audit instrument.Usable data was received from 83 respondents.At a separate session data were collected from 12 additional respondents from a single company.These targeted respondents (Marchand et al., 2001) were largely from the upper echelons of the companies they represented i.e. from the executive and managerial level.

Aim
The aim of the data analysis was twofold.The primary aim was to verify the audit instrument.The secondary aim was to give a first order presentation of the status of KM practices in the SA industry (Botha & Fouche, 2002).

Principal component analysis
Principal Component Analysis (PCA) is but one method of classical linear Multivariate Analysis (MVA) (Van de Geer, 1993:17), but it is probably the oldest and best known of the techniques of multivariate analysis and extensively used in the behavioural sciences.Although the term 'principal component analysis' is in common usage, other terminology may be encountered for the same technique, for example, 'empirical orthogonal functions', 'factor analysis', 'eigenvector analysis' and 'latent vector analysis' may camouflage PCA.
The central idea of PCA is to reduce the dimensionality of a data set in which there are a large number of interrelated variables, while retaining as much as possible of the variation present in the data set.This reduction is achieved by transforming to a new set of variables, the principal components, which are ordered so that the first few retain most of the variation present in all of the original variables (Jolliffe, 1986).The basic idea of the technique is to describe the set of multivariate data in terms of a set of uncorrelated variables each of which is a particular linear combination of the original variables.The new variables are derived in decreasing order of importance so that, for example, the first principal component accounts for as much as possible of the variation in the original data (Everitt & Dunn, 1991:45).Generally if a set of p (>2) variables has substantial correlation among them, then the first few principal components (PCs) will account for most of the variation in the original variables, but the total variation in the original p variables is only accounted for by all p principal components.Algebraically, the PCA involves finding the eigenvalues and eigenvectors of the sample correlation matrix (Gnanadesikan, 1997: 8).PCA can be based on either, sample correlation or population covariance matrices.For this study the former was used

Combination of scores
When several scores aimed at assessing the same characteristic are available it may be advantageous both for parsimony, and to aid interpretation to combine them into a single score that captures the essential information.Therefore the importance of the first principal component is that it can be used to calculate a single score on any of the six main factors -dependent variables -for a single company, using data collected with the audit instrument on that company.
In the following sections the results of applying this technique to certain groups of scores are presented.

Leadership
The three Leadership (L) scores are quite highly correlated, as can be seen from the correlation coefficients in Table 1.LPr1 can be re-written in terms of the original scores, but for the purpose of this analysis the relative weights of the scores are more meaningful, as shown in Figure 2. Because of the high correlation between L1 and L2 it could be considered to use only one of the scores in a revised instrument.

Culture
The correlation between the scores on Culture (C) is as depicted in Table 2.The high correlation between C4, C8 and C9 indicates possibilities of aggregation into a single indicator.

Structure
The correlation between independent variables for Structure is shown in Table 3.The high correlation between S11 and S12 indicates a possible single indicator.
The first principal component, SPr1 explains 55% of the total variation, the second 16% and the third 14%.
Aggregating the five S scores into a single score is not convincing, but because of a convincing first principal, SPr1 the mean of the five S scores can be used as a single score.Note: The importance of the first principal component is that it can be used to calculate a single score on any of the six main factors for a single company, using data collected with the audit instrument on that company.

Processes
The correlation between independent variables (indicators) for the dependent variable Processes (P) is shown in Table 4: Again the correlation between P16 and P17 is such that a single indicator could be construed to collect data for this score.
The first principal component, PPr1 explains 72% of the total variance and therefore highly convincing.Thus the three scores of the factor P can be aggregated to present as a single score as shown in Figure 5.

Technology
The independent variables correlation for Technology (T) is shown in Table 5.With the very high correlation between T18 and T19 sufficient evidence is revealed to combine the two into one indicator.Therefore it might not be necessary to ask respondents to score on both IS architecture (T18) and IT infrastructure (T19), a single indication on one of the two will be sufficient to score Technology (T).
The first primary component explains 69% of the total variance, proving this measure on technology to be a sound and useful entity of the instrument.The three scores, T18-T19 can again be aggregated to produce a single score for the factor T. Note that from the formula for TPr1 it can be deducted that the contribution of T20 is far less than that of T18 and T19 but still sufficient to warrant its retention as an indicator.

Measures
The correlations: These correlations are in general relatively high; all are statistically significant on the 5% level.This signifies that strong causal relationships between the main factors were detected by the instrument as suggested by the KMRM.This provided further evidence to verify the usefulness of the instrument.The instrument was found to be sound.Some results of the testing pointed to possible modifications that could improve the data collection model.

Conclusion
There is ample evidence from the analyses above that the developed survey instrument is sound and that it could be verified as a useful instrument to conduct audits on knowledge management practices in companies.Subsequent to this research the South African Information and Knowledge Management Tool (SAIKMAT™) was developed and successfully applied by management consultants during KM strategy workshops for organizations like De Beers Industrial Diamonds, the Human Sciences Research Council and the Department of Home Affairs.The validation methodology made it possible to customize the audit instrument to suit specific organizations.
Although much progress has been made during the past decade to develop a philosophy and conceptual frame of KM, the discipline still lacks proven practice.Rigorous measurement and assessment of KM practices should contribute to the development of an empirical base of best practice.Further research should be conducted to explore possible causal correlation between knowledge management practices and organizational performance.

Figure 2 :
Figure 2: Relative weights of the Leadership indicators

Figure 3 :
Figure 3: Relative weights of the Culture indicators