An investigation of the efficiency of South Africa ’ s sector education and training authorities ( SETA ’ s )

The performance of South African Sector Education and Training Authorities (SETA’s) has been increasingly questioned. On this premise, the paper investigated the efficiency of the SETAs with respect to their utilization of funds in order to promote a range of education and training outputs was investigated. More specifically, the study investigated the quantity and quality of five training and education outputs, set by the National Skills Development Strategy (NSDS), in relation to the funding received. Furthermore, the study examined the amount of money spend on administrative expenditure by the various SETAs, as well as the SETAs management of financial reserves. In order to guide the study, as well as analyze the data, a conceptual framework to measure efficiency was based on an inputoutput model developed by Gupta and Verhoeven (2001). Data were obtained from the published accounting and annual reports for the period 2006 – 2009. The results indicated only two of the SETA’s were efficient with respect to their utilization of funds and that only five SETA’s consistently met their own targets. The study also shows that if the SETA’ s funds had been applied to education and training outputs, rather than for investment purposes, training outputs could have been considerably increased. The paper has implications for the use of public funds with respect to the critical skills shortage confronting the economy


Introduction
The success record of skills development proposals in South Africa since 2002 has been mixed (Prinsloo & Lategan, 2005).In many instances, moreover, there is an absence of coordination between various skills development initiatives.In this regard, the efficiency and alignment of South African Sector Education and Training Authorities (thereafter SETA's) operations has been increasingly questioned since 2008 (Anonymous, 2009a;Mahlong, 2009).Despite the fact that the SETA sector has been operational since 2000, South Africa still faces a major skills shortage.Grant Thornton's 2009 International Business Report, for example, stated that 41% of South Africa's privately held businesses cite the availability of a skilled workforce as the biggest constraint to growth.Some of the skill shortages in South Africa include engineers, technologists, accountants, artisans and a wide range of other technical skills (Webster, 2010;Garrun, 2009;SAICA, 2008;Letsoalo, 2007a;Letsoalo, 2007b;The Department of Labour, 2007).
Further criticism of the SETA sector's performance indicates that in 2004, for example, only 14% of the 70 000 registered SETA learners had completed their courses since the system was implemented.In 2007, only 19% of the 87 687 registered learners completed their training between 2005 and 2007 (Mahlong, 2009).However, other research indicates that the SETA sector is on a positive trajectory but qualify this observation by stating that many problems still persist and that the sector should undertake a more defined set of key responsibilities (Marock, Harrison-Train, Soobrayan & Gunthorpe, 2008).There is also evidence that there is a wide disparity in the performance of the SETA's with some performing exceptionally well and others equally badly (Marock et al., 2008;Webster, 2010).Finally, there has been a deluge of criticism in the media with respect to the mismanagement of public funds, in particular, with respect to monies not spent to provide services or diverted for other purposes (Hamlyn, 2007;Boyle, 2009;Blain, 2009;Anonymous, 2010c).
The objective of this paper is investigate the efficiency of the SETA sector with respect to the efficiency of their utilization of funds to promote a range of education and training outputs.More specifically, the study will evaluate the quantity and quality of five training and education outputs in relation to the funding received, as well as assess whether each SETA achieved its own targets.Finally, the paper will also comment on the SETA management of cash reserves.While there is literature on the operation and performance of SETA's, there is a dearth of in-depth research on their performance and a lack of application of a conceptual framework for examining efficiency.The study contributes to the performance measurement domain in the public sector by developing a conceptual framework to complement the work of Lee (2003) who suggested the use of decision theoretical utility analysis as a method to estimate SETA productivity.A limitation of our study, however, is that the interpretation of SETA performance on an input-output basis that ignores the quality of training provided.
The outline of the balance of the report is as follows: Section 2 discusses the creation and background of SETA institutions.Section 3 develops a conceptual framework to measure the efficiency of government entities.Section 4 outlines the data and method and Section 5 examines the results.Finally, Section 6 reaches a conclusion and some recommendations are made.

Sector education and training authorities (SETA)
SETA's were established by section 9(1) of the Skills Development Act No. 97 (1998) and came into operation from 1 April 2000 (The Department of Labour, 2005).In this regard, the Department of Labour established 25 SETA's whose members include employers, trade unions, government, interested professional bodies and bargaining councils (Steyn, 2004).SETA's are responsible for the disbursement of training levies payable by all employers and are required to develop and implement an appropriate skills development plan for their sector.All SETA's have five principal objectives.The first is to prioritize critical skills for growth, development and equity.The second objective is to stimulate quality training for all in the workplace.The third objective is to promote employability and sustainable development through skills development.The fourth objective is to assist new entrants into the labour market and self employment.The fifth objective is to improve the quality and relevance of training and learning provisions.In particular, a crucial role of these organizations is to assist government implement the National Skills Development Strategy.Finally, SETA's are required to ensure that all training interventions adhere to the standards set out by the National Qualifications.(Skills Development Act No. 97 (1998)).
SETA funding was previously received from the Department of Labour (from 1 November 2009, the control of the SETA's was taken over by the Department of Education, Anonymous 2009b) via the allocation of 80% of tax levy monies received from the respective sectors.In this regard, 10% is allocated for administration expenses and 70% to promote sector education and training programs.A proportion of these funds (50%) are applied to mandatory grants with the balance (20%) available for discretionary funds which can be used for projects designed to assist in the achievement of sector priorities, including the design and implementation of "learnerships".Currently, the SETA's receive an annual budget of R5 billion to address the skills shortage in the economy (Boyle, 2009).The SETA's each renew a service level agreement on an annual basis (s10A of the Skills Development Act).This concerns the performance of the SETA's functions in terms of the Skills Development Act, the SETA' s annual business plan and any assistance that the Director-General is to provide in order to enable the SETA to perform its functions.

Conceptual framework
Performance measurement in the public sector is complex and there has been much debate as to whether private sector practices can be successfully implemented (McAdam, Hazlett & Casey, 2005).The difficulty of developing performance measurement frameworks for the public sector has been complicated by the need to service the needs of a wide range of stakeholders that include various industry sectors and society, as well as motivate operations at business unit level (Neely, 2005;Johanson, Skoog, Backlund & Almquist, 2006;Johnsen, 2001;McAdam, Hazlett & Casey, 2005).Because of the need to reconcile the interests of a broad range of stakeholders, public sector performance measurement frameworks (PMF) are often compromised by an overload of performance measures (Brignall, 2002;Wisniewski & Steward, 2004;McAdam et al., 2005;Chang, 2007).In this regard, several approaches for measuring the efficiency of government expenditure have been attempted, however, these approaches do not allow for easy comparison or the use of simple proxies to gauge efficiency.
A selective overview of studies examining the efficiency of government expenditure has been provided by Gupta and Verhoeven (2001) who indicate four principal approaches to measure government efficiency.Firstly, some studies have concentrated on gauging and enhancing efficiency in practical applications, often focusing on certain types of government spending in a specific country.Secondly, the efficiency of governments has been addressed in quantitative terms, using data on inputs of government spending but not on outputs.Thirdly, the efficiency of public spending has been based on using outputs but not inputs.Fourthly, a combination of both inputs and outputs has been used to assess efficiency.Finally, it has been proposed that the efficiency of Government spending can be further assessed by comparing the outputs, being the goods or services produced by the government, with the targets or goals that the government entity set with respect to the outputs (Scott, 1996).Gupta and Verhoeven (2001) state that the best way to assess government efficiently is on the basis of both inputs and outputs.The present study, therefore, concentrates on both the inputs and outputs of each SETA.The study primarily addresses the question of whether the same level of output could be achieved with less input-or, equivalently, whether more output could be generated with the same level of input (Gupta & Verhoeven, 2001).With this in mind, the measures adopted in the SETA's case are based on comparisons or inputs (revenue) with measureable outputs (in objectives).

Data and method
One of the purposes of a SETA is to fully utilize its resources to promote training and education outcomes.In this regard, it has been assumed that a principal objective of SETA's is not to (unduly) increase its financial reserves but to utilize these for skills development.We have assumed, therefore, that SETA's that unduly increase their balance sheet reserves are not acting according to their mandate and that an undue increase in cash reserves reflects inefficiency.
The assumption that government departments do not unduly increase their cash reserves is supported in many past studies (Gupta & Verhoeven, 2001).
The data for the study were both cross sectional and time series (panel data) largely of a numerical nature and were obtained from the published financial statements and annual reports of 21 SETA' s for the period 2005 to 2009 (the reports of two SETA's could not be obtained).The data included a record of all income, expenditure and certain items from the statement of financial position.Furthermore, the data included a record of all the training and education outputs that had been achieved for each of the five objectives for the four year period.The objectives as listed in the annual reports were also captured and (where provided) costs were directly assigned to the objectives.The annual report was also analysed to capture information on each SETA's targets within the objectives, and whether these targets had been met over the four year period.All the data was captured on Excel spreadsheets in order for analysis purposes.
The data were analyzed as follows: Firstly, two measures of efficiency were developed.The first measure was based on the conceptual framework of the Gupta and Verhoeven (2001) model.In this regard, the outputs per objective were recorded for each year and compared to each other on a percentage basis with 2005 being regarded as the base level.A percentage growth/reduction was then determined for each output for the three following years.Similarly, the input, namely, total revenue as reflected on the income statement, was compared over the four year period.Growth in the Input over the four year period was initially calculated by comparing the 2005/2006 (revenue) amount with the 2008/2009 figure.This growth (or reduction) was then compared to the list of outputs within each objective.If the percentage growth in output exceeded the percentage growth of the revenue input, this was considered "efficient", however if the output was less than the revenue input this was considered "inefficient" (Gupta & Verhoeven, 2001).
The strength of the relationships between the efficiency rating and the outputs, namely, the five objectives of each SETA was then estimated using two longitudinal models, namely, the fixed effects and random effects models.The suitability of the Random Effects model (more robust) was then determined using the Hausman Test (Hausman, 1978) to ensure if there were significant differences in the coefficients.Because the level of significance was slightly in excess of 10% the Random Effects Model was adopted.The statistical analysis and tests was performed using Stata 10 software.
The efficiency of SETA operations was also evaluated from a cash management perspective.The cash reserves were analyzed over the four year period to see how much they had increased, or decreased.According to the assumption made by this study, an increase in cash reserves reflects inefficiency (Gupta & Verhoeven, 2001).

Results and discussion
The results first present the efficiency measures of the SETA sector before further analysis that presents the statistical significance of the relationships between the efficiency ratings of the SETA' s (the outcome variable) and the achievement of objectives (predictor variables).Finally, the cash management performance of the SETA's are also presented in this section.

The efficiency ratings of the SETA sector
The efficiency of each SETA is illustrated in Table 1.The efficiency ratings were based on the input/output model developed as a conceptual framework, as well as whether they achieved their own targets with respect to a series of five outputs.In order to compare the results with the findings of the Marock et al. (2008) report, the efficiencies of the 21 SETA's were ranked.In order to do this, the scores for each efficiency were aggregated with a 2:1 weighting ratio in favor of the input-output model.
Table 1 shows that only one SETA (FASSET) was efficient in all five objectives based on the computation developed for the input-output model.That is, it was only FASSET that showed greater increases in outputs compared to total revenue input for all five objectives.Conversely, five SETA's (ISETT, ETDP, CETA, CHIETA and SASSETA) were efficient in only one out of the five objectives.Table 1 also shows that six SETAs met all five objective targets (FASSET, FOODBEV, FIETA, BANKSETA, ISETT and ETDP), while seven SETA's (CTFL, MAPPP, LGSETA, THETA, AGRISETA, CETA and SASSETA) met only two objectives.Finally the table shows the highest ranking SETA was FASSET with an efficiency measure of 15, followed by FOODBEV with a score of 13.Two SETAs scored the lowest (4), being CETA and SASSETA.
The efficiency ranking of the top performing SETAs concurs, to some degree, with the ranking of Marock et al. (2008) who also ranked FASSET, FOODBEV, CFTL, SERVICES SETA, MQA and BANKSETA amongst their top performers (see Table 1 column "DPRU rating efficiency").Conversely, the present study ranked CHIETA in the third last position, yet the Marock et al. (2008) gave this SETA an 84% efficiency rating.
The present study found the worst performing SETAs were CETA and SASSETA.Again, the Marock et al. (2008) study concurs with CETA, and there is some difference in SASSETA.Interestingly, the Marock et al. report ranks ISETT as its worst performer yet the present study ranks this SETA in midrange.In this regard, the SETA achieved 1/5 for input output model and 5/5 for achieving all of its own targets.An explanation may be that this SETA (ISETT) could have set inappropriately easy targets thus inflating its efficiency.Interestingly, ETDP was also efficient in one objective for the first efficiency measure, while it met all five objective targets, and this would seem to suggest that the targets were easy to obtain.

Random-effects GLS regression
Both Fixed and Random Effects GLS regression models were investigated.The results, listed below, suggest Objective 3 was the only training variable that was significantly linked to the outcome efficiency variable (5% level).
Group variable: SETA2 R-sq: within = 0,0507 between = 0,3884 overall = 0,2520 Random effects u_i ~ Gaussian corr(u_i, X) = 0 (assumed) The administration of cash reserves Analysis was then conducted on the cash reserves of each SETA.This analysis, illustrated in Table 2 presents cash reserves in 2006 and compares this with the cash reserves 2009 to work out the percentage increase.
The results in Table 2 show that 18 of the 21 SETA's had increased their cash reserves since 2006.The three SETAs that had decreased their cash positions were THETA, MERSETA and MAPPP).Five SETAs had increased their cash position by over 100% during this time (SERVICE, FOODBEV; MQA; CETA and CHIETA).Table 2 also assumes that SETA's are inefficient if the Cash position has increased by more than 3 0%, and in this case, fifteen of the twenty one SETA's are inefficient and six are efficient.
Five SETAs had increased their cash position by over 100% during this time (SERVICE, FOODBEV; MQA; CETA and CHIETA).Three SETAs that had decreased their cash positions were THETA, MERSETA and MAPPP).

Conclusion
The objective of this study was to examine the efficiently of South Africa's SETAs.Prior research on SETA's had not developed or applied a conceptual framework with respect to measuring efficiency using an input/output model (Gupta & Verhoeven, 2001).Each SETAs targets were then examined and this lead to a ranking of the SETAs.A further measure of efficiency was examined being increases in cash reserves.Finally a Random Effects GLS regression model suggested that only Objective 3, namely, the promotion of employment and sustainable development, was significantly (5% level) linked to the outcome efficiency variable.
Overall, the results showed that only one SETA's achieved all five objectives with respect to the input / output model while five achieved all five targets.The best and worst performers were fairly well supported by the limited prior research (Marock et al., 2008).The regression analysis supported these results.Further, there appears an excessive build up of cash reserves as 18 of the 21 SETA's had increased their cash position over the four years, and 15 had increased it by over 30%.
The study has a number of implications for each SETA, the South African Government and the sectors that are in need of critical skills and growth.Firstly in respect of the SETA the study has shown that some are efficient and some are not efficient.While some are meeting their own targets, these might be too easy and so their inefficiencies are not being highlighted.
The study has implications for the South African Government.Currently the SETA's are provided with funding of over R5 billion and yet many are not being efficiently managed.If the government was to put this funding into other areas (such as schools and universities) this may result in a better usage of funds.At the least the government should acknowledge that some SETAs are not efficient and close these down, and use these funds for those areas in critical needs.
This study is not without its limitations, however these could be used to further research in this important area.Firstly, the study was essentially an exploratory exercise whose findings should be further tested by more intensive studies of the individual SETAs.It should be noted, that performance measurement in the public sector is extremely problematic given the wide range of stakeholders, as well as service outcomes (Gupta & Verhoeven, 2001).A further limitation is the fact that the data only examined four years of reports, rather than from the time the SETA's came into operation.Further studies could examine why some SETA's are much for efficient than others and provide indepth reasons for these variations.