About the Author(s)


Caren B. Scheepers Email symbol
Gordon Institute of Business Science, University of Pretoria, South Africa

Paul Whelpton symbol
Gordon Institute of Business Science, University of Pretoria, South Africa

Citation


Scheepers, C.B. & Whelpton, P., 2018, ‘Stakeholders’ different perspectives of critical success and risk factors in a case of large-scale system implementation across Africa’, South African Journal of Business Management 49(1), a195. https://doi.org/10.4102/sajbm.v49i1.195

Original Research

Stakeholders’ different perspectives of critical success and risk factors in a case of large-scale system implementation across Africa

Caren B. Scheepers, Paul Whelpton

Received: 29 Jan. 2017; Accepted: 28 Jan. 2018; Published: 13 Sept. 2018

Copyright: © 2018. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: Most large information technology (IT) projects fail, costing businesses billions of rand while delivering limited benefits. This has stimulated considerable, yet inconclusive, research into the reasons for project success and failure.

Objectives: The study explores a massive IT system implementation project, throughout Africa, that cost the organisation almost four times its annual profits and taken more than 10 years. The majority of South African and other African companies in the financial services sector still run on old legacy IT systems and will have to undergo similar exercises. The conceptual model of critical success factors presented here could be used as a high-level blueprint for these future large information system implementations.

Method: The research questions required in-depth exploration of circumstances and incidents during the project life cycle and the case study method was the most appropriate design. Thirteen stakeholders were interviewed in a semi-structured interview format.

Results: This exploratory case study delivers a comprehensive conceptual model that covers the high-level phases of successful large IT project delivery. It shows that project success only occurs when all critical tasks across the effectiveness and efficiency dimensions of a project are planned, performed and measured accurately.

Conclusion: The differences in perspectives between stakeholder groups in the project ecosystem are highlighted, as well as their consequences. The study also contributes to the existing literature by providing a comprehensive formula for the accurate identification of overall project risk.

Introduction

Over the last decade, many major transformational information system projects have failed. The most recognised source of project success tracking, the CHAOS Manifesto (Standish Group 2013), found that only 39% of all projects are successful. This number declines to 10% for large-scale projects that take more than 3 years to implement (Standish Group 2013). The majority of companies can survive these massive cost and schedule overruns, but it is estimated that a staggering 17% of information technology (IT) projects fail so badly that they threaten the existence of companies (Bloch, Blumberg & Laartz 2012). These so-called black swan events (projects where the budget overruns by more than 200%) (Taleb 2007) happen more often than expected and emphasise the need to understand the reasons for project failure. Further research is essential to identify decision points, stakeholders and other project variables that contribute to the success of large transformational projects, and study the interdependencies between these factors. This study endeavours to offer businesses relevant risk evaluation criteria that could save billions in project failure costs. For the purposes of this study, ‘large-scale project’ refers to any project with an implementation timeline of more than 3 years and a total cost of more than $100 million. The target audience is top management who must take important decisions at crucial points in the life cycle of these projects.

Although there is an abundance of scholarly and practitioner research on the reasons for project success and failure (Hidding & Nicholas 2014), the key success factors in these studies vary significantly and most research has been reasonably indecisive (Fortune & White 2006). In addition, the majority of academic research focuses on specific aspects of project implementation, from in-depth project reviews (Pollack 2012) to studies that pay limited attention to a holistic organisational view of project delivery (Cooke-Davies 2002). This research contributes a consolidated conceptual model for the successful implementation of a large-scale information system.

The study used an exploratory qualitative research approach with an in-depth case study on the largest core system transformation on the African continent to date. The researchers attempted to integrate the vast, but disparate, findings, informing an interview schedule with questions to prompt deeper discussion of particular aspects. The subject of this case study was one of South Africa’s largest institutions – one of the first companies to attempt an information systems project of such large magnitude on the African continent.

Information systems are becoming an ever more important competitive component across a variety of industries. Project size and complexity are increasing, and projects now touch more elements of an organisation, thus creating greater risks for companies if something goes wrong (Bloch et al. 2012). Recent research on projects costing more than US$15 million suggests that there is a 45% cost overrun, a 7% schedule overrun and a staggering 56% benefits shortfall across large project implementations (Bloch et al. 2012). Large projects can contribute to a significant proportion of an organisation’s total costs. The best-performing projects have reported IT costs of $8.81 per $1000 of revenue, while the worst-performing projects cost as much as $34.81 per $1000 of revenue (INEKO Institute at the University of Cologne 2015). In the financial sector, the project cost ratio is even higher and is expected to increase as more firms migrate to new digital platforms to better serve their customers. This case study examines a company operating in the financial services sector.

Literature review

Two schools of thought on information system projects

Two main perspectives around information system projects, based on the seminal work of Peter Drucker in the 1960s (Drucker 1967), are the efficiency and effectiveness schools of thought (Hidding & Nicholas 2014). Research relating to the efficiency of an information system project is largely focused on project management, whereas research conducted on effectiveness takes an external view of the project and focuses on the stakeholders, outcomes and technology. The authors of this article advocate a more holistic approach, but it is nevertheless relevant to discuss these two perspectives.

Efficiencies (or project management) school

Kolltveit, Karlsen and Gronhaug (2007) noted that focus had shifted to leadership and task perspectives. These two are the focus of the bulk of research into the project management view of successful information system implementations. In their meta-study, DuBois et al. (2015) find that significant leadership and interpersonal skills, not only in-depth technical knowledge, strongly correlate with project success. Pollack (2012) contends that the success of a complex knowledge transfer organisational change project is largely driven by the organisation’s emphasis on visibility and senior management support. Remington and Pollack (2007) find that uncertainty in structurally and technically complex projects has a major effect on the ability of leadership to influence the successful outcome of a project.

Project managers who manage projects where there is a clear overlap between personality and project type, perform significantly better than the average manager (Malach-Pines, Dvir & Sadeh 2008). The way in which the project leader influences project members impacts the ultimate success of the project; for example, communication from the leader to the project team can be used to overcome control loss, namely early slippages in project delivery (Narayanaswamy, Grover & Henry 2013). Ika, Diallo and Thuillier (2012) find that the success factors that result in project success are multi-dimensional across a variety of different areas. The five key success factors are monitoring, coordination, design, training and the institutional environment, and these success factors change depending on the perspective of the stakeholder. Jiang et al. (2014) identify a clear need for an internal conflict management programme to ensure alignment between all projects and stakeholders, as well as shared understanding of the overall goals and interdependence between individual goals.

Since the publication of the very popular Agile Manifesto (Beck et al. 2001), there has been extensive focus on the principles of agile projects. Highsmith and Cockburn (2001), early pioneers of agile, conclude that many new IT projects that adopted the agile approach were deemed successful. They find that teamwork and team proximity (especially between business and their IT colleagues) were key success factors of the agile methodology. Other key elements of success in agile programmes were continuous communication and regular feedback regarding technical decisions, business requirements and constraints. Goh, Pan and Zuo (2013) find there were two crucial factors that influenced the development of agile information systems, namely the urgency of the project and the urgency to conclude the implementation. The authors further conclude that very large information system implementations have a much higher probability of creating uncertainty among stakeholders, thus negatively affecting delivery.

Given the literature review, the authors formulated the first research question as follows:

Which efficiency (project management) project actions contributed to the overall outcome of the project?

Effectiveness (or project success) school

The efficiency perspective discussed above focuses primarily on aspects of project management and might pay limited attention to the question of overall project success. The effectiveness school addresses this limitation. Project success refers to the effectiveness of a project (Hidding & Nicholas 2014) and focuses on the multiple stakeholders involved as well as the ultimate results, or project benefits.

Cooke-Davies (2002) contends that anticipated benefits are the key measures for formal and informal reviews during the project life cycle. However, only 13% of projects track actual financial benefits post completion (Hidding & Nicholas 2014). Nonetheless, regardless of the definition of a metric, whenever it is used to evaluate and rate a team’s performance, the value of the metric will move towards the desired value (Bouwers, Visser & Van Deursen 2012). Accurate measurement of information system projects remains a common problem; they should be measured in two categories, namely process efficiency and overall effectiveness (Basten, Joosten & Mellis 2011).

Having executive sponsors actively involved in project implementation is the key factor in project success (Kloppenborg & Tesch 2015). Cooke-Davies (2002) concludes that the success of a programme is not only influenced by the manager and project team, but also by the adoption and buy-in of the operational and line managers. Organisations need to take on risky programmes from time to time to ensure they maintain competitive advantage. It is the role of the senior leadership team to terminate all projects that no longer conform to the strategy of the organisation (Unger et al. 2012). Project termination does not always imply project failure (Boehm 2000). Unger et al. (2012) conclude that senior management involvement has an inverted u-shape relationship with project termination. This leads to the extension of senior stakeholders’ pet projects even if they add no customer or organisational value.

This review was expanded to include risk literature following initial exploratory interviews during which risk management was raised on numerous occasions by the chief information officer of the organisation under study. Risk management stretches across both the effectiveness and efficiency life cycles of a project and is made up of two key categories: hard side risk management (efficiency and project-specific risk factors) and soft side risk management (effectiveness and external risk factors) (Carvalho & Rabechini Junior 2015). Carvalho and Rabechini Junior (2015) find a significant correlation between project success and risk management in large-scale projects, with 10.72% of the success of projects being attributable to the soft side of risk management. Elzamly and Hussin (2014) note that there are various risk rankings for each identified risk during software project implementations and that these risks are highly dependent on the level of experience of the project manager, while Didraga (2012) contends that risk factors identified in previous projects contribute to the success of current project success.

Although the effectiveness school pays attention to the overall question of whether the project is a success, it may overlook the actual project management efficiencies. The authors of this article therefore advocate a more integrative and holistic approach to both the project management and project success questions. As Fortune and White (2006) emphasise, the critical success factors identified in this field of study vary considerably, and although there are some overlapping arguments, they have been reasonably indecisive, prompting more research in the area. Furthermore, there does not seem to be an end-to-end model that successfully encapsulates the entire project life cycle and the various factors that influence project success. The researchers thus argue that a more rounded view, that incorporates both the efficiency and effectiveness perspectives, is required and that there would be substantial academic benefit to a structured, high-level conceptual model that captures the success factors across the entire life cycle. Given the literature review, the authors formulated the second research question as follows: Which effectiveness-related project inflection points and actions contributed to the overall outcome of the project?

Methodology

Case study background

The South African and larger African banking industry currently still runs on legacy mainframe systems developed more than 30 years ago. These systems increase risk for the banking industry and must be replaced within the next decade as scale and complexity increase and banks move towards digitisation. The migration from legacy systems to modern digital platforms is often compared to changing the engine of a large aircraft mid-flight and will most likely be triggered by customer outages, fraud and competitor pressure during the next 10 years (Groenfeldt 2015). In cases where there are 50 or more systems to replace, their replacement is projected to overrun the initial budget by between $80 million and $100 million (Bloch et al. 2012).

It is against this background that the project examined in this case study was started by the IT executive team in 2005 as a core systems replacement initiative across the group. The project was intended to firstly replace the outdated legacy systems, to ensure sustainability into the future, and secondly, to create a single view of the customer across the group. However, by late 2009, not a single line of code had been written. Following the dismal failure of the first small project deliverable in 2010, the project was totally reset to evaluate the slow progress and extremely high-cost burn rate.

There was a fundamental change in approach. Project ownership was moved to the respective business units and the overall objective of the implementation changed. Excessive focus on the schedule, however, resulted in significant de-scoping of the original business case and delivery. Costs started to overrun as more and more resources were deployed to meet the stiff timelines. In 2013, the project saw another change of sponsorship, with the appointment of a new chief executive officer. Costs started to escalate as very expensive resources, often from Europe, were costing more than R2 million a day. The project had run for almost 10 years at this point, at a total cost of more than R16 billion – far more than the originally anticipated timeline and costs. After careful consideration, the project underwent its final reset at the end of 2014 to refocus on regulatory requirements and ensure that it met the minimum required standards of a system upgrade. In essence, the project reverted to its original goal of legacy system replacement.

The researchers consulted, reviewed and evaluated relevant literature to offer a lens through which to understand the incidents and decision points that emerged during the 10 years of this R16 billion project.

Given the background described above, and following an extensive review of relevant published studies, the researchers formulated the research questions as follows:

  • Research question 1: Which effectiveness-related project inflection points and actions contributed to the overall outcome of the project?
  • Research question 2: Which efficiency (project management) project actions contributed to the overall outcome of the project?
Research approach and sample

A research approach involves internal philosophical suppositions as well as external methods and procedures (Creswell 2014). The research questions required an in-depth exploration (Saunders & Lewis 2012) of particular decision points and roles during a project life cycle in the case of a large-scale system implementation. A constructivist world view was therefore appropriate. The research design was qualitative as the interview transcript data had to be analysed and coded. Yin (2003, 2015) described the case study method in qualitative research as context specific; the research questions described above required in-depth exploration of circumstances and incidents during the project life cycle and therefore was the most appropriate design. An inductive approach was thus followed to acquire qualitative insight into the processes underlying the large-scale system implementation and draw inferences from it.

It became apparent in the informal pre-interviews that the interviews with technical staff and non-managerial employees would bear much less information. The researchers thus focused predominantly on executives and senior management interviews, as they had the most project knowledge. The perspectives of various stakeholders, each of whom constructed their own relative reality based on their roles and exposure to the project, were needed; therefore, 13 stakeholders across the different stakeholder groups were identified and interviewed. This was a convenience non-probability sample, reliant on the availability of the relevant stakeholders (Saunders & Lewis 2012). There are certain specific people who were integral to the study, for example the chief information officer (CIO) and project sponsor, and the use of a non-probability sample is therefore well-supported. The interviewees included the current CIO of the South African operations, the previous project leader and now CIO of the African operations, and the executive responsible for change management and business readiness. Other interviewees were, for instance an executive that left the organisation, because of the perceived failure of the project in its later stages, and the chief financial officer, who was responsible for the finances of the organisation during this implementation. These seasoned executives were involved with the project since inception.

Data analysis

On completion of the interview transcripts, categories of codes were developed (Step 1); the unit of analysis to which these codes would be applied was decided on (Step 2); and the units of data were coded in a two-step approach (Step 3). A comparative analysis between the different stakeholder responses was performed to further understand the different opinions within the organisation regarding this project. The results from the qualitative study were used to develop a final conceptual model that illustrates the different decision points and stakeholder actions that impact project success.

One hundred and seventy-one low-level codes were identified during the in-depth coding of the interviews. Interviews were coded in the order they were conducted, and saturation was reached after interview 11. No new codes were identified in interviews 12 and 13. This illustrated that the amount of interviews were sufficient to answer the research questions. The codes were grouped into 19 subgroups, or themes, and ultimately summarised in accordance with the original research questions. Each identified code was also substantiated by a quotation from an interviewee. Some of the most impactful quotations are listed in the next section.

Findings

Research question 1: Which effectiveness-related project inflection points and actions contributed to the overall outcome of the project?

Interviewees identified five project inflection points that contributed to the overall outcome of the project, namely project sponsor or owner, a skilled governance board as a key stakeholder, project outcomes and scope identification, measurement and technology selection. Almost all the interviewees felt that a project of this nature had to be business-owned, for example an interviewee mentioned, ‘If we then go to success factors, first one and most important is senior executive ownership and accountability and this accounts most probably for 70% of the success factor’ (Respondent 1, Male, 51 years old, CIO).

Both IT and business executives and teams all agreed that projects of this nature should be owned by a business sponsor. There was, however, great confusion about who actually owned the project in this specific implementation. Business users kept mentioning that IT owned the project and that this was the core reason for failure. The IT representatives, however, felt that business took over ownership when the chief executive officer became the project sponsor. Throughout most of the interviews, there was a fair amount of conflict between business and IT representation. This is especially true for the most senior executives that participated in this research. Although these groups concurred that ownership should sit with a business sponsor, most disagreements arose when interviewees were prompted as to who the actual project owners were. There is thus a clear need to unequivocally articulate who owns this mandate. Change in ownership seemed to be the factor that caused this confusion, as sponsorship moved between business and IT regularly at the lower executive level.

The data showed a clear emphasis not on the process of governance but on the role of a governance board as a project stakeholder. Business often felt that although governance forums were very strict, they were very ineffective. This seemed to be because of a board and governance forum that did not have the expertise to make correct decisions regarding the project. The addition of an external, independent, experienced governance or project board seemed to significantly improve the perceived effectiveness of project delivery. Skilled consultants with previous experience of a similar project helped speed up decision-making and delivery substantially. Some respondents went as far as saying that leadership was unable to challenge the governance board, ‘Now, they didn’t have the courage to challenge the chief executive officer, that is why that IT system was just like sitting there and nobody was getting any traction’ (Respondent 2, Female; 46 years old, Senior Executive Business Sponsor [head of retail banking]). This fear of choosing the right side between business and IT seemed to have had a significant impact on the delivery of the project, ‘It was almost like there was competition as to who speaks to the chief executive officer first’ (Respondent 3, Male, 38 years old, Chief Financial Officer [Business Sponsor]).

The project business case or scope seemed to have been rewritten almost every year for the duration of the project. The words ‘silver bullet’ were mentioned on numerous occasions and summarised the overestimation of benefits very well. The transformation project had thus moved from a systems replacement to a revenue-generating business project. Several interviewees made reference to the retrofitting of the business case from a cost reduction system replacement to a revenue-generating transformation. These inconsistencies, as well as the continuous changing of requirements and perceived outcomes, resulted in massive delays in delivery, re-scoping and conflict. Business case ‘validation’ was also a theme that came up regularly, as new project members were expected to generate value from the massive investment. Most interviewees felt that the only direct benefit of the system replacement project was increased efficiency, leading to possible cost reduction.

Surprisingly, none of the executive sponsors interviewed referred to the measurement of the project. This group of stakeholders never went into any depth regarding the project metrics even after continuous probing by the interviewer. Most interviewees mentioned that business value or benefit measurement was not possible nor implemented across any areas of the project. Measurement was purely based on efficiencies regarding the time and effort spent to implement, ‘Over time we have found the overhead associated with complex metrics and measurement was too big and what was better is less metrics, but more accurate metrics’ (Respondent 4, Male, 52 years old, Senior IT executive [chief Program and Operating Officer]). No measurement of the effectiveness of the project was being tracked because of the perceived complexity and lack of involvement of the measurement team. It is thus safe to say that the project flew blind when it came to effectiveness measurement.

Neither the project management team nor any of the other technical project resources mentioned the choice of technology as a potential success factor. These stakeholders worked with the platform on a daily basis and their perception of the technology was always reasonably good. The only point mentioned by this stakeholder group was that a more generic ‘vanilla’ version of the technology should have been used. The above views from the project-specific stakeholders are in direct contrast to the themes from the business and IT executives who had strong, opposing views regarding the selected technology type and partner, ‘… first error that you could see, you picked the wrong system’ (Respondent 2, Female, 46 years old, Senior Executive Business Sponsor [head of retail banking]) ‘… the only discretion we had was around how we execute, but the technology choice, software, hardware, integration patterns and systems was all done, chosen, made’ (Respondent 5, Male, 48 years old, Executive Sponsor IT [CIO Africa region]).

Research question 2: Which efficiency (project management) project actions contributed to the overall outcome of the project?

Themes identified as efficiency project actions were project methodology, skills, location and direct, consistent business involvement. One of the most spoken-about themes throughout the interviews regarding project efficiencies was the concept of ‘Schedule is King’. This term referred to a specific project management methodology and was implemented in the organisation after the first project reset. Schedule is King is a trade-off methodology between three project variables, namely project time, project scope and project budget. It refers to a right-to-left project management style where the time of implementation is fixed. This means that the budget or scope can be changed as long as the time of delivery is met, ‘IT’s mandate was actually the quickest way to get this thing delivered’ (Respondent 2, Female, 46 years old, Senior Executive Business Sponsor [head of retail banking]) During most of the interviews, the IT teams and executives highlighted that the right-to-left methodology explained above was one of the core success factors. It is, however, very important to mention the contrasting views of the business, which held that a left-to-right approach, which guarantees the original scope and benefit, is the correct project delivery approach.

All stakeholders perceived the level of skills during this project as insufficient. Many references were made to the use of expensive third-party consultants from developed countries and the impact this had on the overall cost of the project:

If you want 50 business analysts out of the market it is one thing, but if you go out of the market and you want 250 competent people at a reasonable hourly rate it is nearly impossible (Respondent 6, Male, 60 years old, Senior Program Manager [Executive IT]).

Probably the most astonishing theme to emerge during the analysis process was the importance of a central physical project location. ‘Because of our language, culture, distance and if you don’t all sit in the same building you can easily misunderstand each other’; ‘the sponsors of the releases hardly ever saw the teams’ (Respondent 4, Male, 52 years old, Senior IT Executive [Chief Program and Operating Officer]).

The interviewees continuously mentioned the theme of the importance of business involvement during the delivery phases of the project. This was especially dominant from the IT stakeholders who felt that dedicated business resources should have been allocated for the entire duration of the project, ‘You actually need business embedded in the programme’ (Respondent 6, Male, 60 years old, Senior Program Manager [Executive IT]). A question regarding project termination was specifically asked in each interview with the aim of assessing the difference in responses from sponsors on the one hand, and non-biased executives and managers on the other. Most respondents completely avoided questions on the termination or refused to answer. This was clearly a very sensitive subject. Two executive sponsors from the business and two IT executive sponsors did, however, make significant reference to the topic. It seemed that the commitment to market investors was the single biggest reason why the project was never terminated, regardless of its progress. ‘We did try going to the board to say let’s write off this asset and cut our losses and move on’ (Respondent 3, Male, 38 years old, Chief Financial Officer [Business Sponsor]).

Project risk was defined as the probability of the project cost or schedule to overrun its estimated budgeted value or the outcome of the project to be significantly different from what was originally expected. Increased project risk thus correlates with a higher probability of a project being unsuccessful as measured by cost, time and scope. Multiple rich, in-depth discussions were held regarding specific project risks with certain individuals. Although some themes were only mentioned by a few individuals, the intensity and time consumed by some of these inductive findings warranted an in-depth representation of the individual findings. The themes revolved around size of project and team, change of sponsor, time duration and portfolio risk.

Project size was one of the key risk variables identified by stakeholders from operations, information technology and strategy. ‘It is risky because of its sheer magnitude. In our case, we are talking about more than a million man-days of effort’ (Respondent 1, Male, 51 years old, CIO). Transformational projects often entail a change to the core of the business that significantly impacts the way the business functions and increases cost significantly as they usually take a long time to finish. ‘Because you are changing the proverbial Boeing’s engines in mid-flight, you can’t land, change the engines and then take off again’ (Respondent 5, Male, 48 years old, Executive Sponsor IT [CIO Africa region]).

The interviewees indicated that there is a minimum unavoidable risk and then that risk increases according to the type of project. For example continuous improvement projects have relatively lower risk than upgrades and run-of-business projects, whereas transformational projects carry the highest risk.

The change in sponsor increased the overall risk in this project by a significant margin. The case study investigation revealed that the first reset of the project resulted in the dismissal of the initial two project sponsors and the announcement of a new project sponsor, namely the chief executive officer. Senior programme executives, in particular, felt that although the project risk initially declined when the chief executive officer became the sponsor, it significantly increased when a new chief executive officer was announced during execution of the project:

Sponsor should ideally be in place for the duration of the programme; I think if you have that, there is no doubt that your overall risk of implementation drops dramatically. (Respondent 1, Male, 51 years old, CIO)

Most interviewees felt that the only way to ensure on-time delivery after any setback was to increase the size of the team. This naturally increased the cost of the overall programme significantly, as the burn rate per man hour went up. There was, however, a mutual perspective among the project managers and business stakeholders that a team that becomes too big actually reduces efficiency. The majority of the interviewees were in favour of headcount reduction, as it simplifies the very complex process of managing large teams in matrix organisations. The size of the team responsible for the delivery of a project increases its complexity. This is because of the increased strain on managers to steer the team and perform administrative management tasks:

Let’s say on the y-axis you got risk and on the horizontal axis you have a number of people. Call it execution risk and number of people. It is linear up to a point, then it breaks out as non-linear. The moment you get to that point, your execution risk all of a sudden exponentially jumps up. (Respondent 1, Male, 51 years old, CIO)

The researchers illustrated this finding in Figure 1.

FIGURE 1: The relationship between team size and project delivery risk.

The interviewees indicated that the more legacy systems involved in the project, the higher the risk of failure. This project had more than double the amount of legacy systems that needed integration than the next company that had the second most dependable systems. It was further estimated that between 60% and 70% of the total effort and cost of the project was purely driven by the integration of legacy systems. It is intuitively true that shorter projects hold less risk because of their lack of complexity. It is quite different when it comes to large transformational programmes. ‘It is a temporary thing; you can sustain it for 4 to 5 years max’ (Respondent 7, Male, 57 years old, IT Delivery Executive [Program Manager]). This senior IT executive also shared sentiments regarding the duration of the project, saying that an organisation’s appetite for a project can only span up to 5 years before it becomes business-as-usual – a dangerous factor as intensity, focus and control drop significantly.

A large project inherently starts with a massive amount of risk because of the size and uncertainty of the deliverable. This risk then gradually declines as the project progresses towards its conclusion. All the stakeholders agreed that when a project extends longer than 5 years, the risk starts increasing exponentially again. Most of the executives argued that smaller sub-projects would have significantly mitigated the risk of project failure.

Figure 2 demonstrates the 5-year project duration threshold at which risk increases.

FIGURE 2: The relationship between project duration and project delivery risk.

The impact of other programmes on the delivery of the project under study, and vice versa, was also identified by interviewees as an important risk factor. ‘So when you threw the first domino over you thought, ‘okay we’ve dealt that one’, but it beats all the other dominos left-to-right’ (Respondent 5, Male, 48 years old, Executive Sponsor IT [CIO Africa region]).

The quote perfectly summarises the sentiments of the project and delivery executives who felt that no project should be managed in isolation. Initial delays and dependencies on resources or other projects significantly increase the risk of overall delivery. Technical resources were not dedicated to the project and were often required to ensure system stability. The knock-on effect of such delays was devastating, and most stakeholders felt this risk was not well managed during this project. Interviewees mentioned a project portfolio and the need to manage the project ecosystem, resources required and shared, as well as the interdependencies between individual projects, as illustrated in Figure 3.

FIGURE 3: Illustration of the elements in the project ecosystem.

Figure 3 demonstrates an example with a typical project ecosystem, extracted from interactions with the respondents. It illustrates that a resource pool, where, for instance, IT software developers are based, are shared between projects (called project 1 to 4) that are servicing the current business or business-as-usual projects, as well as projects that are part of the transformation project (called project 5 and 6 in the illustration) where new IT systems are developed. These projects are interdependent, increasing the complexity of managing the interfaces between them.

Conclusion

Although previous research suggested two main schools of thought regarding the success of major information system implementations (Hidding & Nicholas 2014), namely effectiveness and efficiency schools of thought (Drucker 1967), this case study revealed that the two schools of thought were not mutually exclusive opposing views, but rather phases in the overall delivery of the project. Interviewees discussed many different aspects of project success that span across both schools of thought. This study proposes a sequencing of the phases that can be seen in Figure 4. This sequencing suggests that any project starts predominantly with effectiveness tasks during the pre-planning and planning stages of a project. In this phase, as stakeholders scope the project, the effectiveness tasks and actions are the most crucial to the outcome. After planning, a project enters the execution phase. During this phase, the most important factors to consider for project success all form part of the efficiency school of thought, as illustrated in Figure 4. The final phase once again returns to the effectiveness school of thought and focuses on the project outcomes and benefits. This includes project measurement at all levels and for all stakeholders involved. This sequencing of the major project phases is a key finding of the study and forms the backbone of the conceptual model for project success.

FIGURE 4: Holistic conceptual task framework for project success, based on synthesis from case study findings.

These phases, with their respective critical success factors, are discussed below. The researchers also point out where the organisation did not adhere to this framework. The conceptual model illustrated in Figure 4 aims to clarify the essential roles in any large project as well as the key tasks stakeholders need to perform. The model further aims to clarify at what point in the project delivery cycle each player should be involved; for example the data gathered in this study showed contradictory evidence regarding a handoff between the project team and operations (Cooke-Davies 2002). Although the rich data set offers several interesting aspects during the phases of a project of this scale, the researchers chose to focus on the stakeholders’ different perspectives and their consequences.

Stakeholders’ involvement

The interviewees emphasised the involvement of all stakeholders, but found a crucial gap when it came to the clarification of each player’s role. Therefore, in Figure 4 there is an asterisk (*) in each stakeholder row. The importance of the tasks of these stakeholders, however, varied across the phases of the project. The conflict between project executives and managers on the one hand, and their operational counterparts on the other, led to great disparity in the perceived outcome of the project. In Figure 4, the conflict resolution task of the independent governance board is marked with an asterisk, indicating that this important principle was not adhered to in the project, for instance the particular task was not performed. The literature suggested that there is often a handoff in large projects between the project team, which is responsible for planning and implementation, and the operations team, which is responsible for institutionalising the benefit (Cooke-Davies 2002). Senior executives are key project stakeholders and are responsible for terminating projects (Unger et al. 2012) and ensuring that the correct balance between risk and innovation is applied (Boehm 2000).

The analysis in this study supported the view of Kloppenborg and Tesch (2015) on the impact of the executive sponsor on the outcome of a project. Interviewees, however, went further than suggested in the literature by specifying three key attributes essential for the executive sponsor. Firstly, the sponsor should not be a committee; a single person should ultimately be accountable. Secondly, the sponsor should remain constant throughout the process and should be responsible for, and involved in, all phases of project delivery. There is therefore an asterisk next to ‘accountable individual’ in all three phases, indicating that this aspect was a major problem in this project. Finally, the project should be business-owned, preferably by the most senior executive or chief executive officer.

The second stakeholder identified in this research was a separate IT executive sponsor. In this specific project, this was the CIO, but this role can vary based on the organisation and project size. It is very important that the roles and tasks of the ultimate executive sponsor or owner and the most senior IT executive are clearly specified. In this project, a gap was identified around this aspect, illustrated by the asterisk added to the ‘evaluate and choose technology’ task of the IT executive sponsor. The model in Figure 4 depicts the three core tasks for the IT executive sponsor. Firstly, the IT executive sponsor is responsible for assisting the business sponsor to evaluate and make decisions about the technology to be used. As an IT expert, this task and accountability should fall to the IT executive sponsor, and the chief executive officer or business sponsor should hold this person accountable for decisions relating to this task. The second core responsibility of the IT sponsor is to choose the appropriate project delivery methodology. The final task responsibility of the IT executive is to provide system stability. The IT executive sponsor should thus not be held accountable for the business benefit delivered but for the efficiency and performance of the system instead.

The third stakeholder is a summary of the roles of what is loosely referred to as the project team and aligns with the definition of Cooke-Davies (2002). This includes the project manager, business and system analysts, and all programmers working directly on the project. The overall accountability for the tasks of this stakeholder group will ultimately fall on the highest-ranking individual (usually the project manager). It is very important to realise that the IT executive sponsor is deliberately excluded from this group. Because of the sheer size of these projects, this model intends to ensure that implementation and planning remain independent. This independence allows IT executives to make decisions regarding technology and the method of implementation freely, without bias from a project manager. In this project, this aspect was neglected.

The project team is firstly responsible for providing the IT executive sponsor with pre-project proof of concepts to assist him or her to make decisions regarding technology and implementation methodology. After project commencement, the project team is responsible for the delivery of the efficiency phase, ensuring accurate management of time, resources and location of work. This case study found a gap in the management of the location, for example indicated by an asterisk in Figure 4. A major part of the efficiency school of thought focuses on the project management task of this stakeholder (Hidding & Nicholas 2014; Thakurta 2012). It is the project team that makes the ultimate decision about the most appropriate implementation style.

The business non-sponsor stakeholder group is the business team referred to as ‘operations’ (Cooke-Davies 2002). This group includes all business partners that will make use of a fully-implemented system or programme and is not limited to the operations team. This stakeholder group thus refers to any business users and could include functions such as finance, human resources, product managers and operations. The literature review and results of this study confirm that the continuous involvement of these stakeholders is a key aspect of project success (Kloppenborg, Tesch & Manolis 2014).

One of the two new stakeholder groups that were inductively introduced following the case study data analysis was the independent governance board. The literature refers to governance as part of the efficiency school of thought, and it is often criticised as being a stumbling block to agile project execution (Goh et al. 2013). However, this study revealed that the addition of an independent, highly skilled governance board as a key stakeholder in project planning, delivery and outcome is a crucial aspect of effective delivery.

The final stakeholder that plays an increasingly pivotal role in the outcome of a large project is the market. Project termination does not always imply project failure (Boehm 2000). It is thus essential to manage the perceptions and expectations of the market regarding large system implementations. This study identified the market as a key stakeholder. It may prevent sponsors and executives from terminating failing projects if communication about project benefits has been issued prematurely. Large listed organisations are prompted to explain large project costs and often fall into the trap of overestimating and over-promising business benefit. This increases the pressure from the market which values firms based on future benefits. Three key actions regarding the market were identified in Figure 4.

Other themes include measurement, scope and technology selection. Chen (2015) contended that a large percentage of overall project success can be attributed directly to the way the programme is measured. During this case study, it became clear that the business case had been adjusted multiple times to manipulate a desired outcome. A number of project reset episodes took place during the 10 years, as requirements, measurement and business rationale were once again reviewed. Basten et al. (2011) suggest that project measurement remains an ongoing concern for most projects and should be broken down into process efficiency and project outcome measurements, which is mirrored by the findings of this study. Two themes that emerged through the inductive analysis of the interviews were that of scope identification and technology selection. Only very few recent studies focus on these themes, although they are very common in articles relating to the project sponsor. Kloppenborg et al. (2014) point out that the executive sponsor’s most important role is to correctly identify and scope a project before initiation. This also includes the selection of the appropriate technology in an IT-related programme. This study, however, suggests that the finalisation of scope should involve all aspects of the business, and especially the operations team which identifies most opportunities for improvement. It also concluded that the selection of technology should be performed by an expert and therefore be the responsibility of the IT project executive.

Different perspectives of stakeholders

Although there is extensive literature about different project execution styles or methodologies, for example the waterfall (sequential format) and agile (adaptive) methodologies (Goh et al. 2013; Thakurta 2012), none of the respondents mentioned these execution models as important success factors in the project. An interesting finding from this study was that the IT teams and executives preferred a right-to-left approach. They felt that this was the most successful project methodology as it forced delivery. The business teams, and in particular, the executive sponsors, however, had completely opposing views. They opined that business benefit should drive all projects and that overruns should be managed by the project team as project efficiencies. Both stakeholder groups, nonetheless agreed that no project should change core high-level methodology in flight. In the case used for this study, the methodology was changed three times and created immense inefficiencies and confusion.

Kolltveit et al. (2007) referred to the task and transaction cost project management dimensions, where the focus is purely on ensuring that the right people are performing the right tasks at the right speed. The final efficiency-related finding was the need for continuous and dedicated business involvement. This is in line with the findings of Cooke-Davies (2002) and Kloppenborg and Tesch (2015) who state that business sponsors and operational teams are essential stakeholders during all phases of project delivery.

Unger et al. (2012) submitted that senior executive stakeholders should be responsible for and comfortable with terminating failing projects. External project risks, such as the market, can in some extreme cases contribute as much as 10% of overall project success (Carvalho & Rabechini Junior 2015). The overwhelming majority of respondents felt that this project had been showing signs of failure since its inception and yet none of the senior executives had managed to terminate it.

The reasons for continuing with the project, even after early warning signs of failure, were market expectations and pressure. Major business benefits were promised and priced into the company’s share price to justify some of the large initial costs. These expectations were not adequately managed right from the start and this resulted in increasing pressure from the market.

A core finding from this study, as depicted in Figure 4, is that the market is an essential project stakeholder. Communication to the market should be strategically planned and not issued prematurely. The market also contributes significant amount of external risk to any project and should form part of the overall risk management capability.

Project delivery risk formula

Project risk and risk management are major contributors to project success (Carvalho & Rabechini Junior 2015). This case study identified five high-level project risk betas that further influence the inherent project risk mentioned in the literature review above. The researchers combined these, using the relationship of the betas to the minimum unavoidable risk (alpha), to create a project risk formula. This formula results in a score out of 100, with 100 being the riskiest a project could ever be. The application of this risk formula to this project resulted in a total project risk score and serves as an illustration of how the formula could be utilised in future projects, see equation 1.

α: Inherent unavoidable project risk

Alpha represents the minimum unavoidable project risk. External project risk can contribute anything between 10% and 25% of total project risk (Carvalho & Rabechini Junior 2015). For the purpose of this study, the researchers assumed that this inherent risk contribution was 20%, including both internal and external risk.

vβ: Project size beta

The project size beta refers to the impact overall project size has on the risk of delivery. The project used for this case was the largest project ever recorded on the African continent, estimated to cost close to R40 billion by final completion. The project size beta thus takes a 0.85 estimated value.

wβ: Legacy integration beta

The more integration and legacy systems involved, the closer to the legacy integration beta one will get. The legacy integration needed for this project was more than double the amount required in any other similar related project. According to one of the senior executives interviewed, legacy integration contributed close to 70% of the total cost. The legacy integration beta thus also scores a 0.85 estimated value.

xβ: Project time beta

The duration of this project is more than 10 years at this point. The original benefit (negative beta) is thus foregone, as this project took longer than the desired 5-year maximum period. The project time beta for this project is thus estimated at 0.70.

yβ: Team size beta

The team size beta was well managed in this case study because of learnings from partners that had previously attempted similar programmes. Team sizes were deliberately reduced to stay below the 700 threshold; therefore, the team size beta is estimated at a value of 0.4.

zβ: Change in sponsorship beta

The change in sponsor beta is a mutually exclusive event that either has a value of 1 (if any change in sponsor occurred during the project) or 0 (when no sponsor change occurred). A change of sponsor will thus contribute an additional 20% to the overall project risk. There were two crucial changes of overall executive sponsor in the project used in this study, and the value of zβ is thus 1.

All these estimated values were added to equation 2 to calculate an overall risk score for this specific project.

The overall estimated project risk score of 76% shows this project to be extremely high risk. This result correlates with the fact that this specific project has cost billions of rand, way over initial budget estimates, and has run many years beyond the original schedule. The formula will have to be empirically tested against other known cases. The alpha value should also be unpacked further to ensure that the 20% given value is the most accurate measure of minimum risk.

Teller (2013) and Teller, Kock and Gemunden (2014) introduce the idea of portfolio risk management. Findings in this case study support this notion, as respondents introduced the concept of a project ecosystem. Most projects share resources from a central pool. These resources thus work on many different projects at any given time. Figure 3 shows the relationship between these shared resources and the projects they are involved in. It is vital to note that there are two types of projects within these ecosystems. Firstly, ‘run the organisation projects’ can be defined as smaller projects that form part of the continuous improvement of any institution. Secondly, ‘change the organisation projects’ refer to large-scale projects that alter the very way an organisation operates. These projects were also found to be dependent on each other in many cases, and the interviewees mentioned that this dependency risk has a major impact on overall project success. Risk management is a continuous process that evolves with the relevant project phases (Didraga 2012). It is thus of utmost importance to use learnings from other projects or external parties and include them into the overall portfolio risk management system.

Managerial implications

The study focused on a phenomenon not often seen, but that is important for business. Insight was gained from the large-scale system implementation to contribute to the body of knowledge on management of these projects. An increasing amount of companies need to do similar, albeit smaller, upgrades of their information technology systems, and the authors hope that they will find value in these findings. This study created a high-level conceptual model of overall project success factors and actions that spans across all phases of the large-scale information system project under review. This high-level, integrated task model is one of the few to provide an organisation-wide view of project success. It provides a high-level blueprint for successful project implementations across the three phases of project delivery. The model highlights that, even when some tasks that have a significant impact on the overall outcome of a project are performed exceptionally well, success is not guaranteed. Project success only occurs when all critical tasks across the effectiveness and efficiency dimensions of a project are planned, performed and measured accurately, and by the correct stakeholder groups, in a project ecosystem.

A major finding of this study is the connectedness of the two schools of thought around information system projects. The first project phase, planning and pre-planning, falls into the effectiveness school of thought. The second phase focuses purely on project efficiencies and can be summarised as the project execution phase. The final phase, the project outcome phase, focuses once again on project effectiveness by measuring the outcome and benefits delivered. This study allows for a paradigm shift in the academic research regarding project success and allows future researchers to build on the concepts introduced regarding the sequencing of the effectiveness and efficiency-related success factors of a project.

Another contribution was the clarification of the roles and major tasks each of the project stakeholders has to fulfil. Stakeholders were carefully grouped in Figure 4, and specific high-impact tasks critical to project success were allocated to each group. In addition to the core project tasks, the study also identified two new core stakeholders: the market and an independent governance board. These two stakeholders make massive contributions to the overall success of a project and should be included in any large IT project.

The study further offers a comprehensive risk management formula that can be used to estimate the risk for a project ecosystem. This risk estimation formula should be used as an indicative metric only until empirically tested and proven in further research.

This study provides the leadership team of the company where the study was conducted, with a comprehensive and unbiased view across all the phases of delivery that can be used to evaluate and learn from for future projects. Other organisations’ top management who are tasked to execute these types of projects could benefit from taking note of the insights that this study offers.

This study contributed to recent academic literature in three ways. Firstly, it combines the effectiveness and efficiency schools of thought into a sequentially phased delivery process that highlights the important interplays across the two methodologies that are usually analysed in isolation. Secondly, it provides a blueprint conceptual model for overall project success that can be used as the foundation for future contributions and empirical evidence. This research thus allows for a paradigm shift away from the current trends towards project management design and agile execution, to a more comprehensive holistic view of project delivery from an organisation’s perspective. It also illustrates some of the integration points and counterarguments to project success factors.

Possible limitations

In terms of limitations, the interview schedule used to guide the interviews was deduced from the literature review and posed very specific themed questions to respondents. Subsequently, new themes were inductively extracted from open conversations during the interviews that could not in all instances be adequately checked and verified by all interviewees. The reason for this was the seniority of some of the executives interviewed. They could not be reached for a second, follow-up interview to discuss the new constructs, as their schedules simply did not permit such interactions.

As this study was a case study regarding a specific entity and project, a clear limitation exists respecting the population sample. Interviewees, although diverse in their involvement, were all from the same organisation, which resulted in a limitation around organisational culture and any other external project factors that need to be verified across organisations or industries.

Future research

Future research should involve empirical testing and validation of the high-level conceptual task model revealed in Figure 4. This model should be tested in other industries or organisations across different geographic areas. A second suggested research topic would be the empirical analysis of the risk beta formula generated by the study. This formula has the potential to provide management with a highly relevant regression formula to predict overall project risk.

Future researchers must build on the concepts introduced in this study regarding the sequencing of the effectiveness and efficiency-related success factors of a project. It moves the focus away from in-depth project management methodologies and allows for the exploration and testing of integrated organisational project models.

Specific suggested research will be the empirical testing and validation of the high-level conceptual task model revealed in Figure 4. This model should be tested in other industries or organisations across different geographic areas. A second suggested research topic would be the empirical analysis of the risk beta formula that was generated. This formula has the potential to provide management and literature with a highly relevant regression formula to predict overall project risk.

Finally, more empirical evidence could also be obtained regarding the two new stakeholders that were introduced in this model: (1) the independent governance board, and (2) the market. Limited research exists regarding these two critical project stakeholders and the impact they have on project success.

Acknowledgements

Competing interests

The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.

Authors’ contributions

C.B.S. wrote the article and was the supervisor of P.W., who was the MBA student who conducted the research project.

References

Basten, D., Joosten, D. & Mellis, W., 2011, ‘Managers’ perceptions of information system project success’, Journal of Computer Information Systems 52(2), 12–21.

Beck, K., Grenning, J., Martin, R.C., Beedle, M., Highsmith, J., Mellor, S. et al., 2001, Principles behind the Agile manifesto, viewed 21 September 2015, from http://www.agilemanifesto.org/principles.html

Bloch, M., Blumberg, S. & Laartz, J., 2012, Delivering large scale IT projects on time, on budget and on value, McKinsey, New York.

Boehm, B., 2000, ‘Project termination does not equal project failure’, Computer 33(9), 94–96.

Bouwers, E., Visser, J. & Van Deursen, A., 2012, ‘Getting what you measure’, Communications of the ACM July, 54–59. https://doi.org/10.1145/2209249.2209266

Carvalho, M.M. & Rabechini Junior, R., 2015, ‘Impact of risk management on project performance: The importance of soft skills’, International Journal of Production Research 53(2), 320–340. https://doi.org/10.1080/00207543.2014.919423

Chen, H.L., 2015, ‘Performance measurement and the prediction of capital project failures’, International Journal of Project Management 33(6), 1393–1404. https://doi.org/10.1016/j.ijproman.2015.02.009

Cooke-Davies, T., 2002, ‘The “real” success factors on projects’, International Journal of Project Management 20, 185–190. https://doi.org/10.1016/S0263-7863(01)00067-9

Creswell, J.W., 2014, Research design, Sage, Los Angeles, CA.

Didraga, O., 2012, ‘The importance of risk management for achieving success in IT projects’, Managerial Challenges of the Contemporary Society 125–129. https://doi.org/10.1016/S0263-7863(01)00067-9

Drucker, P., 1967, The effective executive, Harper Collins, New York.

DuBois, M., Hanlon, J., Koch, J., Nyatuga, B. & Kerr, N., 2015, ‘Leadership styles of effective project managers: Techniques and traits to lead high performance teams’, Journal of Economic Development, Management, IT, Finance and Marketing 7(1), 30–46.

Elzamly, A. & Hussin, B., 2014, ‘A comparison of fuzzy and stepwise multiple regression analysis techniques for managing software project risks: Implementation phase’, International Management Review 10(1), 43–55. https://doi.org/10.3844/jcssp.2014.1725.1742

Fortune, J. & White, D., 2006, ‘Framing of project critical success factors by a system model’, International Journal of Project Management 24(1), 53–65. https://doi.org/10.1016/j.ijproman.2005.07.004

Goh, J.C.L., Pan, S.L. & Zuo, M., 2013, ‘Developing the Agile IS practises in large scale IT Projects: The trust-mediated organisational controls and IT project team capabilities perspective’, Journal of the Association for Information Systems 14(12), 722–756. https://doi.org/10.17705/1jais.00348

Groenfeldt, T., 2015, ‘Replacing-legacy-core-banking-systems-sap-has-limited-success’, The Forbes, viewed 16 November 2015, from http://www.forbes.com/sites#/sites/tomgroenfeldt/2012/12/18/replacing-legacy-core-banking-systems-sap-has-limited-success/2/

Hidding, G.J. & Nicholas, J.M., 2014, ‘Reducing IT Project Mangement Failures: Early Empirical results’, 47th Hawaii International Conference on System Science, IEEE, Waikoloa, pp. 4305–4314.

Highsmith, J. & Cockburn, A., 2001, Agile software development, IEEE Explore, Los Angeles, CA.

Ika, L.A., Diallo, A. & Thuillier, D., 2012, ‘Critical success factors for world bank projects: An empirical investigation’, International Journal of Project Management 30, 105–116. https://doi.org/10.1016/j.ijproman.2011.03.005

Ineko Institute at the University of Cologne, 2015, Total information technology cost per $1000 of revenue, viewed 13 August 2015, from www.bmc-eu.com/dokumente/49…information-technology-cost/download

Jiang, J.J., Chang, J., Chen, H.-G., Wang, E. & Klein, G., 2014, ‘Achieving IT program goals with integrative conflict management’, Journal of Management Information 31(1), 79–106.

Kloppenborg, T. & Tesch, D., 2015, ‘How executive sponsors influence project success’, MIT Sloan Management Review 66(3), 27–30.

Kloppenborg, T., Tesch, D. & Manolis, C., 2014, ‘Project success and executive sponsor behaviours: Empirical life cycle stage investigations’, Project Management Journal 45(1), 9–20. https://doi.org/10.1002/pmj.21396

Kolltveit, B.J., Karlsen, J.T. & Gronhaug, K., 2007, ‘Perspectives on Project Management’, International Journal of Project Management 25, 3–9. https://doi.org/10.1016/j.ijproman.2005.12.002

Malach-Pines, A., Dvir, D. & Sadeh, A., 2008, ‘Project manager project fit and project success’, International Journal of Operations and Production Management 29(3), 268–291. https://doi.org/10.1108/01443570910938998

Narayanaswamy, R., Grover, V. & Henry, R., 2013, ‘The impact of influence tactics in information system development projects: A control-loss perspective’, Journal of Management Information Systems 30(1), 191–226. https://doi.org/10.2753/MIS0742-1222300106

Pollack, J., 2012, ‘Transferring knowledge about management: Implementation of a complex organisational change program’, International Journal of Project Management 30(8), 877–886. https://doi.org/10.1016/j.ijproman.2012.04.001

Remington, K. & Pollack, J., 2007, Tools for complex projects, Gower Publishing Limited, Surrey.

Saunders, M. & Lewis, P., 2012, Doing research in business management: An essential guide to planning your project, Edinburgh Gate, Pearson.

Standish Group, 2013, ‘CHAOS Manifesto’, Standish Group, viewed 10 July 2015, from http://www.versionone.com/assets/img/files/ChaosManifesto2013.pdf

Taleb, N.N., 2007, The Black Swan. The impact of the highly improbable, Random House Publishing, New York.

Teller, J., 2013, ‘Portfolio risk management and its contribution to project portfolio success’, Project Management Journal 44(2), 36–51. https://doi.org/10.1002/pmj.21327

Teller, J., Kock, A. & Gemunden, H.G., 2014, ‘Risk management in project portfolios is more than managing project risks: A contingency perspective on risk management’, Project Management Journal 45(4), 67–80.

Thakurta, R., 2012, ‘Project execution strategy choice and its impact in software projects under requirement volatility’, The XIMB Journal of Management 9, 27–46.

Unger, B.N., Kock, A., Gemunden, H.G. & Jonas, D., 2012, ‘Enforcing strategic fit of project portfolios by project termination: An empirical study on senior management involvement’, International Journal of Project Management 30, 675–685. https://doi.org/10.1016/j.ijproman.2011.12.002

Yin, R.K., 2003, Case study research, 3rd edn., Sage, Thousand Oaks, CA.

Yin, R.K., 2015, Qualitative research from start to finish, Guilford Publications, New York.



Crossref Citations

No related citations found.