Introduction
Projects often face inexplicable time and cost overruns. Studies of the causes suggest weaknesses in the basic tenets of the traditional view of projects. Research efforts aimed at providing a theoretical basis to an alternative conceptualization of projects based on complexity theory indicate that there is no consensus so far. Issues such as the nature of projects as well as a formal characterisation of complexity in the project context are still open to further research. For instance, Soderlund (2004) argues that a project is a phenomenon (rather than a perspective) and this allows researchers “to build strong and interesting theories that increase our knowledge of certain parts of social life”. An understanding of such multiplicity of views needs further reflection, an epistemological analysis (with philosophical underpinnings) of the nature of a project and blending ideas from fields as diverse as teleology and cognitive psychology. While one may debate on the relevance of a meta-theoretical approach to such a practical corporate activity as managing a project, this allows one to start afresh, keeping aside the views from the mainstream ontology. One may need to address both theoretical issues like making sense, rethinking, defining meanings and setting research directions and practical issues like taxonomy of projects, typology of project management styles and the appropriateness of a style to the specific context of a project and its environment. Thus a study of complexity in managing projects is of both theoretical and practical importance.
Multiple perspectives
Recognizing the complexity of projects, Cicmil (2000) makes a case for a framework based on multiple perspectives incorporating conceptual aspects like project context, content and organizational behavior. Williams (2005) points out a need for developing a theory of project behavior as “there have been few empirical positivist studies of projects”. Such a theory can possibly be used to “identify projects with a propensity for failure”. Williams concludes his paper stating that the practical question of how such projects can be managed differently needs further research. These developments have rekindled a debate on the very existence of a “theory of projects” with researchers assuming dichotomous postures (Winter, et al., 2006; Turner, 2006). It is seen that such a “theory” is still evolving at least in the sense that researchers are continually revisiting and questioning the underlying assumptions and framework in the light of practical experience. We briefly explore some of these views.
Traditional view
In this view, a project consists of a clear goal, a temporary organization to achieve this goal and performance criteria such as scope, time and cost. This view is well established and constitutes the dominant ontology of projects (as seen in bodies of knowledge and handbooks). The three characteristics of the traditional model are (Williams, 2005):
Project management is rational (Lundin, 1. 1995) and normative (Packendorff, 1995).
The ontological stance is effectively posi2. tivist (as in Johnson & Duberley, 2000).
Scope can be managed by decomposing the 3. project using work breakdown structures (Turner, 2006; Koskela & Howell, 2002).
Linehan and Kavanagh (2006) find that the dominant ontology in project management is one of being rather than becoming. As pointed out by Williams (2005), the former emphasizes reification and representation (and a decoupling from environment) whereas the latter emphasizes sense making and questioning boundaries. Williams (2005) proposes that one needs to search for alternatives that allow us to move from the being view of projects to the becoming view. Winter et al. (2006) propose a shift in the theory of projects from the single “life cycle” theory to multiple theories that seek to understand the “complexity” of projects. Thus, the realization of inadequacies of the traditional model has spurred interest in multiple views of a project that can possibly address these difficulties. We postulate that such a model can emerge from a framework based on complexity theory.
Success factors view
Several researchers have devoted their efforts to identify the factors that contribute to project success (Fortune & White, 2006; Pinto & Slevin, 1987) and project complexity (Williams, 2005; Tatikonda & Rosenthal, 2000). These efforts have not yielded a consensus of opinion either in the choice of factors or their relative importance (Soderlund, 2004). The limitations of the success factors approach are well known (Fortune & White, 2006). Researchers like Soderlund (2004) find that this approach does not “develop our understanding” of project management success. Engwall (2003) finds that this approach ignores the history and context of a project. For instance, studies of failed projects may indicate factors like “project manager had too little formal authority” or “there was no proper planning and monitoring” as causes. However, in the case studies of Engwall (2003), the reverse was true, and the dysfunctional leadership style of the project manager of the failed project (who had formal authority and used planning methods) is more readily explained within the surrounding context of the project than in isolation. On the other hand, the successful project was headed by a project manager who had an in-depth understanding of the “inner life” of the division. Engwall (2003) thus argues for a non-universal, contingency approach to project management. This contingency approach is different from that of others like Shenhar (2001) in which project management is adapted to the specific project type (rather than context), where the project type is based on attributes like levels of uncertainty, complexity, pace and other factors like environment, product and task that are assumed to be independent of context. Thus, a reductionist approach based on critical success factors alone may not be adequate to manage complexity in projects.
Systems view
Causal feedbacks (with unpredictable time delays) are often exacerbated through management response (such as acceleration) to project perturbation. Several researchers have proposed an alternate view of projects based on the System Dynamics paradigm. Cooper et al. (2002) discuss feedback structures as the root of complexity and identify three particular structures that “generally” underlie project dynamics: the rework cycle, the productivity cycle and the work quality cycle. Incorporation of such practical aspects has contributed to empirical work based on systemic modelling that can explain unexpected time and cost overruns. Williams (2005) finds that his team could explain (post facto) much of the time delay using positive feedback loops ( “vicious circles”) in systems modelling. Their models also explain how the tightness of time-constraints strengthens the power of feedback loops, so that small uncertainties can cause unexpectedly large effects (like attractors in complexity theory).
Lean view
Williams (2005) suggests that projects that exhibit the three characteristics: structurally complex, uncertain and heavily time-limited, (the properties of “complex” projects) can benefit from the “lean” view. Inspired by the work of Shingo (1988) in the lean manufacturing context, Bertelsen and Koskela (2002) develop the TFV model for construction project management that attempts to achieve “integration and balance” by capturing the execution details ignored by the traditional approach. The TFV model is composed of three different perspectives, namely Transformation (for operations), Flow (for process) and Value generation (for meeting customer needs).
Towards a complexity framework
Complexity of projects is a recurrent theme in several research articles. It is known that projects show non-linear and non-intuitive behavior that is difficult for managers to predict (Sterman, 1992). Baccarini (1996) points out that the “concept” of project complexity has not received adequate attention despite the widespread reference to project complexity. Several researchers have studied the behavior of complex systems, (for instance, Lucas, 2000; Baccarini, 1996; Bertelsen, 2003). Williams (2005) points out that a project which is “structurally simple” can tolerate acceleration and change of plans during execution. Similarly, a “complex project” can still follow its original plan, if it does not face any significant outside influences. However, a “structurally complex” project when perturbed can become unstable and difficult to manage.
Lucas (2000) points out that there often exists a bias towards simplification. Complexity thinking allows one to recognise the situations where this approach is invalid, and provides an alternate form of treatment. According to Lucas, the complexity perspective combines three strands of thought: systems thinking (incorporating cybernetics), organic thinking (including evolution) and connectionist thinking (attractor based).
Characteristics of complex projects
Lucas (2000) describes a list of eighteen axioms (or characteristics) that can explain the behavior of complex systems, such as: emergence, self-organization and self-modification. The property of emergence implies that the behavior of a complex system as a whole cannot be deduced by studying its parts (Simon, 1981). Thus a reductionist approach to project management modelling, wherein the dependencies (upon the environment) are ignored for simplicity, is not sufficient to predict project behavior. The self-organization and self-modification properties refer to the ability of a complex system to organize on its own (in response to changes in environment and also to changes in its elements) and to create order through emergent behavior.
Bertelsen (2003) places fourteen of the eighteen characteristics (of Lucas) in three groups to help understand a construction project from a complexity perspective, namely: autonomous agents, undefined values and non-linearity. Bertelsen (2003) also proposes a correspondence between the (lean) TFV model and the concepts from complexity theory as follows: autonomous agents relate to Transformation, non-linearity to Flow and undefined values to Value generation. Bertelsen (2003) explains this analogy as follows: Shingo (1988) observes that one can find emergence in the nature of production process, that is, the behavior or outcome cannot be predicted a priori, but rather arises as a consequence of interactions between multiple independent (or autonomous) agents.
Similarly in a “complex” project, one can find emergence in many outcomes. An example is the emergence of product (project) design from out of multiple solutions, all of which are equally valuable at the initial stages (this is in contrast to a project design imposed from outside without consideration for the consequent behavior of autonomous agents). While self-organization can result in an upward causation (emergence or power asymmetry), it can also result in downward causation where the parts and details learn to adjust with the emergent outcome. Bertelsen (2003) points out that a project, as a complex construction system, does not possess any initial values at the outset. Values are created and attached as the system interacts with itself and its environment and the “optimal solution” must be found each time. On a different plane, a project, viewed as one of the agents, together with the other projects within the construction industry, interact with one another as autonomous agents in a larger system and within themselves as adaptive agents. In the larger system, each project (as agent) represents a structure in time and space. The ongoing competition and development of newer methods of management in this system allows the emergence of increasing order, generating value for the client.
Defining project complexity
An attempt to define “project complexity” is inevitably influenced by the ontological stance of the researcher. For instance, Baccarini (1996) examines two distinct perspectives of project complexity, namely the “systems theory” perspective and the “difficulty” perspective. From the systems theory viewpoint, complexity can be operationalized in terms of differentiation and interdependency. Williams (1999) refers to this dimension of complexity as structural complexity. Baccarini (1996) shows that his definition of complexity can be applied to any project dimension such as organization, technology, environment, information, decision making and systems. He also elaborates upon two types of project complexity, namely, organizational complexity and technological complexity. Thompson (1967) defines reciprocal interdependencies as those in which output of each element becomes inputs for other elements, so the actions of each element must be modified in response to the actions of the others. In the context of construction projects, reciprocal interdependencies represent the highest level of complexity (both organizational and technological) and dominate the process (Walker, 1989). They produce feedback effects that intensify complexity (Williams, 2002).
From the viewpoint of “difficulty”, the meaning of complexity is subjective and open to diverse interpretations (Baccarini, 1996). While some researchers have attempted to operationalize project complexity based on “difficulty factors”, Baccarini, (1996) finds that, this is an unreliable basis for research and in many cases the meaning of complexity is subsumed within the concept of uncertainty.
System boundaries and contingency theory
Williams (2005) points out that the conventional linear approach to managing project often ignores the complex influences of the environment and the interactions between the project elements. Jensen et al.(2006) find that the environmental interactions generate uncertainty for the actors in the choice of methods used to accomplish the tasks, and ultimately the project performance outcomes. Engwall (2003) stresses upon the need to consider the project in its entirety in relation to the organizational context. Engwall (2003) finds that a “lonely” perspective dominates the extant literature. He also finds a small, but growing, body of studies incorporating complexity of project behavior that is providing an alternative approach. Still, studies with open systems and systemic interactions (both within the project elements as well as with the external environment) need further attention. Engwall (2003) calls for an ontological shift from projects as lonely and closed systems (devoid of history, context and future) to conceptualizations as contextually embedded open systems, open in time as well as in “space”. Researchers find that “projects have less characteristics in common than previously considered” (Dvir, et al., 1998). This indicates that the project management methods that are successful in one context may not be necessarily so in a different context.
Nonlinearity
In a complex project, one can find non-linearity of outcomes. For example, since every project is new, even small differences (project’s attractors) between stakeholders can lead to substantially different solutions (or project designs). Even differences in the initial conditions can contribute to this chaos. Changes can take place even during execution, and complex projects are typically affected by deviations from plans. The temporary nature of the project organization may also make it unstable.
Nonlinearity occurs when the systemic interactions could not have been predicted at the time of its design. Chapman and Ward (1997) finds that ‘standard’ management control processes are appropriate for dealing with “known problems” and risk management strategy can be directed to control “known unknowns”. When “unknown unknowns” cause unplanned outcomes to occur (leading to deviations from plan), the reaction of management is often to perceive the project as being “out of control”. Management may attempt to regain control through interventions like new reporting responsibilities, removal of some team members or even the project manager. Bourne and Walker (2005) point out that such actions result in greater turmoil and instability since the project environment and original baselines have already gone awry, and control based on deviations from original plans are no longer valid in context.
A project as a chaordic system
When management action based on “linear thinking” gives rise to predictable results, there exists some order in the system, and when such action results in unpredictable behavior, the system is in some kind of disorder or chaos. However, a system may not be either completely predictable or chaotic. Since we do not know a priori which actions will lead to which outcome, it appears that such a system is in a state of both order and disorder at the same time (Heylighen, 1988).
Complexity theory researchers (Van Eijnatten, 2004; Bourne & Walker, 2005) suggest that the concept of chaordic systems thinking can be useful in understanding such systems. Three characteristics of the term ‘chaordic’ can be found in van Eijnatten (2004). He defines ‘chaordic’ as anything that is: (1) both orderly and chaotic at the same time, (2) has a pattern dominated by neither order nor chaos, and (3) exists between order and chaos.
When one looks back (over time) at such system responses, seemingly chaotic situations (that is, unpredictable behavior when looked at from a linear framework) appear to have some underlying logic (and therefore some order and more importantly perhaps some predictability, although not necessarily linear) from a perspective of chaordic systems. The search for such an underlying logic is broadly categorized as “sense making” by some researchers (for example, Van Eijnatten, 2004; Winter et al., 2006).
Uncertainty and complexity
Tatikonda and Rosenthal (2000) define technology novelty (in product development projects) as the newness (as opposed to familiarity) to the organization of the technologies employed in the product development effort. Higher level of technology novelty leads to greater task uncertainty. Tatikonda and Rosenthal (2000) also suggest that project size captures only part of the project complexity. A large project may have a simplifying modular design while a small project may become complex because of its integrated product design. Further, external factors like aggressive performance objectives (such as unit-cost and time-to-market) can also contribute to project complexity.
We find that their definition of technology novelty implicitly considers the experience of the project organization in handling new technology. Hence, this novelty is not an intrinsic attribute of the project. We postulate that such novelty emerges out of the interaction between the technology content of the current project and the organizational experience in terms of related technology. Thus, higher levels of technology novelty may not axiomatically lead to greater task uncertainty since the task uncertainty caused by such newness is not due to the novelty per se, rather it is due to limited organizational capability in managing this newness. We propose that a project can be termed complex only if the project organization faces difficulty in managing and executing it. This implies that improving the organizational capability in managing novelty can possibly mitigate the complexity (in terms of task uncertainty) induced by novelty.
Williams (2002, 2005) has contributed significantly to the understanding and modelling of complexity in projects. Williams (2005) proposes that project complexity can be characterized by two dimensions. The complexity arising out of the “underlying structure” of the project is called structural complexity. The second dimension of complexity is uncertainty. Structural complexity is influenced by size in terms of number of elements and interdependence of these elements. Uncertainty can be in goals and methods. The structural complexity view closely follows that of Baccarini (1996) (differentiation and interdependency), while the uncertainty view follows the goals and methods matrix of Turner and Cochrane (1993).
We find that the terms “uncertainty” and “complexity” have been given different meanings by different researchers. For instance, while Williams (2005) considers uncertainty as one of the dimensions of complexity, Tatikonda and Rosenthal (2000) propose that complexity is a contributor to uncertainty. On the other hand, Baccarini(1996) emphasizes that the concept of complexity is different from characteristics such as size and uncertainty.
These widely differing views indicate the lack of consensus on whether complexity causes uncertainty or uncertainty causes complexity. While Williams (2005) suggests that uncertainty adds to the complexity and therefore must be viewed as a constituent dimension of complexity, the alternative view provided by other researchers (such as Baccarini, 1996; Shenhar & Dvir, 1996) is that complexity and uncertainty are two separate concepts. Williams (2005) attempts to reconcile the viewpoints by suggesting that while the other researchers use the term complexity to include only “structural complexity”, his view integrates the influence of uncertainty (as a separate dimension) upon the intrinsic structural complexity, together producing an overall “messiness” that he chooses to call “complexity in this overarching sense”. Williams (2005) notes that the term uncertainty includes both aleotoric and epistemic uncertainties. Aleotoric uncertainties can be mitigated through contingency planning (as in reliability calculations), while epistemic uncertainties result from a lack of knowledge (unforeseen events) leading to project complexity.
De Meyer et al. (2002) differentiates between uncertainties caused by variations, foreseen uncertainty, unforeseen uncertainty and chaos. Variations arise from influences that are too small to plan for and monitor individually. Foreseen uncertainties are identifiable and their influences understood and can be managed by contingency plans. Unforeseen uncertainty, by definition, cannot be identified at time of planning, and so there is no contingency plan in place. Typically, these are caused by “unanticipated interaction of multiple events”. Williams (2005) points out that incorporation of systemicity can help, in principle, to move the “unforeseen events” into the realm of “foreseen events”. However, projects subject to chaos cannot be thus modelled. De Meyer et al. (2002) distinguish between projects subject to unforeseen uncertainty as those that start out with reasonably stable assumptions and goals, and projects subject to chaos as those that do not. In projects subject to chaos, even the “basic structure” of the project plan is uncertain. They point out that contingency plans are insufficient in this case because there could be fundamental changes in the project structure that can redefine the entire project. The project may end up with final results that are vastly different from the original intent.
Implications of the complexity framework
It is thus seen that a project cannot be viewed independent of its surrounding context as well as its history. Further, an understanding of the context is in itself not sufficient to prescribe a method (as posited by the project typology approach). Rather, the method to manage the project is embedded in the context and one must allow the emergence of such a method through interaction between the actors and the environment.
The coexistence of predictable and unpredictable elements in a project implies that the management must judiciously balance planning and control with freedom and creativity (Parellada, 2002). A syncretic worldview can be restrictive while a complexity approach can sophisticate our scope (Najmanovich, 2002). Since complexity is incompressible (Cilliers, 2000), any interpretation will always involve a reduction in complexity (Cilliers, 2002). Cilliers (2002) emphasizes the need to acknowledge the dialectical relationship between the knowledge (method) and the system (context), and the importance of codetermination. Since no two project environments are exactly alike, one cannot have an a priori method to achieve the end results for a project. Rather, the manager must be sensitive to the context, accept diversity of views and methods, and allow the method to emerge from the interactions between the project and the context.
Conclusions
Research on the “theory of projects” is seen to span the entire spectrum of perspectives. Towards one end, the view is that the theory of projects is well defined and understood, while on the other end researchers are debating the existence of any theory of projects. Between these dichotomous positions, many researchers find that the existing theory is nascent and needs further research to render it mature and adequate for practical purposes.
Several researchers have argued for the incorporation of complexity into the framework of project management. The nebulous nature of complexity has attracted discourses that are often abstract and far removed from the real world of project management. Concerns about the inadequacy of the existing theory in the management of projects and the concerted research efforts directed at alternate paradigms based on complexity theory is a more recent phenomenon. Such research suggests that there is no universal theory for projects and one can explore multiple conceptualizations (complementing as well as competing) to explain or predict the behavior of projects.
An outcome of ongoing research efforts is the trend towards developing models that are descriptive rather than prescriptive. Such researchers advocate that the appropriate management response (and hence effective project leadership) is to allow the system to “make sense” leading it towards “orienteering” and “path finding” rather than seeking control. An implication of such environmental complexity is that project managers must learn to establish and maintain relationships with multiple stakeholders, both within and beyond the project management organization (Bourne & Walker, 2005). Thus, the management style must emphasise the shift from control to emergence.