How should a manager, software architect, designer, programmer, team lead or product manager,act to organize and manage high tech systems development projects? My goal is to help form answers to this and to the following questions:
- What management techniques, practices, lifecycles and frameworks are applied in contemporary systems development projects?
- How might we integrate the diverse concepts and theories of software systems development, and translate these concepts into personal, team, and management practice?
- How is value generated and delivered in real situations?
- What is the significance of new engineering approaches, from agile and lean methods to more functional approaches like CMMI and RUP?
- How do lifecycles and methodologies balance the tension between a necessity for orderly production and quick responses to changing contexts?
The overarching objective for this work it is to show that the generation of any or all system artifacts is an outcome of underlying social interactions such as joint development, peer review, user involvement, automated testing, use assurance. Consequently system development’s process and its context is itself a kind of social structure for managing production (in the development team) and users. Power is an essential and underlying concept in order to understand the 'assumed' orderly underlying social interactions of programming teams and others they need to deal with in order to 'get the work done'.
OBJECTS OF DEVELOPMENT
Our capability to successfully create and utilise high tech products is linked with a variety of challenges across a number of diverse domains; organisational theory and sociology for management and societal use patterns; economics and systems theory to understand aggregate behaviour; physiology, psychology and ethnography to understand use factors and design for finished products; maths, materials, and physics, for microprocessor and computational innovation.
Let’s take a closer look at the objects of software, the expressive environment for software engineers. The following illustrations are representations of different aspects of a not untypical digital product (Figure below). In this case a rendered simulation of an apartment, a wireframe model of the same scene, the software services for configuring the simulation, an XML description of objects present in the simulation, the build log for compiling the simulation and some source code for one of the objects. Furthermore there are many more layers of source and representation with their related technologies associated with this particular product.
Figure Images from Microsoft Robotics Developer Studio 2010
These are all ‘representations of’ and ‘resources for’ an interleaved complexly reinforcing structure that can be un-picked or un-packed right down to single lines of code. The point of this presentation is to illustrate the argument that “developers solve problems at all levels, between the ‘whole project level’ and the ‘one line of code’ level (and everything in between.” (Racoon, 1995)
Figure Layered perspective of digital production
The many layers of digital production are interleaved and interconnected from the single line of code right through to the whole project as Racoon (1995) suggests. This may account for both the brittleness of digital systems and their complex resilience. The same work dynamic occurs across large and small teams, across large and small organisations, across the world and is an important aspect of the defining qualities of software development work. Significantly, the practice of high tech production also depends on creative and collaborative processes; software engineering is a designing profession regardless of whether the engineer is working on a version 1.0 project or maintaining an existing product through multiple generations and update releases. Yes crafting and designing software is a kind of individual contribution, a highly cognitive process, however in practice it is also a highly communicative and essentially social process comprised of many various interactions in a team.
MAKING OUR DIGITAL LIVES
The modern era is increasingly delineated by a ‘digital life’ or form of engagement that entwines complexly with our existence in a physical world. IT and high tech use-production is seen as the enabling force for overcoming immobilities. They are implicated in transforming our social or organizational interactions and generate heightened perceptions of speed, movement, presence, both global and local awareness, and interconnectedness. We introduce the idea of high tech use-production to try to overcome the idea that these transformations result from push processes originating in the development centre.
Why do we need to think about the processes and structures used for developing high tech products and services? It is true that the emphasis on organizing high tech production has shifted focus from gifted individuals to being a team-based mode of production if not a team-of-teams in case of large-scale infrastructural architectures. Engineers do not start and complete whole projects overnight. Producing value takes time, effort and many small instances of failure and learning to create viable systems. It takes time, people and other technologies to produce these things. Managing the process involves making trade-offs between the ordering of events and activities over time, access to other complex systems (software, tools, work techniques) and the services of other actors. Importantly the very idea of a system implies ‘use’ and modern innovation processes are increasingly reliant on lead user involvement to establish both how the system is used and consequently how it is designed (Von Hippel, 2005).
Positioning the role: managing high-tech production
What does a manager need to know about the IT function in order to manage it? The manager of the IT function needs to know how to go about producing the product, and also, how to go about producing users. However, rather than relegating ‘use’ to a phase of implementation and delivery at the end of the systems production cycle, ‘use’ and ‘users’ have become intrinsic to driving the development process. The following highlights ways of producing, delivering and servicing our demands for ICT, IT and high tech goods. We don’t need to be engineers to manage the process, but we do need to be technology savvy and know enough to make a difference; how to bring technology, business and users together. The technology manager’s role is to bridge the two communities of production and use, to translate, interpret and make sense of the gaps between the technological and customer worlds (Figure below).
 |
| From Scrapbook Photos |
A SYSTEMS PERSPECTIVE ON HIGH TECHNOLOGY DEVELOPMENT AND USE
Mid-Range Systems Concepts and drawing the Boundaries of High Tech
Systems development is closely associated with the production of technological artifacts, in particular with computer based high tech products and projects. Systems development is presented as an engineering design method (Gregory and Richard L., 1963, Brooks Jr., 1995). Indeed engineering design has become doubly system-like as systems development concepts were applied both to the design of high tech products and to the organization of the work of high tech production itself. The very idea of systems and systems thinking is so pervasive as to become an intuitive concept. A system can be considered as any assemblage or set of objects (concepts, things, distinctions) connected in some orderly or meaningful arrangement.
“The systems approach to problems focuses on systems taken as a whole because there are some properties of systems that can only be treated from a holistic point of view.” (Ackoff, 1971)Like any powerful concept the idea of system is flexible and adaptable. The questions posed by systems thinking are what elements to include, how to relate them, and where to draw the outline of the ‘whole.’ The boundary (if any) becomes crucial, what is considered inside or outside, is the boundary itself another element or does the idea of boundaries or interfaces even make sense?
THE PROBLEM OF HIGH-TECH SYSTEMS
Why does it take so long to get software ready?The same questions have been raised at every point in the history of computing and are still being raised today. Furthermore they are as relevant now as they were then. However, in 2003, Nicholas Carr famously declared that IT doesn’t matter anymore (Carr, 2003). He suggested that organisations should spend less on IT, copy other organisations’ technology infrastructure rather than generate innovations, and focus on operational improvements rather than creating competitive advantage through radical innovative development. So, if IT doesn’t matter anymore, why is there an on-going problem managing high tech products and IT governance?
Why are development costs so high?
Why can’t we find all the errors before we give the software to our customers?
Why do we continue to have difficulty in measuring progress as software is being developed?
Adapted (Pressman, 2000).
We observe a continuous current of media, practitioner, and research literature reporting the failures of large-scale system implementations (Ruey-Lin, 2006, Krigsman, 2010, Lyytinen and Robey, 1999, Currie and Willcocks, 1998, Keil and Montealegre, 2000). Much of the reportage and research concludes that IT use, production, delivery and management are brittle. IT is incredibly sensitive to situation, context and human factors (Suchman, 1987) and, evidently, as IT shifts across the product/service spectrum from stand-alone product to service rich high tech systems, outcomes become more costly and chaotic (Brooks Jr., 1987).
IT systems have become more complex and therefore more difficult to manage. Project planning and governance is particularly challenging when the subject matter is highly virtualised and intangible. IT systems become organisationally systemic even as system knowledge and skills become more specialised and rare. Various actor strategies and organisational structures may disconnect or invert necessary links between knowledge, responsibility and power.First some definitions. IT (Information Technology), ICT (Information Communications Technology), and High Tech are broad terms and difficult to pin down. For the purposes of these readings we can consider them to be products and services utilising microprocessors and the related objects they entail. High tech systems utilise hardware (electronics) and software (programs), often in the form of computing devices, computers, networks and Internets, although some high tech systems are less evident or may now be considered old tech, e.g. wrist watches.
Adapted (Avison et al., 2006).
The organization of the high tech development and production is naturally important but equally significant is how the development process is used to understand, formulate and resolve the real world needs or problems addressed by high tech products. Product systems are created against a backdrop of our prior assumptions and understanding of what systems actually are. Our understanding of what constitutes a system needs therefore to be clarified, so, what in fact are ‘systems’?
The idea of systems and ‘systems thinking’ is so prevalent as to have become an intuitive concept or perspective of any aspect in the world. The word system is now linked with the environment, ecology, markets, engineering, society, and politics. The OED defines a system as;
“a set or assemblage of things connected, associated, or interdependent, so as to form a complex unity.” (OED, 2010)The system concept is used for operations research to create models; mapping representations of disparate phenomena and functions, presenting them as an organizational whole. In the most general sense the system concept is applied when a complex phenomena is perceived to
“…display the character of being organized.”The appearance of organization, of being organized, is assumed to arise out of interdependencies between underlying processes or other related phenomena. One of the challenges to studying and interacting in complex relational environments is the very act of defining the system.
(Emery and Trist, 1965)
Systems can therefore be understood as descriptions that limit the extent of an enquiry. A system is constructed by description. A system is a more or less exhaustive statement of more or less arbitrary boundaries, inputs, outputs, interactions, behaviour, performance, relationship, of abstract and concrete objects.
At its most general the very idea of a system helps us to organize ideas about the relationship between things. The system concept offers a way of representing, sometimes modeling, an often arbitrary arrangement of elements that exhibit a relationship or correlation of behaviour.
“A system is a set of interrelated elements. Thus a system is an entity which is composed of at least two elements and a relation that holds between each of its elements and at least one other element in the set. Each of a system’s elements is connected to every other element, directly or indirectly. Furthermore, no subset of elements is unrelated to any other subset.” (Ackoff, 1971)A closed system is self-contained with interaction taking place solely among the system’s elements. Delineating the set of elements in a closed system defines the boundary between that system and its surrounding environment. For a closed system the environment acts simply as a source of variables affecting the system’s initial state or condition.
An open system is a system that interacts both internally and with the surrounding environment. The outputs of an open system alter elements in the environment which in turn affect the system’s state over time and so on. Dynamic systems change over time and may both generate and respond to events thus changing even more over time.
Where systems concepts include human actors, as individuals, or organizations and markets, the act of defining the system becomes a subjective exercise that may or may not correspond to obvious or unambiguous distinctions (Avison and Fitzgerald, 2006). Ackoff (1971) presents a hierarchy to classify systems (Figure below). At the lowest level a mechanical system is state-maintaining. A goal-seeking system uses the environment as a source of information to attain a goal. Multi-goal-seeking systems are devices or objects that can be re-tasked to achieve multiple goals such as a general-purpose programmable computer. Purposeful systems involve humans and are intrinsically open, dynamic and unstable. Purposeful systems are subject to changing goals, motivations and relations as the human actor constructs these things through their interaction with/in the system and their constantly evolving interpretation of goals, values, outcomes, identity etc. Purposeful systems may contain all other system types as more or less well-understood elements and objects. Weinberg (1975) characterised types of systems with respect to associated methods we have of dealing with them (Figure below). Organised simplicity: I (machines), unorganised complexity: II (large populations and markets), and organised complexity: III (mid range systems between the extremes of I & II).
Figure: Classifications of system types.
SYSTEMS THINKING
Systems thinking analysis can be used to illustrate and group the problem domain of trade-offs between the scale of a system and its connectedness as depicted below. We can imagine a problem space for systems thinking to vary along the dimensions of complexity and randomness, or as suggested (Figure below), by connectedness and scale (complexity and complication). I have adapted Weinberg’s terminology in this diagram his original axes were x=complexity, y=randomness however the original intent remains the same; to characterize a conceptual problem space between mechanistic models and statistical models which he termed ‘medium number systems’ or ‘organized complexity.’
Different regions in the problem space are amenable to either: analytical, statistical, or (as Weinberg argues) a systems thinking approach. Recalling the trade-off between scale and connectedness. The first category: organised simplicity, describes relatively well-described simple systems that may be modelled analytically or deterministically as machine-like systems. The second category: unorganised complexity may be modelled statistically and applies in situations with sufficiently large, complex populations when aggregate behaviour begins to approximate stochastic (probabilistic) processes. The third category: organised complexity, describes dynamic structure-determined systems as evidenced by emergent properties and behaviour. These are medium number problems that are…
“too complex for analysis and too organized for statistics. This is the region of systems.” (Weinberg, 1975)
Figure: Types of systems with respect to methods of thinking (adapted from Weinberg, 1975)
Mid-range complex systems are typically the subject matter for management, engineering, sociology, and politics. They deal with problems that are dynamic and interrelated but essentially beyond finite analytical modelling because factors cannot be isolated or parameterised, or the computational load of a detailed model is excessive or based on unrealistic assumptions. Yet these problems are not large enough or randomised such that stochastic or probabilistic methods are useful for predicting or managing outcomes.
MID-RANGE COMPLEX SYSTEMS
So what is a mid-range complex system? Weinberg suggests that it is an open evolving system. The assumption of being bounded and approximately mechanical does not hold. It is a structure-determined system that adapts and evolves over time. A mid-range complex system is coupled with its environment and its boundaries are defined by its interfaces; processes of interaction and transformation with the environment it exists in and which it in part constitutes (Winograd and Flores, 1986).
How do system-like effects arise in the environment for high tech product/services? High-tech products typically invoke new system-like behaviour from users that over time begins to pervade the lives of individuals, families and groups in ways that we cannot predict with certainty.
EXAMPLES: The following examples depict the emergence of system-like interaction and behaviour arise between people and high tech objects (credit: Stefan Klein).
The telephone is not just another way to talk to someone over a wire; the mobile phone is not just a traditional telephone without a wire. The telephone, a spatially fixed communications device, could never attain the pervasive availability and use that the mobile phone has achieved. Mobile telephony has led, in turn, to the discovery of novel forms of coordination, collaboration and surveillance. It has redefined the notion of availability, and dramatically extended the scope of options to get information or call for assistance in everyday situations.
RFID technology was designed for simple product identification, but it can also be used for tracing and tracking things, ranging from the identification of licit products to the surveillance of teenagers. The meaning that is assigned to technologies and their modes of use is highly contingent on the context of use, (groups of) users, their relations, etc.
CONSTRUCTING SYSTEM DESCRIPTIONS
There are many tools offered as frameworks or as aides to the job of gathering information and constructing system descriptions. We will look at two simple but useful approaches for describing complex organisational market relationships with technology.
Organigraphs for Describing Organizations
Organigraphs are a tool for drawing and representing the relationships and activities of an organisation. They are an alternative to organisational charts as a tool for mapping an organisational relationships and interactions.
They can be treated as an antidote in some respects to the traditional organisational chart, instead of the familiar hierarchical view depicting names, titles and formal lines of authority.
“Organigraphs have been able to demonstrate how a place works, depicting critical interactions among people, products, and information. …to stimulate conversations about how best to manage their operations and which strategic options make the most sense,” (Mintzberg and Van der Heyden, 1999)Organigraphs are a view of the organisation that can be more aligned with its activities and their interrelationships or workflows. Activities and functions could be thought of as the organs and muscles of an organisation. The act of redrawing the system of the organisation in this way offers a lexicon for represent and redefining organisational action. It provides a “new vocabulary, and associated pictures... [to bring] choices into high relief.” (Mintzberg and Van der Heyden, 1999) The Organigraph Palette consists of: People, Products, and Information (Figure below). Different actors internal and external to the organisation are included. The boundary of the system is loosely defined; in fact the value of depicting organization in this style is that the boundary can be redrawn and rethought.
Figure: Organigraph elements and simple example applied to a newspaper publisher.
The ‘ecology’ of the organization, its products and the environment within which they exist can be highlighted. Furthermore these views depicts people (as individuals, teams, or divisions), products (things, services), and information, equally. There are three basic representations; chains, hubs and webs. Typically nodes will be people or products and arcs or connections will be product movement, transformation or information. The following example illustrates the multiple ways we may represent the product development and support function in an IT organization (figure below).
Figure: Three representations of customer interaction with an organisation
There are no ‘right’ organigraphs, just those that are more or less useful. Remember, the different organisational metaphors are simply useful ways of thinking about organisation. As an analytical approach this kind of organisational mapping can draw out and juxtapose the various connections and flows. It highlights sequential, radial and web-like relations. They allow us the opportunity to redesign the relations, or highlight them so they can be better understood and supported. The graph provides a system-like view of the organisation in its environment and the activity focus within the organisation and for actors outside the organisation.
Activity Theory as a System for Describing Systems
Activity Theory has been employed as a descriptive theoretical framework to assist systems development, for eliciting and refining requirements with the goal of developing prototypes through to new technology in live user settings. It was adapted to analyse user situations and needs when cooperative work settings are reconfigured with computer support. It has strong theoretical constructs, an iconic structure, it focuses on the individual within organisations and society, and its primary unit of analysis is activity. Activity Theory includes the actors and objects involved in the process of someone transforming something into something else. This is the key process of development where the new product embodies or signifies new meaning for a wider community of others (e.g. customers, other members of the organisation). The approach has also been drawn on as a theoretical foundation to design methods like Interaction Design and Participatory Design (Moggridge, 2006, Cooper, 2004, Cooper et al., 2007, Kaptelinin and Nardi, 2006). Researchers have also used it to look at organisational interaction specifically in the contexts of seeking to attain new outcomes in the wider environment. As such activity theory can be used when we attempt to manage the development or intervention in mid-range systems, to innovate social, organizational, and technological processes. It provides a conceptual framework for making sense of how organisational actors can or should go about taking action in situations of changing requirements in dynamic market environments.
Figure: Generic Activity Theory diagram for describing a workplace/setting.
HUMAN SYSTEMS
Human behaviour in groups has proved difficult to manage well using classical scientific methods. Human behaviour is intrinsically complex and revisable. Social phenomena , purposeful systems (Ackoff, 1971), are the edge cases for scientific enquiry (Checkland, 1981). Checkland describes four classes of systems: natural, designed physical, designed abstract, and human activity systems. In spite of the elusive and changing character of purposeful human systems there remains much we can do when acting mindfully in settings to develop new technology and systems.
This issue of boundary setting and description dogged early attempts to construct detailed models of physical processes (e.g. in biology, materials, etc) and led to the idea of closed and open systems. ‘Systems thinking’ is presented as a systematic approach to thinking about ‘wholes’ rather than simplified reductions. By thinking holistically ‘systems thinking’ makes explicit the role of the problem describer (not necessarily the problem setter) as both observer and as an agent actively involved in articulating and refining the problem statements. There are, potentially, many relevant aspects of human activity systems that remain unknown or may be ignored based on how the object of concern and its environment are understood. For example, we generally assume that the outputs (products) of an organization are its primary means for acting in its market or operating environment. However there exist multiple other avenues that are often overlooked: welfare effects from corporate profits, political actions in industry and public forums, marketing, word-of-mouth, ethical and wider corporate governance etc.
The challenge for defining and describing product systems when humans are involved is in determining what we should include (and exclude) from analysis, and therefore, what is understood as relevant ! In essence the description of a complex real world system we be inevitably incomplete. Quantitative models of human activity systems become increasingly sensitive to initial conditions and are vulnerable to unknown factors, in large measure due to “the messy richness of ‘biological effects’” (Checkland, 1981). The problem of complexity is compounded in social studies and organizational behaviour because the social environment and its conditions of possibility are themselves social constructions; where a social construction can be understood as an on-going process of participation in collective sense-making (Giddens, 1984). Therefore, in human systems (organisations, markets, culture, society) – the subject matter for management, engineering and social science – working hypotheses, laws and generalities may hold for periods of time, but they remain open to revision and reinvention. The added complexity of social processes in society, markets, and organisations, and associated problems we hope to resolve, may often work against the rationality of scientific or engineered solutions (experimentation, observation, reduction).
REFERENCES
