A Methodology for Operationalizing Enterprise Architecture and Evaluating Enterprise IT Flexibility

A Methodology for Operationalizing Enterprise Architecture and Evaluating Enterprise IT Flexibility by Alan MacCormack, Robert Lagerstrom, and Carliss Y. Baldwin

We propose a network-based methodology for operationalizing enterprise architecture. Our methodology is based upon using a “Design Structure Matrix” (DSM) to capture the coupling between different components in a firm’s architecture, including business and technology-related aspects. We apply our methodology to data gathered in a large pharmaceutical firm. We show that this methodology helps to identify layers in the firm’s architecture associated with different technologies (e.g., applications, servers and databases). We also show that it reveals the main “flow of control” within the architecture, as denoted by the classification of components into Core, Peripheral, Shared and Control elements. We analyze the cost of change for a subset of software applications within this architecture. We find that the cost of change is associated with the degree to which applications are highly coupled. We show the best measure of coupling that predicts the cost of change is one that captures all the direct and indirect connections between components. We believe our work constitutes an important step in making the concept of enterprise architecture more operational, improving a firm’s ability to analyze its architecture, understand its performance implications, and adapt and improve it in the future.

Read Paper

Contact the Authors:

Design Capital and Design Moves: The Logic of Digital Business Strategy

Design Capital and Design Moves: The Logic of Digital Business Strategy by C. Jason Woodward, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy

As information technology becomes integral to the products and services in a growing range of industries, there has been a corresponding surge of interest in understanding how firms can effectively formulate and execute digital business strategies. This fusion of IT within the business environment gives rise to a strategic tension between investing in digital artifacts for long-term value creation and exploiting them for short-term value ap- propriation. Further, relentless innovation and competitive pressures dictate that firms continually adapt these artifacts to changing market and technological conditions, but sustained profitability requires scalable archi- tectures that can serve a large customer base and stable interfaces that support integration across a diverse ecosystem of complementary offerings. The study of digital business strategy needs new concepts and methods to examine how these forces are managed in pursuit of competitive advantage. We conceptualize the logic of digital business strategy in terms of two constructs: design capital (i.e., the cumulative stock of designs owned or controlled by a firm) and design moves (i.e., the discrete strategic actions that enlarge, reduce, or modify a firm’s stock of designs). We also identify two salient dimensions of design capital, namely, option value and technical debt. Using embedded case studies of four firms, we develop a rich conceptual model and testable propositions to lay out a design-based logic of digital business strategy. This logic highlights the interplay be- tween design moves and design capital in the context of digital business strategy and contributes to a growing body of insights that link the design of digital artifacts to competitive strategy and firm-level performance.

Read Paper

Contact the Authors:

Technical Debt and the Reliability of Enterprise Software Systems: A Competing Risks Analysis

Technical Debt and the Reliability of Enterprise Software Systems: A Competing Risks Analysis by Narayan Ramasubbu and Chris F. Kemerer

Enterprise software systems are required to be highly reliable as they are central to the business operations of most firms. However, configuring and maintaining these systems can be highly complex, making it challenging to achieve high reliability. Resource constrained software teams facing business pressures can be tempted to take design shortcuts in order to deliver business functionality more quickly. These design shortcuts and other maintenance activities contribute to the accumulation of technical debt, that is, a buildup of software maintenance obligations that need to be addressed in the future. We model and empirically analyze the impact of technical debt on system reliability by utilizing a longitudinal dataset spanning the 10 year lifecycle of a commercial enterprise system deployed at 48 different client firms. We use a competing risks analysis approach to discern the interdependency between client and vendor maintenance activities. This allows us to assess the effect of both problematic client modifications (client errors) and software errors present in the vendor-supplied platform (vendor errors) on system failures.

We also examine the relative effects of modular and architectural maintenance activities undertaken by clients in order to analyze the dynamics of technical debt reduction. The results of our analysis first establish that technical debt decreases the reliability of enterprise systems. Second, modular maintenance targeted to reduce technical debt was about 53% more effective than architectural maintenance in reducing the probability of a system failure due to client errors, but had the side-effect of increasing the chance of a system failure due to vendor errors by about 83% more than did architectural maintenance activities. Using our empirical results we illustrate how firms could evaluate their business risk exposure due to technical debt accumulation in their enterprise systems, and assess the estimated net effects, both positive and negative, of a range of software maintenance practices. Finally, we discuss implications for research in measuring and managing technical debt in enterprise systems.

Read Paper – Forthcoming in Management Science 2015

Contact the Authors:

  • Narayan Ramasubbu (narayanr@pitt.edu)
  • Chris F. Kemerer (ckemerer@katz.pitt.edu)

Design Rule Spaces: A New Form of Architecture Insight

Design Rule Spaces: A New Form of Architecture Insight by Lu Xiao, Yuanfang Cai, and Rick Kazman

In this paper, we investigate software architecture as a set of overlapping design rule spaces, formed by one or more structural or evolutionary relationships and clustered using our design rule hierarchy algorithm. Considering evolutionary coupling as a special type of relationship, we investigated (1) whether design rule spaces can reveal structural relations among error-prone files; (2) whether design rule spaces can reveal structural problems contributing to error-proneness. We studied three large-scale open source projects and found that error-prone files can be captured by just a few design rule sub-spaces. Supported by our tool, Titan, we are able to flexibly visualize design rule spaces formed by different types of relationships, including evolutionary dependencies. This way, we are not only able to visualize which error-prone files belong to which design rule spaces, but also to visualize the structural problems that give insight into why these files are error prone. Design rule spaces provide valuable direction on which parts of the architecture are problematic, and on why, when, and how to refactor.

Read Paper

Understanding the Effects of Product Architecture on Technical Communication in Product Development Organizations

Understanding the effects of product architecture on technical communication in product development organizations by Manuel E. Sosa, Steven D. Eppinger, and Craig M. Rowles

This paper examines the impact of architectural decisions on the level of defects in a product. We view products as collections of components linked together to work as an integrated whole. Previous work has established modularity (how decoupled a component is from other product components) as a critical determinant of defects, and we confirm its importance. Yet our study also provides empirical evidence for a relation between product quality and cyclicality (the extent to which a component depends on itself via other product components). We find cyclicality to be a determinant of quality that is distinct from, and no less important than, modularity. Extending this main result, we show how the cyclicality–quality relation is affected by the centrality of a component in a cycle and the distribution of a cycle across product modules. These findings, which are based on analysis of open source software development projects, have implications for the study and design of complex systems.

Read Paper

Linking Cyclicality and Product Quality

Linking Cyclicality and Product Quality by Manuel Sosa, Jürgen Mihm, and Tyson Browning

Effective communication in product development organizations is widely recognized to be a key element of product development performance. Furthermore, management of product architecture knowledge by the development organization provides important competitive advantage for established firms facing architectural innovation. This research studies how the combination of product architecture and organizational structure determines technical communication in development teams. By documenting and analyzing both the design interfaces between the components that comprise a product and the technical interactions between the teams that design each of these components, we learn how the architecture of the product and the layout of the organization drive development team interactions. Several hypotheses are formulated to explain the unexpected cases when: 1) known design interfaces are not matched by team interactions, and 2) observed team interactions are not predicted by design interfaces. We test the hypothesized effects due to organizational and system boundaries, and design interface strength. Hypotheses are tested using both categorical data analysis and log-linear network analysis. The research is conducted using data collected describing a large commercial aircraft engine development process.

Read Paper

Factors that Influence Technical Communication in Distributed Product Development: An Empirical Study in the Telecommunications Industry

Factors that influence technical communication in distributed product development : an empirical study in the telecommunications industry by Manuel E. Sosa, Steven D. Eppinger, Michael Pich, David G. McKendrick, and Suzanne K. Stout

Understanding the communication process in product development organizations has been recognized as a key element to improve product development performance. It is particularly interesting to study information exchanges in geographically distributed product development teams because of the highly interdependent nature of design organizations. AAdditionally, the use of electronic-based communication media has changed how development teams communicate. By studying the way product development teams use various communication media (face-to-face, telephone, media), we assess how the process of exchanging technical information is influenced by factors such as geographic dispersion, organizational bonds, and degree of team interdependence. We develop a theoretical framework that allows us to formulate several hypotheses about how these factors influence both communication frequency and media choice. We use empirical evidence from the telecommunications industry to test our hypotheses. We confirm previous results about the obstructive influence of distance on technical communication. However, we found that such negative effects may be mitigated by other factors such as the recognizing of highly interdependent team members, the existence of strong organizational bonds, and the use of electronic communication media.

Read Paper

Degree Distribution and Quality in Complex Engineered Systems

Degree Distribution and Quality in Complex Engineered Systems by Manuel Sosa, Jürgen Mihm and Tyson Browning

Complex engineered systems tend to have architectures in which a small subset of components exhibits a disproportional number of linkages. Such components are known as hubs. This paper examines the degree distribution of systems to identify the presence of hubs and quantify the fraction of hub components. We examine how the presence and fraction of hubs relate to a system’s quality. We provide empirical evidence that the presence of hubs in a system’s architecture is associated with a low number of defects. Furthermore, we show that complex engineered systems may have an optimal fraction of hub components with respect to system quality. Our results suggest that architects and managers aiming to improve the quality of complex system designs must proactively identify and manage the use of hubs. Our paper provides a data-driven approach for identifying appropriate target levels of hub usage.

Read Paper

The Impact of Component Modularity on Design Evolution: Evidence from the Software Industry

The Impact of Component Modularity on Design Evolution: Evidence from the Software Industry by Alan MacCormack, John Rusnak and Carliss Y. Baldwin

Much academic work asserts a relationship between the design of a complex system and the manner in which this system evolves over time. In particular, designs which are modular in nature are argued to be more “evolvable,” in that these designs facilitate making future adaptations, the nature of which do not have to be specified in advance. In essence, modularity creates “option value” with respect to new and improved designs, which is particularly important when a system must meet uncertain future demands.

Despite the conceptual appeal of this research, empirical work exploring the relationship between modularity and evolution has had limited success. Three major challenges persist: first, it is difficult to measure modularity in a robust and repeatable fashion; second, modularity is a property of individual components, not systems as a whole, hence we must examine these dynamics at the microstructure level; and third, evolution is a temporalphenomenon, in that the conditions at time t affect the nature of the design at time t+1, hence exploring this phenomenon requires longitudinal data.

In this paper, we tackle these challenges by analyzing the evolution of a successful commercial software product over its entire lifetime, comprising six major “releases.” In particular, we develop measures of modularity at the component level, and use these to predict patterns of evolution between successive versions of the design. We find that modularity has a strong and unambiguous impact on design evolution. Specifically, we show that i) tightly-coupled components are “harder to kill,” in that they have a greater likelihood of survival in subsequent versions of a design; ii) tightly-coupled components are “harder to maintain,” in that they experience more surprise changes to their dependency relationships that are not associated with new functionality; and iii) tightly-coupled components are “harder to augment,” in that the mix of new components added in each version is significantly more modular than the legacy design.

Read Paper

Evolution Analysis of Large-Scale Software Systems Using Design Structures Matrices and Design Rule Theory

Evolution Analysis of Large-Scale Software Systems Using Design Structure Matrices and Design Rule Theory by Matthew J. LaMantia, Yuanfang Cai, Alan David MacCormack and John Rusnak

Designers often seek modular architectures to better accommodate expected changes and to enable parallel development. However, we lack a formal theory and model of modularity and software evolution, which can be used for description, prediction, and prescription. According to Baldwin and Clark’s  theory, modular architectures add value to system designs by creating options to improvethe system by substituting or experimenting on individual modules. In this paper, we evaluate their theory by looking at the design evolution of two software product platforms through the modeling lens of design structure matrices (DSMs) and design rule theory. Our analysis shows that DSM models and options theory can explain how real-world modularization activities in one case allowed for different rates of evolution in different software modules and in another case conferred distinct strategic advantages on a firm (by permitting substitution of an at-risk software module without substantial change to the rest of the system). The experiment supports our hypothesis that these formal models and theory can account for important aspects of software design evolution in large-scale systems.

Read Paper

Visualizing and Measuring Software Portfolio Architectures: A Flexibility Analysis

Visualizing and Measuring Software Portfolio Architectures: A Flexibility Analysis by Robert Lagerstrom, Carliss Y. Baldwin, Alan MacCormack, and David Dreyfus

In this paper, we test a method for visualizing and measuring software portfolio architectures and use our measures to predict the costs of architectural change. Our data is drawn from a biopharmaceutical company, comprising 407 architectural components with 1,157 dependencies between them. We show that the architecture of this system can be classified as a “core-periphery” system, meaning it contains a single large dominant cluster of interconnected components (the “Core”) representing 32% of the system. We find that the classification of software applications within this architecture, as being either Core or Peripheral, is a significant predictor of the costs of architectural change. Using OLS regression models, we show that this measure has greater predictive power than prior measures of coupling used in the literature.

Read Paper

Visualizing and Measuring Enterprise Architecture: An Exploratory BioPharma Case

Visualizing and Measuring Enterprise Architecture: An Exploratory BioPharma Case by Robert Lagerstrom, Carliss Baldwin, Alan MacCormack and David Dreyfus

We test a method that was designed and used previously to reveal the hidden internal architectural structure of software systems. The focus of this paper is to test if it can also uncover new facts about the components and their relationships in an enterprise architecture, i.e., if the method can reveal the hidden external structure between architectural components. Our test uses data from a biopharmaceutical company. In total, we analyzed 407 components and 1,157 dependencies. Results show that the enterprise structure can be classified as a core-periphery architecture with a propagation cost of 23%, core size of 32%, and architecture flow through of 67%. We also found that business components can be classified as control elements, infrastructure components as shared, and software applications as belonging to the core. These findings suggest that the method could be effective in uncovering the hidden structure of an enterprise architecture.

Read Paper

Designing Systems for Adaptability by Means of Architecture Options

Designing Systems for Adaptability by Means of Architecture Options by Avner Engel, and Tyson R. Browning

The value of a system usually diminishes over its lifetime, but some systems depreciate more slowly than others. Diminished value is due partly to the increasing needs and wants of the system’s stakeholders and partly to its decreasing capabilities relative to emerging alternatives. Thus, systems are replaced or upgraded at substantial cost and disruption. If a system is designed to be changed and upgraded easily, however, this adaptability may increase its lifetime value. How can adaptability be designed into a system so that it will provide increased value over its lifetime? This paper describes the problem and an approach to its mitigation, adopting the concept of real options from the field of economics, extending it to the field of systems architecture, and coining the term architecture options for this next-generation method and the associated tools for design for adaptability. Architecture options provide a quantitative means of optimizing a system architecture to maximize its lifetime value. This paper provides two quantitative models to assess the value of architecture adaptability. First, we define three metrics—component adaptability factors, component option values, and interface cost factors—which are used in a static model to evaluate architecture adaptability during the design of new systems. Second, we enhance a dynamic model to evaluate architecture adaptability over the maintenance and upgrade lifetime of a system, formulating a Design for Dynamic Value (DDV) optimization model. We illustrate both models with quantitative examples and also discuss how to obtain the socio-economic data required for each model.

Read Paper

A Decision-Support System Approach to Economics-Driven Modularity Evaluation

A Decision-Support System Approach to Economics-Driven Modularity Evaluation by Yuanfang Cai, Rick Kazman, Carlos V.A. Silva, Lu Xiao, and Hong-Mei Chen

Our research goal is to develop a decision-support system that would allow managers to play out various “what-if” scenarios to manage modularity debt and make informed decisions regarding refactoring. A decision-support system approach would appropriately address the need to simulate and visualize scenarios that include many different (current and future) factors under different assumptions that may affect the decision outcome. Our proposed system is called the modularity debt management decision-support system (MDM-DSS). Our MDM-DSS is built on a scientific foundation that we are constructing for explicitly manifesting the economic implications of software modularization activities, so that the costs and the benefits of such activities can be understood, analyzed, and predicted in a way that is amenable to both engineers and managers. The research questions being addressed include: how to manifest the costs and benefits of modularization activities as functions of modularity variations; how to locate the components with the most uncertainty and risk; how to determine that there is a debt; how to quantitatively account for the relation between modularization activities and their economic consequences; and, in particular, how to manifest the value of modularity as options so as to plan an optimal strategy of evolution based on viewing software as investment?

In addressing these research questions, we have developed an integrated economics-driven modularization evaluation (IEDME) framework that combines code-level analysis, design-level analysis, expert stakeholder elicitation, and history-based cost-benefit analysis. This framework links measures of modularity with quantified economic benefits—that is, maintenance cost-savings—to serve as the foundation of our target MDM-DSS. In what follows, we present the design considerations for the MDM-DSS.

Read Paper

Modeling Impacts of Process Architecture on Cost and Schedule Risk in Product Development

Modeling Impacts of Process Architecture on Cost and Schedule Risk in Product Development by Tyson R. Browning and Steven D. Eppinger

To gain competitive leverage, firms that design and develop complex products seek to increase the efficiency and predictability of their development processes. Process improvement is facilitated by the development and use of models that account for and illuminate important characteristics of the process. Iteration is a fundamental but often unaddressed feature of product development (PD) processes. Its impact is mediated by the architecture of a process, i.e., its constituent activities and their interactions. This paper integrates several important characteristics of PD processes into a single model, highlighting the effects of varying process architecture. The PD process is modeled as a network of activities that exchange deliverables. Each activity has an uncertain duration and cost, an improvement curve, and risks of rework based on changes in its inputs. A work policy governs the timing of activity execution and deliverable exchange (and thus the amount of activity concurrency). The model is analyzed via simulation, which outputs sample cost and schedule outcome distributions. Varying the process architecture input varies the output distributions. Each distribution is used with a target and an impact function to determine a risk factor. Alternative process architectures are compared, revealing opportunities to trade cost and schedule risk. Example results and applications are shown for an industrial process, the preliminary design of an uninhabited combat aerial vehicle. The model yields and reinforces several managerial insights, including: how rework cascades through a PD process, trading off cost and schedule risk, interface criticality, and occasions for iterative overlapping.

Read Paper

Measuring the Life-Cycle Value of Enduring Systems

Measuring the Life-Cycle Value of Enduring Systems by Tyson R. Browning, and Eric C. Honour

A goal of systems development is to produce enduringly valuable product systems— i.e., systems that are valuable when delivered to their users and which continue to be attractive to their stakeholders over time. However, quantifying the life-cycle value (LCV) provided by a system has proven elusive. In this paper, we propose an approach to quantifying a system’s LCV based on the key parameters that have perceived value to the system’s stakeholders. For this, we draw upon insights from the management, marketing, product development, value engineering, and systems engineering literature. We then demonstrate our proposed approach with an example of a cellular telephone system. By designing systems for maximum LCV, systems architects and engineers will provide dramatically increased value to their organizations and other stakeholders. However, to provide maximum LCV, a system may need to be designed to facilitate adaptability to changing circumstances and stakeholder preferences. We conclude the paper with discussions of some of the major difficulties in measuring LCV and some of the opportunities for further research in this area.

Read Paper

The Architecture of Platforms: A Unified View

The Architecture of Platforms: A Unified View by Carliss Y. Baldwin and C. Jason Woodard

The central role of “platform” products and services in mediating the activities of disaggregated “clusters” or “ecosystems” of firms has been widely recognized. But platforms and the systems in which they are embedded are very diverse. In particular, platforms may exist within firms as product lines, across firms as multi-product systems, and in the form of multi-sided markets. In this paper we argue that there is a fundamental unity in the architecture of platforms. Platform architectures are modularizations of complex systems in which certain components (the platform itself) remain stable, while others (the complements) are encouraged to vary in crosssection or over time. Among the most stable elements in a platform architecture are the modular interfaces that mediate between the platform and its complements. These interfaces are even more stable than the interior core of the platform, thus control over the interfaces amounts to control over the platform and its evolution. We describe three ways of representing platform architectures: network graphs, design structure matrices and layer maps. We conclude by addressing a number of fundamental strategic questions suggested by a unified view of platforms.

Read Paper

Organization Design for Distributed Innovation

Organization Design for Distributed Innovation by Carliss Y. Baldwin

Systems of distributed innovation – so-called business ecosystems – have become increasingly prevalent in many industries. These entities generally encompass numerous corporations, individuals, and communities that might be individually autonomous but related through their connection with an underlying, evolving technical system. In the future, I believe the key problem for organization design will be the management of distributed innovation in such dynamic systems. Organization designers must think about how to distribute property rights, people, and activities across numerous self-governing enterprises in ways that are advantageous for the group as well as for the designer’s own firm or community.

Read Paper

Hidden Structure: Using Network Methods to Map System Architecture

“Hidden Structure: Using Network Methods to Map System Architecture” by Carliss BaldwinAlan MacCormack and John Rusnak

In this paper, we describe an operational methodology for characterising the architecture of complex technical systems and demonstrate its application to a large sample of software releases.  Our methodology is based upon directed network graphs, which allows us to identify all of the direct and indirect linkages between the components in a system. We use this approach to define three fundamental architectural patterns, which we label core-periphery, multi-core, and hierarchical. Applying our methodology to a sample of 1,286 software releases from 17 applications, we find that the majority of releases possess a “core-periphery” structure. This architecture is characterized by a single dominant cyclic group of components (the “Core”) that is large relative to the system as a whole as well as to other cyclic groups in the system. We show that the size of the Core varies widely, even for systems that perform the same function. These differences appear to be associated with different models of development—open, distributed organizations develop systems with smaller Cores, while closed, co-located organizations develop systems with larger Cores. Our findings establish some “stylized facts” about the fine-grained structure of large, real-world technical systems, serving as a point of departure for future empirical work.

Read Paper

Managing Technical Debt in Enterprise Software Packages

Managing Technical Debt in Enterprise Software Packages by Narayan Ramasubbu and Chris F. Kemerer

We develop an evolutionary model and theory of software technical debt accumulation to facilitate a rigorous and balanced analysis of its benefits and costs at different lifecycle stages of a large commercial enterprise software package. Our theory focuses on the optimization problem involved in managing technical debt, and illustrates the different tradeoff patterns between customer satisfaction and software quality under early and late adopter scenarios at different lifecycle stages of the software package. We empirically verify our theory utilizing a 10 year longitudinal dataset drawn from 69 customer installations of the software package. We then utilize the empirical results to develop actionable policies for managing technical debt in enterprise software product adoption.

Read Paper