A Methodology for Operationalizing Enterprise Architecture and Evaluating Enterprise IT Flexibility by Alan MacCormack, Robert Lagerstrom, and Carliss Y. Baldwin
We propose a network-based methodology for operationalizing enterprise architecture. Our methodology is based upon using a “Design Structure Matrix” (DSM) to capture the coupling between different components in a firm’s architecture, including business and technology-related aspects. We apply our methodology to data gathered in a large pharmaceutical firm. We show that this methodology helps to identify layers in the firm’s architecture associated with different technologies (e.g., applications, servers and databases). We also show that it reveals the main “flow of control” within the architecture, as denoted by the classification of components into Core, Peripheral, Shared and Control elements. We analyze the cost of change for a subset of software applications within this architecture. We find that the cost of change is associated with the degree to which applications are highly coupled. We show the best measure of coupling that predicts the cost of change is one that captures all the direct and indirect connections between components. We believe our work constitutes an important step in making the concept of enterprise architecture more operational, improving a firm’s ability to analyze its architecture, understand its performance implications, and adapt and improve it in the future.
Contact the Authors:
Managing Technical Debt: insights from Recent Empirical Evidence by Narayan Ramasubbu, Chris F. Kemerer, and C. Jason Woodard
Technical debt refers to maintenance obligations that software teams accumulate as a result of their actions. Empirical research has led researchers to suggest three dimensions along which software development teams should map their technical-debt metrics: customer satisfaction needs, reliability needs, and the probability of technology disruption.
Contact the Authors:
Design Capital and Design Moves: The Logic of Digital Business Strategy by C. Jason Woodward, Narayan Ramasubbu, F. Ted Tschang, and V. Sambamurthy
As information technology becomes integral to the products and services in a growing range of industries, there has been a corresponding surge of interest in understanding how firms can effectively formulate and execute digital business strategies. This fusion of IT within the business environment gives rise to a strategic tension between investing in digital artifacts for long-term value creation and exploiting them for short-term value ap- propriation. Further, relentless innovation and competitive pressures dictate that firms continually adapt these artifacts to changing market and technological conditions, but sustained profitability requires scalable archi- tectures that can serve a large customer base and stable interfaces that support integration across a diverse ecosystem of complementary offerings. The study of digital business strategy needs new concepts and methods to examine how these forces are managed in pursuit of competitive advantage. We conceptualize the logic of digital business strategy in terms of two constructs: design capital (i.e., the cumulative stock of designs owned or controlled by a firm) and design moves (i.e., the discrete strategic actions that enlarge, reduce, or modify a firm’s stock of designs). We also identify two salient dimensions of design capital, namely, option value and technical debt. Using embedded case studies of four firms, we develop a rich conceptual model and testable propositions to lay out a design-based logic of digital business strategy. This logic highlights the interplay be- tween design moves and design capital in the context of digital business strategy and contributes to a growing body of insights that link the design of digital artifacts to competitive strategy and firm-level performance.
Contact the Authors:
Technical Debt and the Reliability of Enterprise Software Systems: A Competing Risks Analysis by Narayan Ramasubbu and Chris F. Kemerer
Enterprise software systems are required to be highly reliable as they are central to the business operations of most firms. However, configuring and maintaining these systems can be highly complex, making it challenging to achieve high reliability. Resource constrained software teams facing business pressures can be tempted to take design shortcuts in order to deliver business functionality more quickly. These design shortcuts and other maintenance activities contribute to the accumulation of technical debt, that is, a buildup of software maintenance obligations that need to be addressed in the future. We model and empirically analyze the impact of technical debt on system reliability by utilizing a longitudinal dataset spanning the 10 year lifecycle of a commercial enterprise system deployed at 48 different client firms. We use a competing risks analysis approach to discern the interdependency between client and vendor maintenance activities. This allows us to assess the effect of both problematic client modifications (client errors) and software errors present in the vendor-supplied platform (vendor errors) on system failures.
We also examine the relative effects of modular and architectural maintenance activities undertaken by clients in order to analyze the dynamics of technical debt reduction. The results of our analysis first establish that technical debt decreases the reliability of enterprise systems. Second, modular maintenance targeted to reduce technical debt was about 53% more effective than architectural maintenance in reducing the probability of a system failure due to client errors, but had the side-effect of increasing the chance of a system failure due to vendor errors by about 83% more than did architectural maintenance activities. Using our empirical results we illustrate how firms could evaluate their business risk exposure due to technical debt accumulation in their enterprise systems, and assess the estimated net effects, both positive and negative, of a range of software maintenance practices. Finally, we discuss implications for research in measuring and managing technical debt in enterprise systems.
Read Paper – Forthcoming in Management Science 2015
Contact the Authors:
- Narayan Ramasubbu (email@example.com)
- Chris F. Kemerer (firstname.lastname@example.org)
How to Analyze and Visualize a Large, Interconnected Software System: A Study of Fedora 20 with Lessons for All by Dan Sturtevant, PhD., and Dave Allan
Presented on January 26th, 2015 – YouTube video of presentation available here, presentation slides here
When working inside a system of enormous scale, people often understand only the part with which they are involved. As long as the whole system is sufficiently modular, people tend to believe they can construct reasonably reliable mental models of their component and how it interfaces with others. However, this is not always safe; research shows that hidden structures can interconnect components of a complex system at higher levels, causing organizational problems that are difficult to see, understand, and address.
Fedora 20 is composed of more than 2,500 interconnected software packages developed and managed by globally distributed teams. Estimates have placed the number of software developers who have contributed at over 100,000. In this webinar, Daniel Sturtevant and David Allan will present research that addresses the architectural complexity of the Fedora Linux operating system and software collection. They discuss:
1) How to visualize the system at multiple levels (including the view from 60,000 feet) and gain meaningful insights about its hidden structure,
2) How to benchmark across the system to better understand its composition and variations in complexity and quality, and
3) How this approach might be applied to other software systems.
Contact the Authors:
- Dan Sturtevant, CEO of Silverthread Inc. and Researcher at Harvard Business School
- Dave Allan, Director of Software Engineering Silverthread Inc.
Technical Debt in Large Systems: Understanding the Cost of Software Complexity by Daniel J. Sturtevant
Many modern systems are so large that no one truly understands how they work. Because these systems exceed the bounds of human understanding, different design teams must work on separate chunks, glue their work together, and hope that the whole thing behaves as expected. In this process, high-level architectural design patterns (such as hierarchies, modules, and abstraction layers) play an important role. By keeping complexity under control, they give systems the ability to scale, make them more able to evolve, and reduce the likelihood of unexpected side effects or integration problems.
Technical debt in Firefox and Chromium by Ali Almossawi
In his previous article How maintainable is the Firefox codebase, the author discussed how an important quality attribute, namely, maintainability, could be quantified. It looked at how Firefox was doing with respect to those measures and suggested some preliminary findings. The aim of this second, much shorter, article is to provide better context by comparing Firefox to another popular and complex software system.
Read Full Article
SDM Best Thesis analyzes costs of software system complexity by Andrei Akaikine
Maintaining a complex system at a “good enough” level of functionality comes at a cost. In systems where stability is
paramount, personnel dealing with small emergencies traditionally rely on “Band-Aid” fixes that increase complexity and heighten the risk of destabilizing the whole system with each new modification, however minor.
Andrei Akaikine’s thesis focuses on the following three questions:
• Is there a relationship between engineering efforts required for a basic
maintenance task and the overall software system complexity?
• Are any components more susceptible than others to change during
system maintenance tasks?
• Can the redesign of a system be economically justified?
Read Full Article
Design Rule Spaces: A New Form of Architecture Insight by Lu Xiao, Yuanfang Cai, and Rick Kazman
In this paper, we investigate software architecture as a set of overlapping design rule spaces, formed by one or more structural or evolutionary relationships and clustered using our design rule hierarchy algorithm. Considering evolutionary coupling as a special type of relationship, we investigated (1) whether design rule spaces can reveal structural relations among error-prone files; (2) whether design rule spaces can reveal structural problems contributing to error-proneness. We studied three large-scale open source projects and found that error-prone files can be captured by just a few design rule sub-spaces. Supported by our tool, Titan, we are able to flexibly visualize design rule spaces formed by different types of relationships, including evolutionary dependencies. This way, we are not only able to visualize which error-prone files belong to which design rule spaces, but also to visualize the structural problems that give insight into why these files are error prone. Design rule spaces provide valuable direction on which parts of the architecture are problematic, and on why, when, and how to refactor.
Linking Cyclicality and Product Quality by Manuel Sosa, Jürgen Mihm, and Tyson Browning
Effective communication in product development organizations is widely recognized to be a key element of product development performance. Furthermore, management of product architecture knowledge by the development organization provides important competitive advantage for established firms facing architectural innovation. This research studies how the combination of product architecture and organizational structure determines technical communication in development teams. By documenting and analyzing both the design interfaces between the components that comprise a product and the technical interactions between the teams that design each of these components, we learn how the architecture of the product and the layout of the organization drive development team interactions. Several hypotheses are formulated to explain the unexpected cases when: 1) known design interfaces are not matched by team interactions, and 2) observed team interactions are not predicted by design interfaces. We test the hypothesized effects due to organizational and system boundaries, and design interface strength. Hypotheses are tested using both categorical data analysis and log-linear network analysis. The research is conducted using data collected describing a large commercial aircraft engine development process.
Factors that influence technical communication in distributed product development : an empirical study in the telecommunications industry by Manuel E. Sosa, Steven D. Eppinger, Michael Pich, David G. McKendrick, and Suzanne K. Stout
Understanding the communication process in product development organizations has been recognized as a key element to improve product development performance. It is particularly interesting to study information exchanges in geographically distributed product development teams because of the highly interdependent nature of design organizations. AAdditionally, the use of electronic-based communication media has changed how development teams communicate. By studying the way product development teams use various communication media (face-to-face, telephone, media), we assess how the process of exchanging technical information is influenced by factors such as geographic dispersion, organizational bonds, and degree of team interdependence. We develop a theoretical framework that allows us to formulate several hypotheses about how these factors influence both communication frequency and media choice. We use empirical evidence from the telecommunications industry to test our hypotheses. We confirm previous results about the obstructive influence of distance on technical communication. However, we found that such negative effects may be mitigated by other factors such as the recognizing of highly interdependent team members, the existence of strong organizational bonds, and the use of electronic communication media.
Degree Distribution and Quality in Complex Engineered Systems by Manuel Sosa, Jürgen Mihm and Tyson Browning
Complex engineered systems tend to have architectures in which a small subset of components exhibits a disproportional number of linkages. Such components are known as hubs. This paper examines the degree distribution of systems to identify the presence of hubs and quantify the fraction of hub components. We examine how the presence and fraction of hubs relate to a system’s quality. We provide empirical evidence that the presence of hubs in a system’s architecture is associated with a low number of defects. Furthermore, we show that complex engineered systems may have an optimal fraction of hub components with respect to system quality. Our results suggest that architects and managers aiming to improve the quality of complex system designs must proactively identify and manage the use of hubs. Our paper provides a data-driven approach for identifying appropriate target levels of hub usage.
The Impact of Component Modularity on Design Evolution: Evidence from the Software Industry by Alan MacCormack, John Rusnak and Carliss Y. Baldwin
Much academic work asserts a relationship between the design of a complex system and the manner in which this system evolves over time. In particular, designs which are modular in nature are argued to be more “evolvable,” in that these designs facilitate making future adaptations, the nature of which do not have to be specified in advance. In essence, modularity creates “option value” with respect to new and improved designs, which is particularly important when a system must meet uncertain future demands.
Despite the conceptual appeal of this research, empirical work exploring the relationship between modularity and evolution has had limited success. Three major challenges persist: first, it is difficult to measure modularity in a robust and repeatable fashion; second, modularity is a property of individual components, not systems as a whole, hence we must examine these dynamics at the microstructure level; and third, evolution is a temporalphenomenon, in that the conditions at time t affect the nature of the design at time t+1, hence exploring this phenomenon requires longitudinal data.
In this paper, we tackle these challenges by analyzing the evolution of a successful commercial software product over its entire lifetime, comprising six major “releases.” In particular, we develop measures of modularity at the component level, and use these to predict patterns of evolution between successive versions of the design. We find that modularity has a strong and unambiguous impact on design evolution. Specifically, we show that i) tightly-coupled components are “harder to kill,” in that they have a greater likelihood of survival in subsequent versions of a design; ii) tightly-coupled components are “harder to maintain,” in that they experience more surprise changes to their dependency relationships that are not associated with new functionality; and iii) tightly-coupled components are “harder to augment,” in that the mix of new components added in each version is significantly more modular than the legacy design.
Evolution Analysis of Large-Scale Software Systems Using Design Structure Matrices and Design Rule Theory by Matthew J. LaMantia, Yuanfang Cai, Alan David MacCormack and John Rusnak
Designers often seek modular architectures to better accommodate expected changes and to enable parallel development. However, we lack a formal theory and model of modularity and software evolution, which can be used for description, prediction, and prescription. According to Baldwin and Clark’s theory, modular architectures add value to system designs by creating options to improvethe system by substituting or experimenting on individual modules. In this paper, we evaluate their theory by looking at the design evolution of two software product platforms through the modeling lens of design structure matrices (DSMs) and design rule theory. Our analysis shows that DSM models and options theory can explain how real-world modularization activities in one case allowed for different rates of evolution in different software modules and in another case conferred distinct strategic advantages on a firm (by permitting substitution of an at-risk software module without substantial change to the rest of the system). The experiment supports our hypothesis that these formal models and theory can account for important aspects of software design evolution in large-scale systems.
Visualizing and Measuring Software Portfolio Architectures: A Flexibility Analysis by Robert Lagerstrom, Carliss Y. Baldwin, Alan MacCormack, and David Dreyfus
In this paper, we test a method for visualizing and measuring software portfolio architectures and use our measures to predict the costs of architectural change. Our data is drawn from a biopharmaceutical company, comprising 407 architectural components with 1,157 dependencies between them. We show that the architecture of this system can be classified as a “core-periphery” system, meaning it contains a single large dominant cluster of interconnected components (the “Core”) representing 32% of the system. We find that the classification of software applications within this architecture, as being either Core or Peripheral, is a significant predictor of the costs of architectural change. Using OLS regression models, we show that this measure has greater predictive power than prior measures of coupling used in the literature.
Visualizing and Measuring Enterprise Architecture: An Exploratory BioPharma Case by Robert Lagerstrom, Carliss Baldwin, Alan MacCormack and David Dreyfus
We test a method that was designed and used previously to reveal the hidden internal architectural structure of software systems. The focus of this paper is to test if it can also uncover new facts about the components and their relationships in an enterprise architecture, i.e., if the method can reveal the hidden external structure between architectural components. Our test uses data from a biopharmaceutical company. In total, we analyzed 407 components and 1,157 dependencies. Results show that the enterprise structure can be classified as a core-periphery architecture with a propagation cost of 23%, core size of 32%, and architecture flow through of 67%. We also found that business components can be classified as control elements, infrastructure components as shared, and software applications as belonging to the core. These findings suggest that the method could be effective in uncovering the hidden structure of an enterprise architecture.