5 System Age and Updates
The next section of the survey asks questions about (1) the age of the majority of CPI Production Systems and (2) the update frequency of the majority of CPI Production Systems.
5.1 System Age
Respondents are presented with the following question.
In this question, we aim to understand for how long the majority of CPI Production Systems at NSOs have been in operation.
In the context of this question, a system could be long-lived because (1) it was built in an extensible fashion and has been easy to update and maintain over a long time period, (2) the requirements of the system have changed very little over a long time period, or (3) every component of the system has been replaced at some point without ever doing a full “system rewrite” (see Theseus’s Paradox).
Therefore, without more information about the maintenance history of these systems, we cannot say whether older systems are “good” or “bad”. Nevertheless, it is interesting to understand how old the typical CPI Production System is.
We share some notable observations and explanations below.
The system age distribution for NSOs with modular representative systems is approximately uniform, while NSOs with monolithic representative systems are more likely to report that the majority of their systems are between 6-20 years old, and less likely to report that the majority of their systems are more than 20 years old.
One possible explanation for this observation is that monolithic systems are more likely to reach a level of complexity where it becomes too difficult to reliably make changes to the system due to a high degree of coupling (inter-dependency) between components.1 If this is true, it could be the case that relatively few monolithic systems reach an age greater than 20 years before a complete system rewrite is necessary.
NSOs with IT-Only teams appear relatively less likely to have the majority of systems be less than 20 years old, compared to their Analyst-Only and Stream-Aligned counterparts. One possible explanation is that IT-Only teams are more likely to have the technical skills to perform a full system rewrite when it becomes necessary, although we don’t have sufficient data to say for sure.
Another noteworthy observation is that NSOs with Other Mix teams (see Chapter 3) are most likely to have the majority of systems be between 6-10 years old. It is noteworthy that most of the NSOs with Other Mix teams also have Monolithic representative systems, so there is a high degree of overlap between Other Mix teams and Monolithic representative systems.
Our hypothesis for this observation is that teams comprised of individuals with little shared domain context (e.g., a team with individuals from Corporate IT as well as Domain Analysts) are more likely to produce overly complicated and tightly inter-connected systems that are difficult to change.2 After 6-10 years, these systems become so difficult to maintain that it becomes necessary to undergo a full system rewrite.3
5.2 System Update Frequency
Respondents are presented with the following question.
In this section, we ask respondents how often the majority of CPI Production Systems are usually updated.
Unless software is being written in a domain where (1) the problem is “solved” (i.e., requirements never change) and/or (2) the software’s correctness can be proved mathematically, routinely updating systems (e.g., fixing errors in source code or enhancing the system with new features) is a practical reality of software development.
Given this practical reality, updating systems at a somewhat high frequency is generally considered to be good practice.4
We share below some observations on how frequently respondents update their systems.
NSOs with monolithic representative systems are much more likely to never update systems compared to NSOs with Modular representative systems.
We suspect this is once again related to the fact that monolithic systems tend to be complex and involve many inter-connected components. In the most extreme case, it can be too risky to update these systems due to the possibility of a change in one part of a system causing unintended consequences in another (seemingly unrelated) part of the system.
Modular systems, in contrast, tend to be easier to reason about, and involve a lower degree of coupling between system components that are unrelated. Hence, making a change to a system component is less likely to lead to an unintended consequence elsewhere in the system.
Across all respondents, almost one fifth of respondents indicated that the majority of their systems are never updated. This may be partially due to the fact that some data and methods used with traditional field collection approaches change very rarely. Moreover, these approaches can be very simple to implement in certain cases, which could mean no bugs were ever introduced to the implementation. In such cases, it’s conceivable that non-trivial updates are never performed.5
More generally, the distribution of update frequencies seems to be left skewed, with a majority of NSOs having the majority of systems updated once per year or less, or not updated at all.
When looking at the system age distributions broken down by team type, we do not see much difference from the overall trend. One noteworthy exception to this is the Other Mix team category, which seems to have a relatively high proportion of cases updated less than once per year or not at all.6
Monolithic architectures are not guaranteed to result in highly coupled systems. However, it requires considerable effort, skill, and knowledge of software architecture patterns to ensure monolithic architectures remain loosely coupled over a long period of time (Ford et al. (2021)).↩︎
In our experience, these systems become overly complicated because of the need to introduce new abstractions to bridge the lack of shared domain context. These abstractions aren’t necessary when all parties involved have a common understanding of the domain problem.↩︎
This hypothesis is consistent with our professional experience and prior knowledge in software architecture (e.g., Evans (2004), Ford et al. (2021), Richards and Ford (2020)). However, it is important to note that this particular observation is made on a relatively small subset of our survey data, so it could be an anomaly.↩︎
N.B., having the ability to update systems at a high frequency is unambiguously good practice.↩︎
Another possibility is that some systems are never updated because there was only one individual who knew how the system worked, and that individual has since left the organization, so the system can no longer be updated. We did not collect sufficient data to diagnose why this one fifth of NSOs never update their systems, so we cannot say for sure what the explanation is one way or the other.↩︎
Once again, we note the small sample size caveat of NSOs containing Other Mix teams.↩︎