Month: August 2020

This week, I met with some companies in the embedded systems space that are looking to start A/B experimentation in their products. These products have safety-critical functions and subsystems and of course, it’s difficult to even imagine A/B tests that aren’t, in some way, touching safety-critical functionality. The proponents of experimentation at these companies outlined the challenges they were experiencing and interestingly, all challenges were internal to the organization. To transition from the traditional specification-based R&D to A/B testing requires changes to the way testing takes place, requires a significant reduction in system variants, needs support for deployment of new software versions (preferably over the air), changes the way safety certification takes place, the way we measure data from products in the field, and so on.

Although I believe the adoption of experimentation practices is critical for any company, my point here is to focus on the fact that in most companies I work with, to change anything requires changing everything. For the longest time, companies focused on optimizing the efficiency of their operations. As a result, every activity, every process was integrated into a larger whole, carefully aligned and then tuned to the point that the amount of slack in the system has been reduced to virtually zero.

This doesn’t just take place inside companies but across business ecosystems as well. Automotive companies are famous for demanding their suppliers to build factories next to theirs so that the OEM doesn’t need to run any inventory as it will see a constant flow of product coming in from their suppliers’ factories next door. Deeply integrated, highly aligned, very efficient.

Initially, this integration and alignment starts with individuals coordinating with each other. The next step is to define a repeatable process that everyone needs to follow. The step after that is to build systems that automate the processes. The challenge is that with every one of these steps, changing anything becomes harder and harder.

The focus on efficiency and automation works as long as you’re operating in a stable, unchanging environment. The moment disruption hits, though, the brittle system falls apart. To avoid that, companies and entire business ecosystems fight as hard as they can to resist change. In the short term, the arguments are always valid: the cost of changing the entire set of systems, business processes, interactions between individuals and even the culture of the company in response to change is always higher than the short-term gains.

The problem is, of course, that every time you resist change, you get a bit behind. You accumulate some business, process and technical debt. You become a little less “fitting” to the environment in which you’re operating.

One way to characterize the history of humanity is a constant battle to control our environment and to minimize the need to adapt to the context in which we operate. From rooting out animals that hunt humans to building houses that control the temperature and humidity of the air we breathe and from vaccines to eradicate infectious diseases to the internet that allows us to connect with others, independent of time and space – all of these accomplishments can be viewed as ways to control our environment.

Our success has created a perception that the world is stable and predictable and therefore, we can afford to build deeply integrated, highly aligned and fully automated systems. During the last months, however, we’ve been reminded of the fact that we control much less than we think. The Covid-19 situation has disrupted numerous highly efficient systems.

My point is that we need to be careful to avoid building these brittle systems. We need agility, flexibility and the ability to rapidly change direction. Instead of resisting change, we should welcome it, even initiate it by constantly reinventing ourselves. As Peter Drucker famously said: “Business has only two basic functions, marketing and innovation; all the rest are cost.” Innovation, by its very definition, changes things from how we did them yesterday. It should be easy to innovate and innovation needs experimentation as you never know what works until you try it out. Rather than focusing on efficiency through alignment, processes and systems, focus on making innovation as easy, seamless and fun as possible. In the end, that’s where the real battle is won!

Some decades after the term “Agile” was introduced in software development, one would expect that – assuming it was a good idea (which it is) – the concept should have been fully embedded in industry and we’re busy with other things. Still, and this never fails to surprise me, everyone is talking about “beyond Agile.” The notion seems to be something along the lines of “we’re still busy here implementing Agile, but perhaps we can start to think about what we can or should focus on next.” If you haven’t adopted Agile, then, for Pete’s sake, get a move on.

Business agility and all its enablers, such as agile software development, are critical for long-term business success. Because of this, going beyond Agile is, of course, the only reasonable thing to do, but one can interpret the “beyond” in a variety of ways. Here, I’ll share five “beyond Agile” concepts that I’ve been working on during the last years.

First, the traditional discussion around Agile was concerned with software and software development only. Especially for companies selling products, including mechanics and electronics, as well as software, the main question was how to combine the waterfall process associated with atoms with the Agile process associated with bits. Taking Agile beyond the realms of software and have it include the other technologies making up the product is a critical step in achieving business agility and allows for new and different business models.

The second concept is concerned with going beyond requirements. Traditional Agile, for all its discussion about customer involvement, sprint planning and retrospectives, is still concerned with building towards a specification. Especially SaaS companies have shifted towards outcomes, meaning that teams focus on improving measurable outcome metrics, such as conversion. This requires a fundamentally different approach to development as it’s now the team that has to brainstorm approaches to “move the needle,” build the alternatives to test, measure their impact and then decide on which versions to keep and which ones to drop.

Third, traditional Agile assumed that development takes place in sprints and that there’s a shippable version of the system at the end of every sprint. However, it doesn’t assume that the release of the system actually follows the sprint process. As many companies focusing on continuous deployment and DevOps have figured out, including the release process in the Agile sprint model is a natural way to increase the frequency of value delivery to customers, on top of offering real, tangible data from the field at a much higher frequency. This is a big topic and if you’re interested, you can read more here, here and here.

Over the years, software has been joined by other digital technologies, specifically data and AI. These technologies benefit from the same principles as those that underlie Agile in software and this has led to the definition of DataOps and MLOps to reflect this. The fourth aspect is concerned with adopting Agile principles for all digital technologies. Of course, there’s a lot to read, including on my blog, but you can also watch this video to learn more about my view.

The fifth and final concept is concerned with business agility. Traditional Agile, as I mentioned, is mostly concerned with software but, for sure, is confined to product development. However, the holy grail is business agility, ie the ability of a company to rapidly respond to changes in its environment, the market, customers and strategy.

Concluding, the simple words “beyond Agile” can refer to a variety of concepts. I’ve discussed five of the primary ones, including applying agile to entire systems and not just software, shifting focus from specifications to outcomes, adopting continuous deployment (DevOps), adopting Agile for data and AI and, finally, reaching full business agility. Next time someone brings up the idea of going beyond Agile, make sure you know what they’re talking about (or rather, make sure that they know what they’re talking about). It’s easy for things to get lost in translation!

During my vacation, one of the questions I spent time thinking about is why companies function as they do. And why companies that are digitally born operate and “feel” different from traditional ones. Why companies that were different from the beginning tend to fall back into traditional patterns and become like traditional ones.

I realized that, of course, the patterns in organizations are the consequence of human nature. This drives many recognizable behavioral patterns. First, the formation of hierarchies. As I wrote a few years ago, the need to organize ourselves in hierarchies goes, according to some, back hundreds of millions of years in our evolution, long before there even were mammals. Much of the behavior in society and companies is driven by our innate need to compete and jockey for position in the vast number of different hierarchies that are available to us in modern society.

Second, and a direct consequence of the first pattern, is the need to exert power over others. A higher position in a hierarchy gives power and, unless checked carefully, the need to use that power for personal gain rather than the general good.

The third pattern is that humans rely very strongly on their own beliefs and have a tendency to operate based on opinions, rather than facts. To most of us, it feels like you’re losing face if something that you believe in turns out not to be accurate. And we’ll strive to realign reality with our beliefs to the largest extent possible and easily become defensive if we’re challenged with data that counter our beliefs.

These human behaviors cause traditional organizations to operate in the way they do, including the hierarchies, the bosses that demand to be obeyed even if decisions don’t make any sense and the general tendency to ignore inconvenient data that’s just staring you in the face. This is exacerbated by the fact that most organizations became successful because of their being different in some way from the rest of society. This “us versus them” mentality only reinforces our basic human behaviors.

The emergence of digital technologies, however, brings along a fundamental shift in how companies operate. At the heart of this fundamental shift lies the awareness of recognizing human behavior and not ignoring it (which many have done at their peril), but at the same time streamlining it to ensure that these behaviors generate the desired outcomes, rather than falling back into traditional structures.

These companies are so successful because they harness instinctive human behaviors differently. For example, hierarchies aren’t ignored but are to the maximum extent possible based on meritocratic principles. In addition, they’re structured to be temporary, rather than cast in stone.

Second, the desire to exert power over others is mitigated by employing empowerment and related mechanisms that limit the power of one individual over another. Rather than relying on a boss positioned in a hierarchy, every individual is free and expected to take responsibility for his or her own decisions and actions.

Third, these companies use data to continuously challenge beliefs held by individuals and groups and encourage alternative interpretations of data to avoid the organization getting trapped in groupthink and defensive around key ideas. For instance, most SaaS companies use A/B testing extensively – an incredibly effective tool to break down “shadow beliefs” in the organization.

Over the last years, however, I’ve come to realize that the difference between traditional and these modern, digital companies is fundamental and starts with the basic assumptions and culture underlying the organization. In response, over time, the scope of my work has gone from specific areas in software R&D, such as software architecture, product lines and continuous integration and deployment, to systems engineering and end-to-end R&D. From there, I included the operations part, including DevOps, DataOps and MLOps, and the front-end part of product management. More recently, I did work on business models, ways of organizing, business ecosystems and other topics not strictly in the scope of technology.

However, I feel that what’s lacking is a holistic and complete company and organizational perspective that includes not just the product and services scope, but all functions in the company, including general management, sales, HR and finance. Although my posts over the last years have provided puzzle pieces of this picture, it’s my intention to, over the coming months, provide a more integrated perspective with the intent of helping you, dear reader, to adopt the modern best practices and avoid the pitfalls that many traditional organizations get stuck in.