Enterprise Platform Integration; An Iterative Approach

Image courtesy Google

Harness the tools of Iterative product development to drive your enterprise platform integrations with speed, transparency and control over risk

With a lilt in her voice, one of the strongest technology leaders I’ve ever worked with recently said, “I love the Agile for new product development, but I’m still not convinced where platform integration is concerned…”

It’s an understandable position, and addresses issues I have wrestled with in my own practice for many years. After all, with an enterprise platform integration:

• We know where we’re starting, and
• We know where we expect to end up

The shortest distance between two points will always be a straight line, and that looks a whole lot like a Waterfall project.

Don’t Believe the Hype

As a dedicated Agilist, my refutation of the model above isn’t around the “straight line” part, but rather the assertion that we “know where we expect to end up”.

YES, we may intellectually understand, in broad strokes, the scope and target end-state of the initiative. But the devil lies in the details, and the penetrating truth is that broad strokes don’t cut it when millions of dollars and professional reputations are on the line. Below the level of broad strokes are thousands of choices and challenges that must be metabolized during this journey. Very few of these are clearly identified or adequately understood at the beginning of the effort, when Waterfall would have us execute and lock our planning.

We need a mechanism to decompose those broad strokes into details granular enough to build and test. And that’s where an iterative methodology really shines.

The benefits of an iterative approach have been described in rich detail elsewhere, but just as a quick review of some top-contenders:

• Risk/Value Balance: Agility is all about keeping cycle times fast, queues short, and holding time at a minimum. All of these strategies compress the distance between cash invested in the build and value delivered to the marketplace (internal or external). In a world as chaotic as ours, holding back delivery simply for the sake of a grand, “ta-dah!” moment is insane. Think of this in the context of manufacturing: would any manufacturer build all of its products, store them in a warehouse, and only ship them into the consumer pipeline when every last one was completed? Of course not! One warehouse fire and the entire investment would be up in smoke. Manufacturers build, ship, and earn on a rolling basis to mitigate risk and fund the machine.
• Better Business Control: Iterative methodologies all revolve tightly around direct, day-to-day engagement of product-focused stakeholders in the build. Not only does this keep the business in the driver’s seat, but also ensures that stakeholders understand the context and nuance of challenges along the way. No more limitations to visibility based on the lens established by the service provider.
• Real-World Feedback: We’re not as smart as we think we are. Yes, even you. It turns out that much of the functionality that we invest the enterprise’s precious capital to build is barely ever utilized by real users. The best way to combat this risk is to put the tools into the hands of real users as early and often as possible so that we can learn from their feedback and behavior. What they find useful, we extend. What they ignore, we abandon. This approach demands a willingness to deploy features in an unpolished state, and the relationships to do so without driving the user community bananas. In exchange, our product roadmap shifts from theoretical to practical with a bullet.

Okay, so we all agree that applying an iterative approach to enterprise platform integration is worth the trouble. Let’s talk about one strategy for making that happen.

Acme Custom Widget

I will create a brief narrative so that we can run through an iterative approach to enterprise platform integration, together.

Acme Custom Widget manufactures and installs custom widget solutions to a global customer base

Cheaper widget options are available, but Acme differentiates with quality, range of configurations, expert installation, and managed services

Acme currently runs its operations on a portfolio of platforms that have grown organically over time. They are looking for a more holistic solution

Acme has found a vendor/platform/set of platforms [doesn’t matter for this narrative] that they believe will close current operational gaps and position them for success in the coming years

Acme operations involve the following basic Business Functions:

• Commercial: Sales pipeline and relationship management
• Estimating: works closely with sales, and acts as a bridge to Operations and Finance
• Resourcing: Specialty widget manufacturing and installation is tricky business. Among its differentiators is the portfolio of experts that Acme can bring to custom widget installations across the globe
• Operations: Deal execution from post-close through delivery
• Change in Scope: Many specialty widget deals require adjustments along the way based on requirement maturation, market conditions, and ground truth. The Change in Scope business function manages these in-stream adjustments from the Commercial, Operations, and Financial perspectives
• Financial: Corporate finance, cost tracking, AP/AR, etc.

NOT our Strategy

Replacing the current portfolio of tools in place at Acme with a new suite of tightly integrated platforms is a massive job. Based on the breadth and importance of integrations across these platforms, the initial idea might be to make ALL of the changes, test them together, and then launch to users.

Fortunately, five decades of IT experience have convinced us that while that may seem the most logical approach, it is not the approach most likely to succeed.

Fortunately, five decades of IT experience have convinced us that while that may seem the most logical approach, it is not the approach most likely to succeed

Entire books have been written about why the monolithic approach to enterprise platform integration no longer works, and a complete examination is beyond the scope of this post. Suffice it to say that this endeavor is simply too large (too many stakeholders, too many integrated business processes, too many different tools and platforms, too many use cases, too many competing priorities) to be effectively metabolized in one shot.

Rather than a monolithic approach, ours will be iterative.

An Iterative Approach

Leveraging an iterative approach to enterprise platform integration, at Acme Speciality Widget or anywhere else, requires us to make fundamental decisions about how the total scope will be parsed into a serialized set of components to be addressed in priority order.

Leveraging some syntax created in a previous post (https://drewharteveld.medium.com/avoiding-the-quiet-disaster-6078a7608411), we will organize the work according to Business Functions.

As the name suggests, Business Functions describe elements of the business, not technology tools or platforms. Organizing the scope in this manner helps ensure understanding by and continued relevance to the stakeholders so dependent upon this program’s success.

The logical map below helps us visualize Acme’s Business Functions, as well as key points of intersection:

Iterative Delivery Across the Map

In a nutshell, our approach will be to carve off individual Business Functions from the larger map and deliver them one at a time. This, as the only alternative to delivering in a monolithic fashion.

Based on our planning at Acme, this is how we expect to organize and deliver the total scope:

Business Function Integrations (and Embracing Difficult Decisions)

Having a shared master plan is good, but let’s not get ahead of ourselves… Carrying forward our Acme narrative, based upon a robust analysis of job complexity, readiness, and platform foundations, it has been decided that the first Business Function to be tackled will be Resourcing.

What this really means is that the team will invest the first Flight of the program [more on Flights, below] focused on meeting the needs of the Resourcing Business Function. The expectation is that:

• Scope of this Business Function will be broken down into individual Functional Attributes
• Functional attributes will be constructed in a strictly prioritized fashion
• Only as many Functional Attributes as can be metabolized by the team during this Flight can be included
• At the end of the Flight, all Functionality Attributes will be deployed to Production
• Real users will begin utilizing these Functionality Attributes in their everyday work
• Unless other specific accommodations have been planned, this portfolio of Functionality Elements will represent the sum total of scope included in the Resourcing Business Function for this program
• After the program is complete and all business functions have been addressed, the enterprise is welcome to make additional passes to add ever-more richness to the solution

This is Acme’s shot at tackling Resourcing at this juncture, and the team will plow forward as aggressively as it can. It is likely that the team will not be able to address every envisioned/defined element of Resourcing during the Flight, which is why prioritization of those elements is so incredibly important. Also, to the greatest extent possible, each Functionality Element must be constructed such that it can stand on its own, in case some of its closely related brothers and sisters don’t make the cut. Understanding the contours of what _absolutely_ must be included for Resourcing to stand as a Business Function on its own — at least initially — is the nature of Minimum Viable Product strategy.

Don’t fool yourself, these are excruciating decisions and the business of product development demands that we make them every single day.

Iterative Exhaust

Note the squiggly lines at each edge of the Resourcing solution in the diagram above. These represent the borders where the New solution the team is putting in place rubs up against the Legacy solution that is already there. These squigglies represent the need to integrate the New into the Legacy, at least temporarily. Eventually, the *entire* solution will be covered by new components. Until that time, we must ensure that the New and the Legacy can work together, in Production, seamlessly. We refer to these New-to-Legacy integrations, collectively, as “Iterative Exhaust”, an unavoidable byproduct of our iterative approach.

Those who know more about arithmetic than about product development might look at the Iterative Exhaust and label it ‘waste’. After all, these involve code that is not intended to be utilized for the long term… As soon as the Business Function next door receives its own new treatment, these points of integration will have to be refactored or removed. “Wouldn’t it be more efficient,” they ask, “to just do the whole job and launch once at the end in order to avoid the cost of these disposable elements?” It’s a fair question, and the answer is “NO.”

Seventy years of dodgy IT initiative results have proven that the ‘build it all and then launch at the end’ approach is fraught with risk. Our choice is to embrace an iterative approach, instead, in order to help mitigate that risk. Iterative Exhaust is an unavoidable byproduct of that decision.

The ‘waste’ actually only exists on paper, anyway. My own involvement in twenty-five years of enterprise initiatives has taught me that while the physical manifestation of our work is software, the assets that lay just inches below that surface are all about knowledge. From a very real perspective, the product of the enterprise’s investment in the program is knowledge, of which working software is just an actionable byproduct. Our program isn’t a research project with somebody else’s money, but that doesn’t preclude us from understanding and maximizing the role that accrued knowledge plays in its outcomes.

From this point of view, the work involved, and knowledge accrued, in addressing Iterative Exhaust is anything but waste. Indeed, the more we learn about the borders of the New and the Legacy now, the better prepared we will be to expand the New into fresh territory during subsequent Flights.

From a very real perspective, the product of the enterprise’s investment in the program is knowledge, of which working software is just an actionable byproduct

IMPORTANT — While the goal of the integration program is to improve operations, the Iterative Exhaust should not include these aspirations. These are temporary bridges we are building between old and new. Their target in terms of convenience, robustness, and scalability should be limited to “not materially worse than the user experience running in Production, today.” It’s easy to get so swept-up in creating an improved solution that we lose sight of the actual yardstick against which progress will be measured by our users. We need a different yardstick for Iterative Exhaust than the one we use for the new functionality that represents our core mission.

Do not underestimate the capacity of those users to accept and support a temporarily less-than-optimal solution if they know it is in service of more fundamental improvement down the road. Especially if they feel included in the thought process along the way.

Flights to Production

As explained above, each Business Function is addressed, all the way to Production, within the context of its own Flight. So what’s a Flight?

A Flight is an organized set of Sprints, complete with its own expectations around planning and delivery. For the purposes of our Acme Custom Widgets example, each Flight has a duration of twelve weeks. Assuming sprints of 10 business days, that makes 6 Sprints per Flight. The diagram below helps illustrate each Flight’s internal structure:

The point of the Flight structure is to galvanize attention of the entire team around a limited body of scope during a designated block of time. Once the Business Function to be addressed during the Flight has been chosen, all other scope in the program is pushed out of view. We can’t solve all the Business Functions at once, so those that aren’t Resourcing will need to wait. We’ll wire ’em up when the time comes.

The point of the Flight structure is to galvanize attention of the entire team around a limited body of scope during a designated block of time

There are three key components to each Flight —

New Functionality Sprints:

Assuming a SCRUM methodology, as utilized at Acme, the team has 4 Sprints, or about 8 weeks, to tackle all of the Functional Attributes to be included in the Flight. Four sprints is not a long time, so, as explained above, prioritization and componentized construction will be critical.

As with all SCRUM, testing of the software must be managed within the bounds of each Sprint. Functionality Attributes that do not pass testing get tossed back into the Backlog for future reprioritization.

One unique element of this approach is that the team does not deploy code all the way to Production at the end of each Sprint. Doing so is a laudable aspiration of Agile, and the absolute best way to maximize value to the enterprise. But within the structure being described in this post, our operational goals are best met by temporarily delaying deployment until the end of the Flight

Hardening and Deployment Sprint:

The Hardening and Deployment Sprint does not involve any new functionality. During this Sprint, any few outstanding bugs are addressed, and all of the code is retested in an integrated fashion. Further, the team prepares the code for deployment to Production and plans any associated cutover activities. Any Functionality Attributes that are not able to successfully pass testing are stripped from the scope and tossed back into the Backlog. These will not be included in this Business Function for the foreseeable future.

At the end of the Hardening Sprint, all Functionality Attributes are deployed to Production.

Hypercare and Prep Sprint:

The two weeks following deployment of the Business Function are referred to as a Hypercare and Prep Sprint. It is imperative that no new functionality be constructed during this time. Instead the focus is on:

• Keeping the team on the balls of its feet to support any bugs or issues discovered in Production
• Providing training and other support to the real-life users employing the solution in their real-life jobs
• Engaging in high-level planning for the next upcoming Flight

If your enterprise is operating in a Scaled Agile framework, the Flights as described above look a whole lot like SAFe’s “Program Increments”. That is by design. The point here is that this cadence is valuable and accessible even if your company hasn’t jumped into the deep end of the Scaled Agile pool.

Production or Bust

As described above, the team needs to think of this Flight as their single shot at addressing Resourcing during this program. That’ll require brutal prioritization and a coding architecture that leaves accessible ‘hooks’ for currently unknown integrations down the road.

Unless leadership is willing to rebaseline the entire program, what we can’t do is “just tack-on a few extra sprints of Resourcing to get all the goodies.” Even if leadership IS willing to rebaseline, this is rarely the best course of action. It is always better to plow forward across the larger scope, working toward a total-breadth solution. We can always charter a second phase program, after this one is complete, to lay-in more functionality under one or more Business Functions. And if there isn’t time/money to do that, all the more reason for us to keep our build lean and focus on delivering some functionality across the entire breadth of the solution.

One more quick reminder that the development team needs to utilize a set of best practices around code and branching such that any single Functionality Attribute that cannot pass testing can be stripped away without the entire solution falling apart. Because at the end of the Flight we are going to Production.

Rinse & Repeat

Once the Business Function/ Flight/ Sprinting rhythm is in place, the program takes on a life of its own. Subsequent Business Functions are snapped-together like Legos into an ever-extending row from the beginning of the program to its carefully managed end. As each new Business Function is addressed:

• Any Iterative Exhaust that was previously put in place is identified and refactored into the new solution
• New Iterative Exhaust is created at the borders of this Business Function in order to allow it to play in Production with all elements still in Legacy mode

Product development teams are quick to learn architectures that largely mitigate any churn caused by Iterative Exhaust.

The diagrams below illustrate the succession of Flights through which Acme’s entire map of Business Functions is addressed across its enterprise platform integration program:

One Team, One Dream

All of this operational architecture is worthless if senior leadership can’t make the mental shift that lies at the root of all Agility:

• “This is the team I have chosen to assemble, and I provide it with the authority to responsibly steer the program as well as my continued faith in its intention to do so”
• “I have my own agents acting as full members of this team, as responsible for success, failure, and working through challenges as every other member”
• “This team will work as hard as it can, every single day, and keep the strategy of the program and enterprise as the core of every decision and action”
• “I understand that I cannot have everything, thus recognize the pivotal responsibility of my embedded agents to prioritize the work in order that the most important components can be completed, first”
• “I know that the best way for us to get from the start to the finish of this foot race is to focus on momentum and not get bogged-down in functional minutiae along the way. We can always come back for another pass, later, if warranted”

If senior leadership can’t make that mental leap — and many leaders cannot — all the operational architecture in the world isn’t going to solve the problem. Forget an iterative approach. Forget a waterfall approach. Stick with running your business on index cards and good luck scaling your enterprise.

Agile Evolution

It is true that iterative methodologies were germinated in the world of new product development. The primacy of the Big Tech early adopters in today’s marketplace proves Agile’s potency in that scenario. Do not underestimate the utility of these concepts in an enterprise platform integration context, as well, however.

Given a lightweight architecture for defining and organizing scope, basic shared commitments about prioritization and momentum, and an acceptance that any puzzle needs to be assembled one piece at a time, we can leverage an iterative approach to de-risk and maximize our enterprise platform integrations.