The Secret Recipe for Middleware Success

RecipeIt seems like just a few minutes ago, I was talking about the recipe for success, right?

The most successful companies in the Application Integration and Middleware (AIM) space have done what large software consulting firms have done. They go towards the money. Consulting costs money. Middleware requires consulting. The Fortune 500 requires middleware. They have money. It’s a win-win, right?

What this strategy has led to is an industry that fights over the top 5000 companies world wide and ignores the bottom 9,995,000 companies that can’t afford to have their own IT department, let alone hire a team of integration ninjas to come in and build a custom middleware solution to connect their WooCommerce, QuickBooks and Salesforce accounts.

The really expensive bit in the current middleware market is that huge up-front fee. Because middleware deployment is commonly modeled after software consulting and application deployment, middleware consultants use a traditional software development approach. This generally leads to custom deployments and customizations. This approach is not lean.

In the book, Blue Ocean Strategy,

Update on the Verge of Launch

541921main_atlasvcloseup425xI’m in a bit of a quandary. When I left Caterpillar three and a half years ago, I had one thing in mind. Start a business that addresses a huge pain. The recipe for success, right? My huge pain was technology-based. The recipe for success, right? This pain existed in a very large industry. The recipe for success, right?

So my first order of business was solve the problem. And this is the problem. Integrating computer programs together is really expensive. Not just “Not on sale, today” expensive. Not just “Customizing your own car” expensive. But maddeningly-inefficient-beyond-belief expensive.

When a person wants to move data between programs, they “Copy” or “Cut” and then they “Paste”. This is called (derisively by many in the software industry) manual integration. Most of the time, using the clipboard goes well. If you happen to be cutting from a spreadsheet and pasting to a word processor, though there are many different ways this can be done, the results are close to what you expect.

When a person wants to move data in a different direction, say from a word processor to a spreadsheet, or from a graphics editor to a database, things don’t go as smooth as expected. This is because applications have data models. They are built around thinking of data in a specific way.

A word processor thinks about the data you type into it as a letter to someone, or maybe a book or a pamphlet. A spreadsheet thinks about data as information you want to perform a variety of analytic exercises with. A word processor then thinks that when you want to move information from another application you want to make it “visual”. You want to show other people “hey, look what I’m doing with this spreadsheet” or “look what I just drew”.

But a spreadsheet has no idea what to make of that 12 page essay with mixed fonts, styles and graphics that you just pasted into it. Analyze what? So pasting something the wrong way tends to produce unpredictable results, simply because applications don’t think the same way about data.

Now, imagine we are no longer talking about applications like e-mail, spreadsheets and drawing programs. Let’s start talking about the big applications that drive industry. Banking systems, healthcare systems, factory systems, human resource systems and engineering systems are good examples of what I mean by applications. These systems are just as driven by their data models, but the data models for enterprise-level systems are hundreds of times as complex as a pixel editor.

So people accept that integrating enterprise systems is complicated and therefore expensive. But it doesn’t have to be. Once the solution has been developed to connect a human resources system with an accounting system, programmers don’t have to pay more to duplicate the solution. They can just install the same solution between the same two applications at another company.

But they don’t. Current practices in the Application Integration and Middleware (AIM) market are to start a business engagement at a new company with a three step process.

  1. Roll up your sleeves.
  2. Pretend like you’ve never seen use cases like this before.
  3. Begin customizing the tar out of everything the company wants to use middleware for.

Now, a lot of industry practitioners are going to say this is balderdash. They’ve never once rolled up their sleeves. I get that. It’s a bit of an embarrassment to hint that the AIM industry loves to customize data models. As a matter of fact, it might be a revelation to some. You see, we don’t do this consciously. We look at customer requirements and our creative juices start flowing.

No two companies behave the same way. So, their use cases will always be different. My recommendation, as a customer, has always been to help us find ways we are similar to other companies with similar use cases. Please. Pretty please, with the least expensive sweetener we can place on top. You see, it isn’t middleware that’s expensive. It’s customization. And if your middleware consultant isn’t showing you ways you can change your own business processes and data models to more accurately reflect industry standards, you can save a lot of money by switching to a consultant who will.

I left Caterpillar because I thought Caterpillar was just doing it wrong. I thought we had no clue what we were doing and we were letting bad actors from the AIM industry steer us into the most expensive way of integrating applications together. After I left, I started talking to people. I started researching integration hairballs and found out the AIM industry does this to everyone. And it’s for a very simple reason. The AIM industry has grown, from its infancy in the 1980s, into an industry just like the automotive or software or hardware industries.

But it has nothing in common with those industries. It needs a totally different business model. And that’s the recipe for success.