Why is Business Intelligence innovation so slow?

[ This was originally posted on an old blog: Why is Business Intelligence innovation so slow? ]

You might have gathered that I'm a big fan of Business Intelligence. The potential that it has to impact a business is incredible (and largely unrealized - but that's another post).

I'm also a big fan of innovation. I had subscriptions to Popular Science and Omni as a kid (can you tell?) and now I get my fix for innovation thinking from the Killer Innovations podcast.

The point is that I've been wondering why the pace of innovation is slow in Business Intelligence.

One reason could be entrenchment. Classical BI/DW projects have been so costly and so complicated that the solutions, once delivered, have an incredibly high level of entrenchment. After all, the business thinks, now that we've spent so much on this solution we need squeeze every last bit of value out of it. I see this in action across all the elements of the solution.

How many BI "shops" do you know that have changed from, say, Oracle to SQL Server of from Cognos to BusinessObjects. I honestly don't know of any significant examples of an established shop switching (maybe a couple of instances of DW appliances being added on top). It seems that once the vendor is in it takes dynamite to get them out again.

This level of entrenchment is potentially extremely costly. If your vendor wants to double your "suport" fees or gouge you on additional licenses then you're pretty much forced to put up with it because you don't have an easy way out.

Another reason may be time. BI projects take time and lots of it. Kimball has taught us that successful BI projects are incremental and iterative (and big bang projects fail) but this implies a long term vision and the timescale to go with it. My hunch is that most shops aren't that receptive to "new and improved" tech when they're knee deep in the political quagmire half way through a project.

I could probably come up with more reasons... but (at the risk of repeating myself) I think it comes down to open standards and how thin they are in the BI space.

Sure we have ODBC, which is brilliant, but many key technologies don't use it, particularly if you're in a dedicated Oracle or SQL Server environment. For example, SQL Server Integration Services doesn't even give you the option to use ODBC against SQL Server databases (AFAIK). Meaning that you have to keep your warehouse on SQL Server of face up to the nightmare of re-writing all your ETL packages.

My plea to the vendors is this:

• implement open standards in your products  (this will help me trust you)
• make migration to your product easy, whatever it takes (or else I'll never change).

You (BI vendors) will have to take the risk of being the first to open up. You'll have to give me a good reason to move and proof that I can trust you. After all, most of the businesses who want BI now have it, so you future health depends on converting your competitor's customers into your customers (and surely that's cheaper than just buying your competitors!).

[ All of this reminds me that I need to do a post on open source BI. ]