2011 Preview: BI-DW Top 5

Here are the trends I expect to see in 2011, but beware my crystal ball is hazy and known to be biased.


Top 5 for 2011


5) Niche BI acquisitions take off   
  Big BI consolidation may well be finished, but I think 2011 will be the start of niche vendor acquisitions as established BI vendors seek new growth in a (hopefully) recovering economy.  I don't expect any given deal size to be huge (probably sub $100m) however we could easily see half a dozen vendors being picked up.

  The driver for such acquisitions should be clear; Big BI vendors have ageing product stacks and many have been through post-merger product integration pains.  Their focus on innovation has been sorely lacking (non-existent?).  Also, there is huge leverage in applying a niche product to an existing portfolio.  The Business Objects / Xcelsius acquisition is a great example of this (although BO seems to think Xcelsius is a lot better and more useful than I do).

  I will not make any predictions about who might be acquired. However, here are some examples of companies with offerings that are not available from Big BI vendors.  Tableau's data visualisation offering is 1^st class IMHO and is a perfect fit for the people who actually use BI products in practice.  Lyza's BI/ETL collaboration offering is unique (and hard to describe) and a great fit for business oriented BI projects.  Jedox' Palo offering brings unique power to Excel power users and appears to be the only rival to Microsoft's PowerPivot offerings; I suspect a stronger US sales force would help them immensely.

4) GPU based computing comes to the fore
  I blogged some time ago about GPU's offering a glimpse of the many-core future.  Since then I've been waiting (and waiting) for signs that GPUs were making the jump into business servers.  Finally, in April 2010, Jedox released Palo OLAP Accelerator for GPUs. And this autumn I discovered ParStream's new GPU accelerated database (I blogged about it last week).  Finally in December we saw the announcement of a new class of Amazon EC2 instance featuring a GPU as part of the package.

  Based on these weak signals, I think 2011 will be the year that GPU processing and GPU acceleration starts to become a widely accepted part of business computing.  The most recent GPU cards from Nvidia and AMD offer many hundreds (512+) of processing cores and multiple cards can be used in a single server.  There is a large class of business computing problems that could be addressed by GPUs: analytic calculations (e.g. SAS / R), anything related to MapReduce / Hadoop, anything related to enterprise search / e-discovery, anything related to stream processing / CEP, etc.  As final note I would strongly suggest that vendors who sell columnar databases or in-memory BI products (or are losing sales to such) should point their R&D team at GPUs and get something together quickly. Niche vendors have an opportunity to push the price/perform baseline up by an order of magnitude and take market share while Big BI vendors try to catch up.

3) Data Warehousing morphs into Data Intensive Computing

  I once asked Netezza CTO Justin Lindsey if he considers Netezza machines to be supercomputers.  He said no he didn't but that the scientific computing 'guys' call it a "Data Intensive Supercomputer" and use it in applications where the ratio of data to calculations is very high, i.e., the opposite of classical supercomputing applications.  That phrase really stuck with me and it seems to describe the direction that data warehousing is headed.

  If you've been around BI-DW for a while you'll be familiar with the Inmon v Kimball ideology war. That fight illustrates the idea that data warehouses had a well defined purpose simply because we could argue about the right way to do 'it'.  I've noticed the purpose of the data warehouse stretching out over the last few years. The rise of analytics and ever increasing data volumes mean that more activities are finding a home on the data warehouse as a platform.  Either the activity cannot be done elsewhere or the data warehouse is the most accessible platform for data driven projects with short term data processing needs.

  In 2011 we need to borrow this term from the supercomputing guys and apply it to ourselves.  We need to change our thinking from delivering and supporting a data warehouse to offering a Data Intensive Computing service (that enables a data warehouse).  Those that fail to make the change should not be surprised when departments implement their own analytic database, make it available to the wider business and start competing with them for funding.

2) SharePoint destabilises incumbent BI platforms

  SharePoint is not typically considered a BI product and is rarely mentioned when I talk to fellow BI people. Those who specialise in Microsoft's products occasionally mention the special challenges (read headaches) associated with supporting it but it's "just a portal".  Right?  Not quite.  Microsoft has managed to drive a nuclear Trojan horse into the safety of incumbent BI installations.  SharePoint contains extensive BI capabilities and enables BI capabilities in other Microsoft products (like, um, Excel!).  Worst of all, if you're the incumbent BI vendor, SharePoint is everywhere!  It has something like 75% market share overall and effectively 100% market share in big companies.

  So what?  Well, when you want to deploy a dashboard solution where is the natural home for such content?  The intranet portal.  When you need to collaborate on analysis with widely dispersed teams, what can you use that's better than email?  Excel docs on the portal.  If report bursting is filling up your inboxes like sand in an hourglass, where can you put reports instead?  Maybe the intranet?  You get the point. We have a history in BI of pushing yet another friggin' portal onto the business when we select our BI platform.  Our chosen platform comes with such a nice portal, heck that's part of why we bought it. A year later we wonder why it doesn't get used.  We wonder why we spend more time unlocking expired logins than answering questions about reports.

   Right now businesses are only using a small fraction of SharePoint's capability. But they pay for all of them and I expect business to push for more return from SharePoint investments in 2011.  I expect a lot of these initiatives to involve communicating business performance (BI) and collaborating on performance analysis (BI again).  The trouble for incumbent vendors is clear: SharePoint has no substitute; your BI suite has direct substitutes, Microsoft offers some substitutes for free, your BI content is going to end up on SharePoint, once it's there its SharePoint content. BI vendors should expect hard conversation about maintenance fees and upgrade cycles in any account where dashboards are being hosted on SharePoint.

  As a final note, I would suggest that vendors who sell to large customers need to have a compelling SharePoint story.  It's basically a case of "if you can't beat them, join them".  If you have a portal as part of your suite you need to integrate with SharePoint (yesterday).  You need to make you products work better with SharePoint than Microsoft's own products do.  This will be a huge, expensive PITA - do it anyway.  You must find a way to embrace SharePoint without letting it own you.  Good luck. 

1) BI starts to dissolve into other systems

  My final trend for 2011 is about BI becoming bifurcated (love that word) between the strategic stuff (dashboards and analysis) and everything else. That "everything else" doesn't naturally live on a portal or in a report that gets emailed out.  It belongs in the system that generates the data in the first place; it belongs right at the point of interaction. James Taylor and Neil Raden talked about this idea in the book "Smart Enough Systems". I won't repeat their arguments here but I will outline some of the reason why I think it's happening now.

   First, 'greenfield' BI sites are a thing of the past. Everyone now has BI, it may not work very well but they have it.  New companies use BI from day 1.  The market is effectively saturated.  Second, most of the Big BI vendors are now part of large companies that sell line of business systems.  There is a natural concern about diluting the value of the BI suite, however "BI for the masses" is a dead-end and I think they probably get that.  Third, deep integration is one of the last remaining levers that Big BI vendors can use against nimble niche vendors and against SharePoint.  They will essentially have to go down this route at some point.  Finally, many system vendors have reached an impasse with their customers regarding upgrades. Customers are simply refusing to upgrade systems that work perfectly well. These vendors must create a real, tangible reason for the customers to move. I suspect that deep BI integration is their best bet.

  I have had too many conversations about 'completing the circle' and feeding the results of analysis back into source systems.  Sadly it never happens in practice, the walls are just too high.  Once the data has left the source system it is considered tainted and pushing tainted data into production systems is never taken lightly.  Thus the ultimate answer seems to be to push the "smarts" that have been generated by analysis down into the source system instead.  Expect to see plenty of marketing talk in 2011 about systems getting 'smarter' and more integrated.