Tuesday 5 October 2010

The trouble with SaaS BI - it's all about the data

    Some data was released yesterday that purports to show that SaaS BI customer's are very pleased with it's ease of use, etc., etc. Boring. Seriously, I really like the idea of SaaS BI but I haven't seen anyone making great leaps forward. I'd say that they *can't* take us forward because of the box that they've painted themselves into. The box actually has a name: it's called BI.

The BI sandbox
    Eh? What? Here's the thing; BI as we currently know it is the last stage in the information pipeline. It's the beautiful colours on the box that holds the cereal. But it's not the cereal and it's not even the box. It is *very* important (who would buy cereal in a plain cardboard box?) but is also *very* dependent on other elements in the pipeline.
    I don't want to get into a long discussion about definitions of BI. Suffice it to say this: why are terms like 'data warehouse' and 'OLAP cube' still prevalent? Simply because BI does not imply data gathering, preparation and storage. Last example on this theme. If I tell you I'm a Business Intelligence manager, what would you guess is my remit? Does it include the entire data warehouse? The OLAP cubes? All of the ETL processing? No? It could but it rarely does.

It's all about the data
    I once worked for a clever chap who's mantra was "it's all about the data". His daily struggle was to get the business to invest more time, effort and money into the data itself. It was a hard fight. We had a very fast data warehouse (NZ) and some perfectly serviceable BI software (BO) and nearly a dozen newly minted graduates to turn out our reports. What we did not have was a strong mandate to get the data itself absolutely sorted, to get every term clearly defined and to remove all of the wiggle room from the data. As a consequence we had the same problems that so many BI teams have. Conflicting numbers, conflicting metrics, and political battles using our data as ammunition.

Data is the 'other' 90%
    I'd estimate that gathering, preparing, and storing the data for BI represents at least 90% of the total effort, with analysis and presentation being the last 10%. I really hope no one is surprised by that figure. I'd think that figure is consistent for any situation in which decisions need to be made from data. For instance a scientist in a lab would have to spend a lot of time collecting and collating measurements before she could do the final work of analyzing the results. A research doctor conducting a study will have to collect, organize and standardize all of the results study data before he can begin to evaluate the outcome.

It's NOT about speed
    One of the tragedies of the Inmon-Kimball data warehouse definition war is the data warehouse has been conceived as something that you create because you want to speed up your data access. It's implied that we'd prefer to leave the data in it's original systems if we could, but alas that would be too slow to do anything with. What a load of tosh! Anyone who's been in the trenches knows that the *real* purpose of a data warehouse is to organize and preserve the data somewhere safe away from the many delete-ers and archive-ers of the IT world. We value the data for it's own sake and believe it deserves the respect of being properly stored and treated.

Nibbling at the edges
    So, back to the topic, how does SaaS BI help with this issue?  Let's assume that SaaS BI does what it claims and makes it much easier for "users" to produce reporting and analysis. Great, how much effort have we saved? Even if it takes half as much time and effort we've only knocked 5% off our total.

The real opportunity
   And finally I come to my point: the great untapped opportunity for the SaaS [BI-DW-OLAP-ETL] acronym feast is the other 90% where the most of the hard work happens. Customers are increasingly using online applications in place of their old in-house apps. Everything from ERP to Invoicing to call centre IVRs and diallers are moving to a SaaS model. And every SaaS service that's worth it's salt offers an open API for accessing the data that they hold.

The holy grail - instant data
    This is the mother-load, the shining path for data people. Imagine an end to custom integrations for each customer. Imagine an end to customers having to configure they're own ETL and design their own data warehouse before they can actually do anything with their data. The customer simply signs up to the service and you instantly present them with ready to use data. Magic. Sounds like a service worth paying for.


  1. Agree totally.

    Note how even from this vantage point, you still fell into a trap, near the end you pointed out that "online" applications all have open APIs for accessing the data, and....
    the trap is that's only *access* to data so it doesn't give you the meaning of the data, guidance on proper interpretation/rationalization of that data. True the models in these modern SaaS based apps are likely to be cleaner than older legacy stuff, but that's not the same as them being "pre-integrated" and ready for BI. You still need to create a DW from these, rationalize across them, etc. So not such a holy grail.

    We all seem to fall into this trap occasionally. I've made this mistake - e.g., assuming Netsuite would be easy just because they have JDBC/ODBC. A few hours later I said "oh yeah, Netsuite is a big hairy rich ERP system with customizations". The ODBC/JDBC helps, hey it's way better than having to use ABAP in SAP to get at the data, but it's just the access layer. There's still the whole usage issue, not to mention ERP-customizations to deal with.

    Regardless of that overall I couldn't agree more that the data is the problem. My company is founded on this principle. We do BI solutions - the data integration/rationalization, the DW, and the whole analytical application on top. (www.oco-inc.com) I no longer see how projects succeed that don't do it this way.

    Anyway, thanks for hammering on this point - great post.

    mike beckerle
    CTO Oco

  2. Mike,

    Yep, I hear you. I'm not really saying it's easy though. I know from painful experience just how hard it is to create a data warehouse from web APIs that are designed for programmatic OLTP style actions.

    I was trying (and possibly failing) to point out that these online applications are the same for everyone. That means that we (as an industry) can start doing more upfront and start to make a real dent in that 90% for the first time.

    It's probably worth pointing out that I'm mainly thinking of smaller businesses in this post. They're is essentially no such thing as BI in small businesses because the upfront invest isn't justified.

    All that said, I'd love to hear about how you get on-premise SAP integrated into a SaaS BI solution. SAP customisation is a killer.


  3. Interesting read, Joe. It really is all about the data.

    I am founder and CEO of a business intelligence platform company (SiSense) and it is unbelievable how much hype there is over this SaaS BI concept while we never encounter these guys in deals (and we sell to both large corps and small startups).

    We are VC backed and they have constantly been on my butt regarding going cloud, even though most cloud BI vendors have either died (LucidEra) or dying (won't mention any names here).

    Until all business data runs on the cloud, having BI run in the cloud adds more complexity to an already complex enough process.

    I actually wrote about this recently, if you're interested: http://tinyurl.com/29oyrue

    Elad Israeli
    Co-Founder, CEO

  4. Elad,

    Thanks for the comment. I half-agree with your view that "having BI run in the cloud adds more complexity". However, a lot of data is already on the cloud (especially for smaller businesses). I think it's perfectly reasonable to have a cloud BI offering that only supports cloud data (I may be in a minority there).

    I think *everything* will go to the cloud just as surely as factories got rid of their "engine rooms" and converted to electricity. It's a question of timing though and companies who want to bridge that change need to have a foot in the cloud camp. How large that "foot" is will depend on the size of the target customers, with bigger customers dictating a smaller emphasis on cloud.

    Regarding SiSense itself I have some concerns about your market positioning. It reminds me a little too much of Algebraix (http://www.algebraixdata.com/), who have not exactly set the world on fire.

    The talk of "no need for a DW" worries me. As I mention above "the *real* purpose of a data warehouse is to organize and preserve the data somewhere safe". It's a central resource that can be accessed by different end users that may be using Excel or SAS instead of the companies BI tool. Once I get my data into SiSense, how else can I access it? In my experience, good Excel integration is the minimum bar for a successful BI implementation.

    Positioning SiSense as a "unified BI solution" may appeal to some customers but we live in an incremental world. Few customers will be able to completely embrace a new solution in one swoop. Fewer still will have BI/Analysis needs that are completely served by one vendors product. I would like to see a lot more discussion on your site about how your product integrates with the other pieces of the puzzle.

    Finally, I wish you every success. Must BI software is needlessly complicated and a fresh approach is clearly needed.


  5. Joe,

    Thanks for your input.

    ElastiCube IS a data warehouse. It's just a data warehouse software appliance that does not require restructuring for the benefit of query performance. If you have an existing DW, you can use it. If you don't, you have ElastiCube.

    Regarding Algebraix - they seem to be a database vendor to me. SiSense's entire offering is codeless BI development that includes a database but also a very powerful analytics/reporting frontend. We're getting 2 new customers per week and our list is growing nicely. So I guess for the time being, I am more optimistic than you are about our positioning :)

    Thanks again,

  6. Elad,

    Great! I'm glad you're doing well.

    Even if you disagree with everything else I said (which is entirely reasonable!) please, please, please add something to your marketing about Excel integration.

    Even if it's just a detailed explanation of why you make it unnecessary! Excel is a big check box on many customer's short list criteria.


  7. Haha, fair enough Joe. :-)



Disqus for @joeharris76