There was a time when the concept of the semantic model as “the single source of truth” was the most fundamental pillar of the BI discussion at the enterprise level. Haunted by multitude of reports from different developers, applying different ways of accessing the data, and interpreting joins and business rules as they see fit, business and IT managers alike were looking for a solution to a plague of “different versions of the truth”. If no one in the organization can agree on the numbers, then how can we have any meaningful discussion about, well, anything..? That was the problem most BI vendors set their eye on a decade ago, and their solutions were attractive and technically brilliant. Business Objects with its Universe layer, Siebel with the Admin tool, Cognos with the framework manager, they all served the same purpose. Allow the IT organization to rationalize the access to the data, and apply consistency in how data is being reported out from different systems. The ethos of the single version of the truth was so strong and resonated so well with technical and business folks alike. I remember the conviction and near exuberance I felt on each discussion, demo or presentation on the topic of BI, where the first topic covered was the need (and usually lack of) this meta data layer to separate the physical database world from the reporting world. Well, several years have gone by, SAP bought Business Objects, Oracle bought Siebel, IBM bought Cognos, and new names such as Qlikview, Tableau and dozens of others have changed the conversation to the point where I do not remember anymore the last time I heard anyone mention “the single source of truth” in a BI related call. The BI space has changed a lot, and big data trends continue to shatter technologies and ways of thought that dominated the space in the past, but I do think that the huge influx of data exploration desktop BI tools is more of a “pendulum swing” then a new way of thinking.
Visually stunning solutions that are capable of connecting to data sources and update information automatically are popping up everywhere. Our smart phones continue to fuel this frenzy and the big data appliances that allow us to fathom such massive amounts of data as never conceived before add more fuel to the fire. It’s all about speed now. Speed and sexy. Our data visualizations have to look good, be very interactive and we need to be able to develop and deploy them with almost no resources in hours (or a couple of days at most..). Now, whether the multitude of information created in different departments of the organization agrees or not, well, that’s a problem for someone else. Or for some other day. We have things to do, and information to deliver, and we need it big and we need it now. And so it goes…
I think that the pendulum is still swinging and we will see a moment of reckoning in which the explosion of self-service BI will recoil and re-emerge in a new form, more tightly connected and uniform, and with more canny attention paid to the importance of the single version of the truth. Though, I am also glad that fast and sexy are here to stay..
Maybe the “new form of self service BI” will be ETL developers/ Power Users using the new generation of Viz tools in the Cloud to create rapid analyses that are published for simple information consumption? (Exploration views in SAP speak)
But to avoid duplication of data and lots of pockets of data stored in the Viz tool layer we need a common source for these tools to access, hence we are back to the single version of the truth (data warehouse), but making these work is often too slow.
So the challenge is empowering users to manipulate and create their own data sources but consolidating all the business logic back into a single database?
Is this “modelling in HANA”?
Or just good configuration/ knowledge management to rapidly proto-type and then re-engineer for all to share and use within current enterprise architecture?
Interesting post Ron. I agree that semantic layers won’t go anywhere, and whilst all the desktop data viz tools are helping in bringing information and better visualisations forward, all too often I think we will end back at the same place of ‘your data is wrong, mine is right’ and lots of manual manipulation.
A balance needs to be struck, and hopefully that happens sooner rather than later 🙂 Maybe changing tech like HANA will make it less painful to build and maintain data warehouses and semantic layers, when it can all be done in-database?
Thank you Joshua, yes, i agree that modeling in HANA is a plausible answer, but not sure we got all the answers yet. In the mean time, my feeling is that we just have to hang in to our seats and enjoy the ride… 🙂