A buzzword is defined as a word or phrase that is “fashionable at a particular time”. Five of the more fashionable buzzwords these days include Cloud, Internet of Things (IOT), Industrial Internet of Things (IIOT), Machine Learning and the omnipresent and all-inclusive buzzword of Big Data. But for business owners and managers, these buzzwords can sometimes feel more like problems than opportunities because what they all have in common is that they all generate massive amounts of data. Big Data? I don’t think so. More like monstrously huge data: scary petabytes, zettabytes and whatever type of bytes comes after that (do we start over again at alphabytes?).
“Big Data is not a fad. We are just at the beginning of a revolution that will touch every business and every life on this planet”, says Bernard Marr in Forbesmagazine. Wow. No pressure there. Sometimes Big Data feels more like a tidal wave than a trend for those of us who own and manage businesses.
What’s the solution? Well, the quick answer is: make your corporate culture lean and agile and transform all that Big Data into Business Intelligence (BI), which then produces actionable insights that lets you make data-driven decisions. Buzz. Buzz. It’s a swarm. I squeezed five into one sentence. Did you notice? But there is a reason that these words are buzzy. Because every business owner and manager knows that this is where they need to be. Alan Deutschman’s famous quote: “Change or die” is a bit harsh but not so far off in our perpetually transformative digital economy. Google, Apple and Microsoft will probably be around in 10 years. Perhaps they will be Gapplesoft, but nonetheless, is any other company guaranteed to be around in 10 years?
So, as the Goliaths fall and the Davids rise, companies know they need to be lean and agile. And yet, the great big elephant in so many boardrooms and meeting room these days is this: the more companies talk about being lean and agile — the less they are lean and agile. The more they seek to elicit BI out of their Big Data the more they can become mired in paralysis by analysis, to the point where ‘dying’ is sometimes easier than changing in some corporate cultures.
Wise and successful managers tend to shun BI in favor of making decisions based on their gut feelings because, after all, they got wise and successful by trusting their gut feelings. Well, here’s the thing. It’s not an either/or choice. The manager of the future can still follow his or her gut feeling, but why not back it up with some truly intelligent BI.
But, and this is a big ‘but’, the fundamental problem today is how to turn Big Data into BI. Big Data is accumulating in a vast, and diverse, set of data stores, data warehouses, databases, Excel spreadsheets, PDF files and on it goes ad nauseam.
Sure, you can extract all this data into a single consolidated data warehouse, transform it as required, create a new data repository and migrate your data into it, and then use this new repository to run queries that allow you to provide analytics to your front-end applications, and this gives you some semblance of BI. Only problem is that this process can typically take months or even years. Gapplesoft will be ruling the world by then. This process is what is called Extract-Transform-Load or ETL. It is what we have been doing with data for the last 25 years. And ETL has seen its heyday. No one can wait years for BI anymore. In fact, months is too long. Days is too long. Today, you need to deliver BI in hours or, even better, in real time. If you can derive BI from your ‘live’ data in the morning and implement changes based on that BI in the afternoon then you are truly lean and agile.
But how is it possible to achieve real-time BI without the expense and pain of implementing a 100% in memory platform across your entire business? And this is where Data Virtualization enters the discussion. If you are like me, you probably don’t trust the word ‘virtual’. Virtual means not real. Virtual machines in computers always had a weird ring to it. How did they get a machine inside the machine? Virtual reality just seems weird too. I like the real reality! And those headsets make me dizzy.
But Data Virtualization is very real and it is here to stay. Why? Because it is both a stepping stone and Rosetta Stone. It is a stepping stone from Big Data to BI in the sense that it provides a technological solution for the aggregation and transformation of data, but without ETL, without data replication and using data from any data source that you can imagine. It is a Rosetta Stone in that it provides the critical deciphering of the language of Big Data into the language of BI. If Big Data is a tidal wave, Data Virtualization can be your surfboard.
Let’s take a closer look at the fundamental problem with Big Data these days. Companies today are scrambling to collect vast amounts of structured, semi-structured and even unstructured data because somewhere in there is crucial information that can be used to make strategic business decisions. However, massive amounts of data requires heavy processing and there’s the rub: most companies are still struggling with their structured and semi-structured data, which is sitting in various disconnected data silos throughout the company, which poses and challenge itself.
So, the real solution to the Big Data conundrum is Data Virtualization. Fundamentally, it’s a way to connect all those disconnected data silos. More specifically, here’s a high-level technical overview: your Big Data is stored in Oracle, Sybase, PeopleSoft, SAP, Siebel, Word, Excel, PDF, etc. Some of it is structured or semi-structured. Some of it is unstructured. Putting the details aside, this is you back-end. Your front-end is your BI applications: dashboards, queries and reports that enable you to make decisions. Data Virtualization is a middleware layer that sits between your back-end and front-end and ensures they connect and communicate seamlessly. At a high level, it is that simple.
Ding! Lightbulb. This is where it went off for me. After 25 years of trying to connect back-ends to front-ends, this was the key to understanding Data Virtualization: it decouples your back-end from your front-end. Input: any data. Output: any BI application you like. Data Virtualization handles everything in between.
Not convinced you need Data Virtualization yet? Consider a merger or acquisition of two companies. I’ve been involved in a few and the ‘seamless IT transition’ to a single company is still probably going on and won’t be done until Gapplesoft rules the world. One company with all its data silos is a big challenge. Add a second company with a bunch more data silos and the challenges can be insurmountable. (Note that I’ve left out the personnel issues involved here and the perennial IT guy locked in to his own proprietary technology—although it’s really Oracle’s proprietary technology—you know the deal).
With Data Virtualization, all these problems go away. Connect to the acquired companies’ data sources and a seamless IT transition is possible.
Here’s another way of looking at it: think of Big Data as flour. There is lots of it. It is cheap. It is essential to baking a cake. But it doesn’t taste very good on its own and it’s messy and ends up all over the place in your kitchen. BI is your cake. You use it to make your decisions in a data-driven way. Data Virtualization is your batter. It’s the mix of flour with sugar, chocolate, baking powder. So, with Data Virtualization (and with apologies to Marie Antoinette) you can have your cake – and your batter – and eat it too.
Beyond the buzzwords then, what does this all mean for your company over the next few years. Here, a simple example of a restaurant chain is useful. Assume you are the manager of a chain of 100 restaurants. You started with 10 successful restaurants but you have just added 90 more and created a franchise. To begin, you need to see the numbers: the profit/loss for each restaurant and all 90 restaurants store data differently. With Data Virtualization, it’s easy. Simply connect to all 90 of the new data stores, build the abstraction layer, choose your BI tool of choice and you’re done! You now have real-time financial reports presented in a single consolidated view for all 100 restaurants.
Next, say you add an app with a survey that gives you instant results related to customer satisfaction. From the survey data, you realize you have a waitress in your New York restaurant who is very popular to the point that she is even bringing in additional customers and revenue? You need to hold on to employees like that so you can arrange to give her a bonus this month? Or, for example, everyone dislikes a certain dish in your Munich restaurant? Get it off the menu asap. Data Virtualization means you have this information in the morning and you can take action before the end of the day.
But business owners and managers know that new customer acquisition is the key. So how about a push notification when app users are within 1 kilometer of your restaurants, with perhaps a 10% discount for the next hour if they eat in your restaurant. With Data Virtualization, this BI can be instantly available to your marketing team so they can find hundreds of potential new customers in every city each morning, and then email them discount coupons that afternoon. The result: thousands more in income that very same evening!
Or perhaps a loyalty program to reward your repeat customers who represent recurring revenues. Data Virtualization can provide a 360 degree customer view so you can easily find your loyal customers and reward them appropriately.
Now let’s get even more specific. Let’s say 60 of your restaurants have outdoor patios. Use Data Virtualization to mashup weather patterns with your new database of potential customers in each city. So that on a Friday morning your BI tool can analyze the forecast for the weekend and find it is going to be super-sunny on both weekend days, so you can send those potential customers a 20% off coupon on Friday night to be used at one of your patios on the weekend.
Now that’s agile! Real-time BI with an immediate impact on the bottom line. Whether you’re a CEO, CFO, CTO, CIO or a business owner who is all four, this finally is the kind of BI that pays off.
Recognizing the game-changing significance of Data Virtualization, the field has already become quite crowded with vendors and their solutions, including both large and smaller players. Among the smaller (read: lean and agile) companies, dataWerks is emerging as a leader with its patent-pending Infinite Intelligent Indexing (triple-I) technology. Recall the surfboard I mentioned earlier for riding the tidal wave. Well, dataWerks is like a surfboard with a turbo engine at the back.
Here is why: the key to Data Virtualization is the data abstraction layer (i.e., middleware) and here the triple-I technology is superior. It enables the incorporation of data from an unlimited number of data sources and, in turn, integrates and aggregates this data, which then enables the delivery, within milliseconds, of BI in the form of business insights and business reports. Basic dashboard reports are provided with the dataWerks solution, but the real power lies in being able to connect to any BI solution you choose. Add to this the ability to connect to any data store and you can see this is some super-tasty batter. Bake any cake you want with it!
The underlying triple-I technology is extremely sophisticated, but boiled down to its essence, it functions like a good old-fashioned book Index. It stores references to your data and keeps them up to date in real-time. And this allows dataWerks to deliver real-time BI better than most vendors out there.
Big Data and BI may conjure images of a brave new world that is a dark dystopia, but it doesn’t have to be this way. Indeed, for business owners and managers, the ascendance of Data Virtualization is hardly a dystopia, but rather, at long last, a business manager’s utopia where they get their BI in the morning and can take action before the end of the day. In other words, this is what so many of us have been wanting from IT for 25 years! Grab your turbo surfboard and ride the wave! It is going to be a wild ride.