Web Analytics supplants Business Intelligence?

Post written by Wade W., BI Consultant at Ideaca. Read more about BI on his blog: Pragmatic Business Intelligence. 

In reading industry material, I recently came across a statement that can only be, in my opinion, the product of tunnel vision. It was one of the most short-sighted and fundamentally erroneous statements I have seen in some time. Analytics

“At the 2005 Emetrics Summit in London, Bob Chatham from Forrester Research described what it means to be the key. He told the assemblage that we are the leaders of tomorrow – and he wasn’t just preaching to the choir to curry favor – he made sense. Chatham told us that “web analytics” would eventually be subsumed into business intelligence, thereby changing the game. Instead of giant data warehouses being sifted in hopes of finding patterns, it would be the likes of us web analysts in charge.” (Jim Sterne, Target Marketing of Santa Barbara, edited by Erika Lindroth, The Weather Channel Interactive, Inc.)

I agree that web analytics will be (and is starting to be) subsumed into BI. However, I question the sentiment that “giant data warehouses [are] being sifted in hopes of finding patterns” and that Web Analytics would “change the game.” Is Web Analytics really going to revolutionize the art of Business Intelligence so significantly? The implication in this quote is that somehow traditional Business Intelligence is somehow inferior to Web Analytics.

I think this is an excellent example of what happens when someone seen as a leader in a field becomes too engrossed in what he is evangelizing…he becomes blind to the bigger picture.

The fact is that Web Analytics, though impressive in its power to aggregate user behaviour and use this to optimize website profitability, it is by nature a limited field. You are able to track user behaviour – generally anonymous at that – through a single customer-facing channel. Web Analytics is Business Intelligence, that only leverages a single source.

“Giant Data Warehouses,” however, are repositories of cross-organizational data, in most cases that extracted from up to hundreds of disparate data sources – Legacy systems, ERPs, CRM systems, finance, operations, HR, desktop apps, web services, external sources – and loaded into a database of a very specific architectural design optimized to return query results on the huge amounts of data very quickly.

Further, this data will certainly have different meanings across and organization – what does “Customer” mean? How do we define this? Part of the process is to work closely with the business to define common business definitions of business entities…so all that data of all that depth and breadth and richness is (should be….) based on common meanings that have been agreed to by key stakeholders. We can mine the data to identify unknown customer segments. We can do Predictive Modeling. Starting with a business mentality, there is the potential to leverage some powerful Business Intelligence.

But I do agree that Web-sourced data represents a substantial opportunity. We can take those Web-specific data sources that power our Web Analytics Apps, and add that to the existing Data Warehouse, passing through the same business rules to ensure heterogeneous data has a single meaning. Now we are talking organization wide, multi-source Business Intellligence.  Plug BI’s powerful analytical tools into our database, and with some targeted, business-driven KPI’s, and we have another, very powerful means of driving profitability

Web Analytics could be said to be proportionally less expensive than traditional BI – same basic cost range for the analytics tool, but less demand for investment in multiple software licenses from different vendors (possibly), less complex data massage (or not…) and shorter time to implement.  And that in itself is a strong argument in favour of Web Analytics – reduced time to market.  However, you won’t have the spectrum of information you have in a well-implemented Data Warehouse.

I believe that Web Analytics is a complement to BI. It can be integrated into a dashboard, or can stand alone to guide developers and webmasters to optimize content. It does have an effect on our database architecture – we must adapt the design of the database to integrate web data. But does it “change the game”? No – it  makes it more interesting. And as a Business Intelligence professional, I welcome another tool that will add value to my service offering and to my clients.

Wade Walker

The data has the answers

Post written by Evan Hu, Co-founder of Ideaca. View his blog here: evanhu.wordpress.com

In a 2001 research report by META Group, Doug Laney laid the seeds of Big Data and defined data growth challenges and opportunities in a “3Vs” model. The elements of this 3Vs model include volume (the sheer, massive amount of data or the “Big” in Big Data), velocity (speed of data processed) and variety (breadth of data types and sources). Roger Magoulas of O’Reilly media popularized the term “Big Data” in 2005 by describing these challenges and opportunities. Presently Gartner defines Big Data as “high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.” Most recently IBM has added a fourth “V,” Veracity, as an “indication of data integrity and the ability for an organization to trust the data and be able to confidently use it to make crucial decisions.”

The volume of data being created in our world today is exploding exponentially. McKinsey’s 2012 paper “Big data: The next frontier for innovation, competition, and productivity” noted that:

• to buy a disk drive that can store all of the world’s music costs $600
• there were 5 billion mobile phones in use in 2010
• over 30 billion pieces of content shared on Facebook every month
• the projected growth in global data generated per year is 40% vs. a 5% growth in global IT spending
• 235 terabytes data was collected by the US Library of Congress by April 2011
• 15 out of 17 sectors in the United States have more data stored per company than the US Library of Congress

IBM has estimated that “Every day, we create 2.5 quintillion bytes (5 Exabyte) of data — so much that 90% of the data in the world today has been created in the last two years alone." In their book “Big Data, A Revolution That Will Transform How We Live, Work, And Think,” Viktor Mayer-Schonberger and Kenneth Cukier state that “In 2013 the amount of stored information in the world is estimated to be around 1,200 Exabytes, of which less than 2 percent is non-digital.” They describe an Exabyte of data if placed on CD-ROMs and stacked up, they would stretch to the moon in five separate piles.

This sheer volume of data presents huge challenges. For time-sensitive processes such as fraud detection, a quick response is critical. How does one find the signal in all that noise? The variety of both structured and unstructured data is ever expanding in forms: numeric file, text documents, audio, video, etc. And last, in a world where 1 in 3 business leaders lack trust in the information they use to make decisions, data veracity is a barrier to taking action.

The solution lays ever more inexpensive and accessible processing power and the nascent science of machine learning. While Abraham Kaplan (1964) principle of the drunkard’s search holds true: “There is the story of a drunkard, searching under a lamp for his house key, which he dropped some distance away. Asked why he didn’t look where he dropped it, he replied ‘It’s lighter here!’” A massive dataset that all has the same bias as a small dataset will only give you a more precise validate of a flawed answer, we are still in early days. Big Data is the opportunity to unlock answers to previously unanswerable questions and to uncover insights unseen. With it are new dangers as the NSA warrantless surveillance controversy clearly exposes.

I have had the privilege of listening to Clayton Christensen speak several times. In particular he has one common through line that stuck with me and forever embedded itself in my consciousness. “I don’t have an opinion. But I have a theory, and I think my theory has an opinion.” I believe the same for Big Data. The data has an opinion, the data has the answers.