Big Data: A Mysterious Giant IT Buzzword

Post written by Niaz T., Senior Solution Architect/SAP BW Consultant at Ideaca. Read more about SAP HANA on her blog: Discover In-memory Technology.

In the world of technology there are a hundred definitions for “Big Data.” It seems confusing to come up with a single definition when there is a lack of standard definition. Like many other terms in technology, Big Data has evolved and matured and so has its definition. Depending on who we ask and what industry/business field they’re in, we will get different definitions. Timo Elliott summarized some of the more popular definitions of Big Data in “7 Definitions of Big Data You Should Know About.”

You may be familiar with three “V’s” or the classic 3V model. However, this original definition does not fully describe the benefits of Big Data. Recently, it has been suggested to add 2 more V’s to the list such as Value and Verification or Veracity which are resulted from “Data Management Practices.” As a BI expert who is been involved in Big Data, my approach is to have a practical definition for my clients by emphasizing the main characteristics of data and purpose of Big Data related to each specific area. I like Gartner’s concise definition. Gartner defined Volume, Velocity and Variety characteristics of information assets as not 3 parts but one part of Big Data definition.

Big data is high-volume, high-velocity and high-variety information asset that demands cost-effective, innovative forms of information processing for enhanced insight and decision making. (Gartner’s definition of big data)

The second part of the definition addresses the challenges we face to take the best of infrastructure and technology capabilities. Usually these types of solutions are expensive and clients expect to have cost effective and appropriate solution to answer their requirement. In my opinion this covers the other V which is related to how we implement Data Management Practices in Big Data Architecture Framework and its Lifecycle Model.

The third part covers the most important part and ultimate goal which is Value. Business value is in the insight to their data and to react to this insight to make better decisions. To have a right vision, it’s important to understand, identify and formulate business problems and objectives knowing practical Big Data solutions are feasible but not easy. So when I define Big Data for my clients, I use Gartner’s definition and explain the journey we need to take together to achieve their goal.

In any Big Data project, I start with BDAF or Big Data Architecture Framework which consists of Data Models, Data Lifecycle, Infrastructure, Analytic tools, Application, Management Operation and Security. One of the key components is having high performance computing storage. Since Big Data technologies are evolving and there more options to be considered, I’m focusing on SAP HANA capabilities which enable us to design practical and more cost effective solutions. HANA could be one part of overall Big Data Architecture Framework but it’s the most essential part. The beauty behind SAP HANA is that it is not just a powerhouse Database but it is a development platform to provide real time platform for both analytics and the transactional systems. It enables us to move beyond traditional data warehousing and spending significant time on data extraction and loading. In addition we’re able to take advantage of hybrid processing to design more advance modeling. Another big advantage of HANA is the capability of integrate it with SAP and non-SAP tools.

So, why am I so excited about it? Looking around I see tons of opportunities and brilliant ideas which could get off the ground with some funding. So far, HANA has been more successful in large enterprises with big budgets and larger IT staff. However I’m also interested to encourage medium size enterprises to see the potential of HANA to provide a solution for their problems. The majority of businesses don’t spend their budget to develop a solution. They are eager to pay to solve a particular problem. Now, our challenge as SAP consultants is to help businesses see this potential and how HANA can address their challenges. The good news is SAP supports by providing test environment and development licenses for promising startups.

Got your attention? Well, just to give you a glimpse, take a look at some of the success stories. In addition there are many many other cases if we look around. For instance, these days many applications capture Geo-location data like trucking company, transportation, etc. it means capturing data every 10 seconds or so from every section, every piece of equipment, every location. This could add up to a Petabyte of data! This is an excellent way to bring insight into data and drive intelligence out of it and have it circulated back to scheduling and movement processes. Another example could be companies needing to mine information from social media regarding to their products and connecting this intelligence back to their back end processes to increase customer engagement and satisfaction.

So, do you have any Big Data Challenge? With some funding, we’re able to provide cost effective and practical solution for your challenge to add value to your business.

Web Analytics supplants Business Intelligence?

Post written by Wade W., BI Consultant at Ideaca. Read more about BI on his blog: Pragmatic Business Intelligence. 

In reading industry material, I recently came across a statement that can only be, in my opinion, the product of tunnel vision. It was one of the most short-sighted and fundamentally erroneous statements I have seen in some time. Analytics

“At the 2005 Emetrics Summit in London, Bob Chatham from Forrester Research described what it means to be the key. He told the assemblage that we are the leaders of tomorrow – and he wasn’t just preaching to the choir to curry favor – he made sense. Chatham told us that “web analytics” would eventually be subsumed into business intelligence, thereby changing the game. Instead of giant data warehouses being sifted in hopes of finding patterns, it would be the likes of us web analysts in charge.” (Jim Sterne, Target Marketing of Santa Barbara, edited by Erika Lindroth, The Weather Channel Interactive, Inc.)

I agree that web analytics will be (and is starting to be) subsumed into BI. However, I question the sentiment that “giant data warehouses [are] being sifted in hopes of finding patterns” and that Web Analytics would “change the game.” Is Web Analytics really going to revolutionize the art of Business Intelligence so significantly? The implication in this quote is that somehow traditional Business Intelligence is somehow inferior to Web Analytics.

I think this is an excellent example of what happens when someone seen as a leader in a field becomes too engrossed in what he is evangelizing…he becomes blind to the bigger picture.

The fact is that Web Analytics, though impressive in its power to aggregate user behaviour and use this to optimize website profitability, it is by nature a limited field. You are able to track user behaviour – generally anonymous at that – through a single customer-facing channel. Web Analytics is Business Intelligence, that only leverages a single source.

“Giant Data Warehouses,” however, are repositories of cross-organizational data, in most cases that extracted from up to hundreds of disparate data sources – Legacy systems, ERPs, CRM systems, finance, operations, HR, desktop apps, web services, external sources – and loaded into a database of a very specific architectural design optimized to return query results on the huge amounts of data very quickly.

Further, this data will certainly have different meanings across and organization – what does “Customer” mean? How do we define this? Part of the process is to work closely with the business to define common business definitions of business entities…so all that data of all that depth and breadth and richness is (should be….) based on common meanings that have been agreed to by key stakeholders. We can mine the data to identify unknown customer segments. We can do Predictive Modeling. Starting with a business mentality, there is the potential to leverage some powerful Business Intelligence.

But I do agree that Web-sourced data represents a substantial opportunity. We can take those Web-specific data sources that power our Web Analytics Apps, and add that to the existing Data Warehouse, passing through the same business rules to ensure heterogeneous data has a single meaning. Now we are talking organization wide, multi-source Business Intellligence.  Plug BI’s powerful analytical tools into our database, and with some targeted, business-driven KPI’s, and we have another, very powerful means of driving profitability

Web Analytics could be said to be proportionally less expensive than traditional BI – same basic cost range for the analytics tool, but less demand for investment in multiple software licenses from different vendors (possibly), less complex data massage (or not…) and shorter time to implement.  And that in itself is a strong argument in favour of Web Analytics – reduced time to market.  However, you won’t have the spectrum of information you have in a well-implemented Data Warehouse.

I believe that Web Analytics is a complement to BI. It can be integrated into a dashboard, or can stand alone to guide developers and webmasters to optimize content. It does have an effect on our database architecture – we must adapt the design of the database to integrate web data. But does it “change the game”? No – it  makes it more interesting. And as a Business Intelligence professional, I welcome another tool that will add value to my service offering and to my clients.

Wade Walker

Ideaca To Become Hitachi Solutions Canada!

Same people, same values, different name

On December 2, 2013, Ideaca was officially acquired by Hitachi Solutions and will become “Hitachi Solutions Canada.” As Hitachi Solutions Canada, we look forward to providing our customers with a wider array of proven industry solutions and access to global resources.

"Ideaca is extremely pleased to join a global brand with the outstanding caliber of Hitachi Solutions,” said Muneer Hirji, newly appointed president of Hitachi Solutions Canada.

“We look forward to integrating our experience and strengthening our synergies to bring great industry-focused solutions to both regionally-focused and multinational companies throughout Canada. With its long history of technology excellence, industry leadership and employee-driven culture, Hitachi Solutions will make a great home for our employees.”

Our name may be changing, but our people and our values will stay the same!

Click here to read more.