Web Analytics supplants Business Intelligence?

Post written by Wade W., BI Consultant at Ideaca. Read more about BI on his blog: Pragmatic Business Intelligence. 

In reading industry material, I recently came across a statement that can only be, in my opinion, the product of tunnel vision. It was one of the most short-sighted and fundamentally erroneous statements I have seen in some time. Analytics

“At the 2005 Emetrics Summit in London, Bob Chatham from Forrester Research described what it means to be the key. He told the assemblage that we are the leaders of tomorrow – and he wasn’t just preaching to the choir to curry favor – he made sense. Chatham told us that “web analytics” would eventually be subsumed into business intelligence, thereby changing the game. Instead of giant data warehouses being sifted in hopes of finding patterns, it would be the likes of us web analysts in charge.” (Jim Sterne, Target Marketing of Santa Barbara, edited by Erika Lindroth, The Weather Channel Interactive, Inc.)

I agree that web analytics will be (and is starting to be) subsumed into BI. However, I question the sentiment that “giant data warehouses [are] being sifted in hopes of finding patterns” and that Web Analytics would “change the game.” Is Web Analytics really going to revolutionize the art of Business Intelligence so significantly? The implication in this quote is that somehow traditional Business Intelligence is somehow inferior to Web Analytics.

I think this is an excellent example of what happens when someone seen as a leader in a field becomes too engrossed in what he is evangelizing…he becomes blind to the bigger picture.

The fact is that Web Analytics, though impressive in its power to aggregate user behaviour and use this to optimize website profitability, it is by nature a limited field. You are able to track user behaviour – generally anonymous at that – through a single customer-facing channel. Web Analytics is Business Intelligence, that only leverages a single source.

“Giant Data Warehouses,” however, are repositories of cross-organizational data, in most cases that extracted from up to hundreds of disparate data sources – Legacy systems, ERPs, CRM systems, finance, operations, HR, desktop apps, web services, external sources – and loaded into a database of a very specific architectural design optimized to return query results on the huge amounts of data very quickly.

Further, this data will certainly have different meanings across and organization – what does “Customer” mean? How do we define this? Part of the process is to work closely with the business to define common business definitions of business entities…so all that data of all that depth and breadth and richness is (should be….) based on common meanings that have been agreed to by key stakeholders. We can mine the data to identify unknown customer segments. We can do Predictive Modeling. Starting with a business mentality, there is the potential to leverage some powerful Business Intelligence.

But I do agree that Web-sourced data represents a substantial opportunity. We can take those Web-specific data sources that power our Web Analytics Apps, and add that to the existing Data Warehouse, passing through the same business rules to ensure heterogeneous data has a single meaning. Now we are talking organization wide, multi-source Business Intellligence.  Plug BI’s powerful analytical tools into our database, and with some targeted, business-driven KPI’s, and we have another, very powerful means of driving profitability

Web Analytics could be said to be proportionally less expensive than traditional BI – same basic cost range for the analytics tool, but less demand for investment in multiple software licenses from different vendors (possibly), less complex data massage (or not…) and shorter time to implement.  And that in itself is a strong argument in favour of Web Analytics – reduced time to market.  However, you won’t have the spectrum of information you have in a well-implemented Data Warehouse.

I believe that Web Analytics is a complement to BI. It can be integrated into a dashboard, or can stand alone to guide developers and webmasters to optimize content. It does have an effect on our database architecture – we must adapt the design of the database to integrate web data. But does it “change the game”? No – it  makes it more interesting. And as a Business Intelligence professional, I welcome another tool that will add value to my service offering and to my clients.

Wade Walker

Ideaca To Become Hitachi Solutions Canada!

Same people, same values, different name

On December 2, 2013, Ideaca was officially acquired by Hitachi Solutions and will become “Hitachi Solutions Canada.” As Hitachi Solutions Canada, we look forward to providing our customers with a wider array of proven industry solutions and access to global resources.

"Ideaca is extremely pleased to join a global brand with the outstanding caliber of Hitachi Solutions,” said Muneer Hirji, newly appointed president of Hitachi Solutions Canada.

“We look forward to integrating our experience and strengthening our synergies to bring great industry-focused solutions to both regionally-focused and multinational companies throughout Canada. With its long history of technology excellence, industry leadership and employee-driven culture, Hitachi Solutions will make a great home for our employees.”

Our name may be changing, but our people and our values will stay the same!

Click here to read more.

How Technology Changes Us: Canada In 10 Years

Post written by Niaz T., Senior Solution Architect / SAP BW Consultant at Ideaca. Read more about SAP HANA on her blog: Discover In-memory Technology.

The theme of the Ideaca Blogging Network for the month of August is a very interesting subject. Certainly, technology changes the way we do things on a daily basis. Not only in Canada, but also globally. There could be some specific cases in Canada, such as Green technology to combat climate change. However, most of the technology changes impact us globally, specially in more advanced countries.

The first thing that it comes to my mind is technology will enable us to convert Zettaflood (10 to the 21st bits, or a thousand exabytes) of data in to meaningful information which we are very dependent. Like it or not, business intelligence already has an increasingly important part of our life. The challenge will be how to deal with the explosion of data coming from all types of gadgets and smart technologies because value-based intelligent information helps us to get things faster, better and easier. The speed of rising adoption of cloud, mobile, real-time applications and social technologies and exponential data growth is a big challenge of staying current.

How technology solves this challenge? Some of the biggest improvements have been around networking. We will be able to move more data faster from many sources and applications to where is needed. We won’t have any restriction in terms of capacity, scalability and processing speed. Organizations will be able to leverage the three “V’s”, volume, variety and velocity, of data to augment the value of data for their decision making. Powerful in-memory technology such as SAP HANA enable us to design complex predicative and preventative models for all type of data from structured and unstructured like audio and video files. Next generation of data visualization and intelligent reporting tools empower users to slice and dice information any way it is demanded. We will be able to tell stories with data by connecting millions of data points to get a bigger picture. Big data will change our world and it will blow our mind by providing us tons of opportunities. It will make our word smaller and we will be all connected.

I believe in the next 10 years, another significant change will be human and machines interaction. It seems that human interaction, communication and relationships will be more efficient, faster and stronger through smart technologies. Also, we will be able to have better understanding of machine behaviors and machines will have a better understanding of ours. Ideally humans and machines will work alongside each other and hopefully not replacing human with machines. Although there are ongoing developments and opportunities to replace human with machines, it’s required to consider all potentials dangers and associated risks.

Personally, I’m very excited to see how technology will enable us to access information easily, increase our potential and creativity, improve our lifestyle and promise of longevity, and improve communication and social networking. On the other hand, I believe we need to keep things in balance with respect to human identity and our social behavior. For example, neuroscientists are concerned about how modern technology is making us not use our brains to their full potentials. Based on the evidence, loneliness and depression is increasing and people are less happy in modern society. It’s been observed that the newer generation—equipped with all kinds of smart technology—is less effective in terms of communication skills and human interaction.

The bottom line is we use technology to change the world to suit us better. The important thing is to control it so it doesn’t destroy human intelligence and social interaction. For instance, it would be great to get a relaxing massage after a long day by a smart robot that has already taken care of the house chores. However, nothing will replace a nice face to face conversation with your favorite person or a warm friendly hug to someone you care about. I don’t think we could ever replace our human connection with human-robot connection.

Is Big Data Only About…Big Data?

Post written by Wade W., BI Consultant at Ideaca. Read more about BI on his blog: Pragmatic Business Intelligence.  

If nothing else, IT is all about buzzwords, and “Big Data” is one of the new arrivals to the party.
It is, however, a descriptive one. “Big Data” evokes images of enormous relational databases, providing analytical (or operational) reporting.

Big Data is not only about size however. Rather, it refers to attributes of the data that together challenge the constraints of a business need or system to respond to it. Those attributes can include any or all of attributes such as size/volume (of data), speed (of generation), and number and variety of systems or applications that simultaneously generate data. Another thing that is unique about Big Data is how it varies in structure. Elements of “structure” would include the diversity of its generation (eg. Social media, video, images, manual text, automatically generated data, such as a weather forecast, etc), information interconnectedness and interactivity.

I heard somewhere a thumbnail statistic that 80% of data in companies is unstructured or semi-structured. Just to clarify the meanings of those terms, an unstructured data artifact would be a document, an email, a video or audio clip. A semi-structured data artifact would include data that does not conform to the norms of structured data but contains markers or tags that enforce some kind of loose (or not so loose) structure. XML documents would be an example of semi-structured data.  Tagged documents in a Knowledge Management system would also fit into this definition.

Structured data is what we would find in any database – a Data Model has been defined and the data is physically arranged within this model into tables. The data in these tables is described with metadata (i.e. data types (such as “character”) and the maximum length of that data (number of bytes)).

The methods of data creation are multiplying and the velocity of its creation are increasing. And that, in itself is a complicating factor. Some analysts (IDC, for example), predict that the Digital Universe -  that is, the world’s data – will increase by 50x by 2020. There will be in the same period, a growing shortage of storage, which will drive investment in the cloud as both individuals and corporations look for scalable, ubiquitously accessible, lower-cost and environmental data storage options. In addition, the same study predicts that of all that data, unstructured data, especially video, will account for 90% of that data.

There is also an important historical dimension to Big Data. For decades, companies have been hoarding structured, semi-structured and unstructured data in hopes of one day being able to extract value from it at some point in the future.

A large percentage of all this data will come with a wrapper of automatically generated Metadata – that is, (as indicated above), data about (or that describes) that data. A practical example could be the generation of a data artifact coming wrapped with metadata from those GPS enabled, media rich, socially linked mobile devices we all carry with us that transparently capture location, GPS coordinates, time, weather conditions and a plethora of other data elements when you click that holiday photo with your mobile phone. IDC predicts that such metadata is growing twice as fast as data.

It is clear from the last three paragraphs that Big Data describes explosive growth in data and metadata and an equally explosive opportunity to capture, tame and corral that data to extract value from it.

So the case has been made that we have a lot of data today and we will have even way more tomorrow, but should your organization be investing in Big Data today?

In a sense, probably you already are. Enterprise Business Intelligence environments lay a solid foundation for the next phase of Big Data. EBI is an earlier iteration of Big Data and, married to tools such as Hadoop and NoSQL databases for example, enable a natural evolutionary growth curve to your mastery of your information ecosystem.

Big Data has a requirement for a new way of thinking, new tools, clustered commodity hardware and probably, substantial investment. It comes down to your business, and if there is a clear value-based case to present that data to your company’s brainpower. The actual needs for this will be radically different depending on your industry. Oil and Gas may be interested in leveraging real time alerts in wellhead data or analyzing petabyte seismic datasets.  Packaged Goods multinationals may be interested in monitoring and engaging advocates, detractors and influencers across multiple Social Media platforms, mining and understanding sentiment and identifying problem areas in real time in order to identify opportunity or identify and avert potential brand-damaging events. Financial institutions may be interested in monitoring international money traffic to identify fraud or illegal activity. Government entities may mine extremist forums, or other unstructured data traffic to identify national threats.

Big Data can serve these needs in real time, enabling rapid (or even automated) response to flagged events. Whether it is a fit for your organization today would be determined through viewing your industry and business through a critical lens on your current Information Intelligence maturity, a strategic assessment of the data and information assets currently owned or available to your organization, and a prioritization of potential initiatives. How much data you harness and convert into information should be a key outcome required from this exercise. The opportunities are legion, but initiatives should have clear objectives and success metrics understood prior to a project kickoff.
Whether it is today or tomorrow, Big Data is becoming mainstream through necessity. Whether that is a road your organization wants, or needs to drive today, is something all medium and large organizations should be considering now.

What are your thoughts on Big Data? Is your organization currently considering Big Data as a strategic imitative or Proof of Concept?

Project Management and Big Data – as a project

Post written by Jason Z., Project Manager at Ideaca. Read more about project management on his blog: Unnatural Leadership.

As part of this month’s Ideaca blogging network challenge, we were tasked with discussing our thoughts on Big Data.

This is going to be a 2 part post:

• The first part will cover how you, as a project manager, should approach a project that carries the mantle of “Big Data.”
• The second part will cover how you, as someone in a Project/Program Management Office, can use Big Data without getting snookered by the hype.

Part 1 – So you’ve been asked to “implement Big Data”… what now?

Defining Your Terms
I am going to assume that you – like me – tend to be baffled by the marketing speak until you can speak with someone intelligently about a topic. In the case of Big Data, I have heard a few definitions. The one that seems to stick the most for me is the one from Wikipedia:

• Data sets that are too big for traditional database management systems to handle
• Data sets that comprise information from multiple sources to try to infer correlation

Sounds easy enough, right?
Where it starts to get complicated (thanks Wade!) is when you try to integrate “unstructured and semi-structured data with our 'traditional' structured data.”

You will never “implement Big Data”
When it comes to Big Data, you do not implement it. You may be implementing a technology to support the analysis, but you will never actually implement this “thing.” A project of this sort relies on understanding the user requirements, selecting the right technology, and taking an exploratory approach when developing reporting capabilities.

Understanding the User Requirements
In the case of a new process and technology, such as this, your user requirements may be fairly light. "We want to correlate information from disparate sources to identify predictive trends” or “I don’t know – but I really want some cool looking reports” may be common lines that you hear. Like all projects, the user requirements are your definition of success. Because “Big Data” is still a technology in the exploratory stage, though, expecting detailed requirements may be the wrong sorts of requirements. The ones that you should be really focused on are the data sources and ensuring that the information being presented is right.

To wit, if I were to ask you to present the information on the average CEO compensation for the top 50 companies in North America, how would you start? How would you define the Top 50?  By Market Capitalization? By Environmental Performance? By Stock Price? By Revenue? What about getting access to private company information? All of the sudden, a fairly simple question about the average CEO compensation gets a little more complex.

The same will be true of your Big Data project. Start by understanding that to present the information your users want, you will either have to ask a whole lot of detailed questions, or provide a platform to enable them to answer their own questions.

Understanding the available technology
As Project Managers, we know that when we are asked to Implement something, it’s never that simple. Understanding what the technology can and cannot do is critical to ensuring that your project can meet the user’s definition of success.

One might want to satisfy the guiding principles of a company’s Enterprise Architecture. A quick scan of the landscape will reveal that tools like SAP HANA, Oracle’s Exadata, and Amazon’s AWS can all fulfill the technology requirements quite nicely and potentially support a company’s Enterprise Architecture. However, since this is a new application of technology, fulfillment of requirements needs to trump Enterprise Architecture.

Take an Exploratory and Iterative Approach to reporting
Some organizations will judge success of your project by its ability to deliver a load of reports. If this sounds like your organization, be realistic as to what can be delivered. Deliver a robust and reliable dataset, some transactional reports, and one report that really helps demonstrate the art of the possible.

Smarter organizations will judge the success of your project by its ability to deliver analytic capabilities to the user base. The robust and reliable dataset is still mandatory, but the ability for users to generate their own reports will satisfy all of the “what about …?” requirements that would blow your project budget and schedule out of the water.

In the end… it’s the people that matter
If we believe all of the marketing hype, Big Data will help us explore all the myriad of ways our world is constructed. But from the perspective of a Big Data as a project, an empowered user base will produce much more value than some canned reports.

Have you been asked to “implement big data”?
If so, what did your project look like? Let me know in the comments down below. Stay tuned for another post on making the most of Big Data in a PMO.


Special thanks to Wade Walker and Chris Sorensen for keeping me honest with this post.

Just Imagine...

Post written by Chris S., BI Consultant at Ideaca. Read more about BI on his blog: The Outspoken Data Guy.

For quite some time I have been imagining what the possibilities of Big Data might be. I am certainly no expert in the area but being the data guy that I am, I often wonder what might be able to be done with data that may be being collected at any point in time. Face it, we are so connected now that our every move generates some form of data and often multiple pieces of it.

For example, if a marketer wanted to know everything about Chris Sorensen in a given day, chances are that most of that data is logged somewhere. What time I leave my house is available via my cell phone, my driving directions and speed are also available there as well. When I sit on the train I surf the web, send emails and organize my task list, all of these actions generate recorded data. What time I log into work, how often I am active on my computer and what I am do all day long is logged. Where I shop, what I buy (if I have a rewards card) is all tracked. My Facebook views, tweets all contain things that could be used to build a personality model of myself and my habits.

It is not really that big of stretch to think that this data could be used in one gigantic model to predict my next move and perhaps even entice me to make a different one. Maybe instead of stopping at Home Depot to get my painting supplies, an app could suggest the best place for me to go based on what I am doing. Sound like a stretch? Not really…Think about the labor that gold miners went through just to get a few stones. Now gigantic machinery does the same thing. The same thing is happening with Big Data where machines are able to gather information from a variety of sources and store large volumes of it in order to form predictive models. We are only at the tip of the iceberg but just imagine what the possibilities might be

The importance of a shared vision

Post written by Jason Z., Project Manager at Ideaca. Read more about project management on his blog: Unnatural Leadership.

In a post from my series “Advice for Junior PMs," I touched on the concept of saying what you mean when working with your project team. The same concept should be applied when communicating outside of your project team.

There’s a fairly common graphic that gets passed around IT departments, and it’s somewhat self-deprecating. It shows that project teams tend to not understand what the customer needs – which is endemic of lacking a shared vision.

This graphic makes me cringe every time I see it.

As we all know, a project is a temporary group activity designed to produce a unique product, service or result. However, more often than not, project teams take an “I know best” view of the world when designing solutions for their customer.

A strong project manager will not only sit with their customer to understand what is required, but will bring the whole project team along to understand as well. We all have our own perceptions and filters, and as a result may play broken telephone.

At this point, you may be asking if a shared vision is different from the project scope statement. It is, in that the shared vision is what the customer will see as the product, service, or result of the project, whereas the project scope is everything that will be delivered (including training, documentation, organizational change management).

To create a shared vision of what the project will produce (be it a unique product, service, or result):

1. Bring everyone to the table to ensure open communication

1. Define what is to be produced in simple language – do not say “we are going to produce a tree swing,” and leave it there, say “we are going to produce a tree swing, which is comprised of a tire hanging from a sturdy branch of a large oak tree by a piece of polyester rope.”

1. Involve the customer in design meetings. Subject Matter Experts (SMEs) should definitely lead, but should be eliciting feedback so that the customer’s requirements are re-confirmed by the team.

1. Revisit the shared vision often. Ask your customer at difference acceptance testing points if what is being developed meets the shared vision.

Most importantly, communicate the shared vision often. Use it as the first line in your status reports, use it as part of your elevator speech, and when people ask you what you are working on, relay your project’s shared vision.

What are your tips for creating a shared vision? What have you seen work well? Do you have any stories of spectacular failures? Share your tips and stories below!