Social Analytics meet Business Intelligence

 Post written by Wade W., BI Consultant at Ideaca. Read more about BI on his blog: Pragmatic Business Intelligence.

If your company is a well-known brand, somebody, somewhere is publicly talking about it. Right now. It may be on your own Social channels, in an Internet forum, a blog or other user-generated content site. Social Media Monitoring, which put simply is keeping a constant eye on Social sites including Twitter, Facebook and hundreds of other platforms to monitor what is being said about your brand, has become a necessity for most large organizations, and it is an art and a science to manage this well.  Manage it badly, and you can have a catastrophic image issue (i.e. the 2010 Nestle Palm Oil debacle on Nestle’s own  Facebook page).  Handle it well, and you can cement a solid relationship with existing clients and convert new clients to your brand (i.e. HP’s little-known but truly brilliant efforts to provide temporary replacement HP hardware to certain individual users on social platforms complaining of broken computers).

From a commercial aspect, companies are increasingly looking at Social Media to contribute to driving revenue, largely through lofty concepts such as “engagement” and “conversion.” Social is unique in not only the speed of the communication, but also the intimate nature of content.  In addition, and importantly, what companies must understand is that in the Social realm, the customer controls the conversation. The implication here is a fundamental paradigm shift for Customer Relationship Management and Marketing, to understand the customer on a personal level, and to handle – with great sensitivity – both the positive and negative sentiment expressed on Social platforms.

(Social) Business Intelligence
A growing and compelling new flavor of Business Intelligence is attempting to tap into the unstructured content on social platforms and attempt to structure that data into a format that can be analyzed and mined using new methods such as Sentiment Analysis, which measures the aggregate sentiment across user posted content. Social Business Intelligence uniquely sits in the convergence of Knowledge Management, Social Media Monitoring, Collaboration, Social Networking, Analytics , Customer Relationship Management (CRM) and Business Intelligence (BI).

Social Business Intelligence is at a unique convergence point between several key technologies.

First, there is an important roadblock to get out of the way. Today there are a selection of tools to do everything I am discussing below in one way or another.  With a simple sentence I have rendered technology irrelevant for the purposes of this blog. So let’s focus on what Social BI is, how it is done and what it means because that is what is important to business.

I’m not really a catch-word kind of guy, but this is Big Data in its truest form. There are thousands of platforms and sites, of course, but if we only talk about  the current Big Guys (Facebook, Twitter and Foursquare for example), this would add up to billions or trillions of conversation segments over a given  (even conservative) time horizon. To put this in context: that customer data warehouse you have built over all these years probably doesn’t come close…

Social Business Intelligence offers both Internal and External Opportunity
There are both internal and external opportunities to be realized through Social Media Business Intelligence, and many tools are evolving to support these, some even going so far as to adopt a “Facebook-like” or “Twitter-like” interface, mimicking social interaction and Social Networking site features.

Social Business Intelligence applied internally to an organization could be termed Social Collaboration. For example, certain tools might feature collaborative review where colleagues can ask questions and link those answers to specific reports, or collaboratively comment and markup objects such as Business Intelligence ad-hoc analytics,  graphs or reports. This functionality to comment in real time on powerful business intelligence (even if it is only based on Traditional data sources that exclude Social Media data) has the potential to add value to interpretation of the reports that companies produce and use today to base key decisions upon, thereby potentially improving decisions made from today’s Decision Support Systems. Many traditional software vendors already have adopted such functionality.

Of course, where Social Business Intelligence as a disruptive technology becomes particularly interesting is when we start gathering and analyzing that unstructured user-generated content, or even more compelling, when we combine it with our existing “traditional” Enterprise Analytics environments. This empowers organizations to produce new innovative products that target user segments more accurately and respond better to customer support or relationship development opportunities. The value of Social Business Intelligence is not really “about” the frequency of words and phrases users post on social platforms. The value is in segmenting, categorizing, mining and understanding the aggregate of the users’ behavior, and the sentiment of those posts across products, segments and channels.

Social Media has its own unique segments, which include Employees, Partners, Influencers, Detractors and Advocates. We can analyze social network traffic, understand and identify our segments, and tailor personalized/semi-personalized interaction to individuals or one of these segments,  flagging key comments, monitoring Likes, +1’s, trending subjects and use of hashtags, enabling rapid and targeted response to user comments to avert public relations crisis, measure success of our Social Marketing programs or capitalize on new opportunities.

It’s all about the conversation. And you don’t control it.
Again, companies need to understand that the customer controls the conversation. However, the tools exist that can arrange and present structured knowledge from unstructured noise, providing key information input to areas such as Marketing and Manufacturing to be responsive and agile, acting on data that correlates highly to real-life fact.

At its root, Social Media is about the conversation. This implies new requirements for how to manage our link to the customer, and how to most effectively target and market to them. Increasingly, consumers are mistrustful of the marketing messages and advertising. They are more likely to find more relevance and see more value in the reviews and purchasing of their friends and peers.

Social Business Intelligence in Practice
I thought to finish, I would provide two examples that support the claim that through mining user-generated content, we can correlate with very high level of confidence, to known and validated facts.

Google Flu Trends
An example of single-source user generated content analysis is  Google Flu Trends.  Google has been analyzing aggregated web search terms to see if it is possible to correlate geographic frequency of user search terms on Google’s search engine to real data on flu epidemics.

While I recognize this is not Social Media  per se, this example is very relevant to the argument that user-generated content can be tied to sentiment and can also be used as a predictor for future events, when we clearly understand and define the objective, then identify and measure indicators supporting that objective.

Google’s site http://www.google.org/flutrends/ca/#CA provides up-to-current-day results to allow tracking of current and developing flu incidents and epidemics. In addition, on this site there are historical graphs over a multi-year period for regions around the globe that prove, using known, validated historical data, that reality and future events can indisputably be predicted by user-generated content.

United Nations Global Pulse.
Between 2009 and 2011, the United Nations and SAS studied how Social Media and other user-generated content from public internet sources such as blogs, Internet forums, and news published in Ireland and the US could be correlated to validated statistics and leveraged as a compliment and an qualitative indicator of real-life events.

For Global Pulse, the focus was on employment status. To summarize from the document found at http://www.sas.com/resources/asset/un-global-pulse.pdf,  the UN identified keywords indicating changes in employment status (i.e.”fired”), level of anxiety (i.e.  “depressed”) or economic indicators (i.e. loss of housing or auto repossession, cancellation of vacations) in order to  monitor sentiment.  The results were astonishing. The analysis of sentiment allowed them to predict  increases in unemployment as much as four months in advance of an uptick in unemployment claims with a 90-95% level of confidence. Further, they were able to predict precisely, again with a 90-95% confidence, how long after an uptick in unemployment that there would be an increase in clear economic indicators in the form of talk of loss of, or negative changes to housing, changes of transport method or cancellation of travel plans.

These two examples underscore that user-generated content in the social realm represents a new and potentially highly accurate source of knowledge when tied to clearly defined objectives and supporting metrics (leveraging appropriate keywords). Indeed, Social Media Business Intelligence has the potential  to facilitate very personal customer understanding and when backed by a well defined strategy, to strengthen the relationship with our customer, avert PR disasters and increase customer engagement and conversion.

What are your thoughts? Is the world ready for Social Business Intelligence? Has your company thought about imposing order and structure to the chaos that is Social Media user-generated Content?

You've Collected Data...But Now What?

Post written by Peter T., Management Consultant at Ideaca. Read more about visibility on his blog: Visibility.

The list of technologies that allow us to capture vast amounts of data is quite extensive. This list varies in magnitude of use and exposure within organizations. Companies today can, and most often do, use multiple means of collecting data, such as: Spreadsheets, Databases, Operational specific Software, Enterprise Systems; ERP, CRM, HRM, Various Portals; Personal Portals, News Portals, Enterprise Information Portals, Self-Service Portals, e-Commerce Portals, Collaboration Portals… And the list goes on and on.

It is very evident that companies are really good at collecting data. Whether the data management function within an organization is primitive or advanced, gathering data in spreadsheets or in elaborate enterprise systems and databases: the majority of organizations are great at data collection. Hard copy, Soft Copy, e-Copy, web displayed; data in all forms, shapes and sizes is being collected at an enormous pace. If you can write it, print it, draw it, type it, sketch it, draft it, and capture it, you can rest assured it is being gathered.

The question is not what data to capture next, but now that we have all this data, NOW WHAT?  

Once data is collected, do organizations use it in the most efficient way? The overarching question is: now that you have all this data, what value are you getting from it? The following are five steps that will assist organizations in gaining the most value out of their data.

STEP 1 – IDENTIFY YOUR VALUE DRIVERS

Before we can successfully answer the question of value derived from data, we need to understand what the value drivers are for an organization. Are the value drivers; profitability, reputation, market share, productivity, customer service? The list can certainly be expanded upon. Getting value out of your operational data is imperative, but if you don’t link the data that you are capturing with the value drivers of the organization, you could be spinning your wheels and not realizing the full potential of your systems and efforts.

STEP 2 – LINK DATA TO YOUR VALUE DRIVERS

The next step to ensuring you are making intelligent decisions based on relevant information is to verify that all data captured is linked to the value drivers of your organization. Every piece of information that is collected and processed is intended to provide new intelligence, thereby improving the positive outcomes of critical operational decisions. The way to optimally perform this is by linking significant data retrieval and performance functions to your value drivers. Furthermore, these links can be expanded upon where multiple associations exist.

Dissecting the specific data captured will allow organizations to assess data accuracy, timeliness, depth, and most importantly the interconnection with various other data sets and systems. The key is to ensure that crucial data is modeled to display how it is gathered, at what interval, and how data from one source is related to data in another.

STEP 3 – ANALYZE

Now that you have modeled all significant operational data, you will be able to focus on the highest impacting pieces. By designing new processes or re-engineering solutions, you will be able to increase the usefulness of the information. The analysis will be focused on interconnecting data, assets, management, and operational systems. This exercise will require a thorough look at the data to ensure that standards are in place and the collection of information is from across the entire organization in order to ensure corporate-wide accurate reporting. The outcome from the analysis is to design a roadmap that will focus on operational improvements tied directly to the value drivers of the organization. This can be initiatives such as: identifying ways to increase production, improve safety records, decrease maintenance costs, improve asset visibility, reduce compliance risk, and much more.

STEP 4 – SOLUTIONING

After defining opportunities to improve operations, organizations need to devote some time to developing a realistic plan of achieving these goals. A key step in the Solutioning process is developing the overall vision and detailing the various components of development in palatable sizes ready for execution. Increasing the capabilities of the organization through the design of new automated systems or enhanced analytics, processes and interfaces are just some of the improvements that can be realized. If structured properly, these enhancements can provide the organization significant wins by capitalizing on the information captured along the way.

Information Technology has assisted organizations in navigating from simple and non-existent data management environments, to an optimized level where data can be used for benchmarking and analysis to drive their strategic and operational initiatives. This cannot be successfully done however without ensuring that all data captured provides value and that value is something that drives the automation, analysis and design of advanced systems and integration opportunities. The following diagram depicts the stages of data management and provides a visual of where organizations currently are and how far they may have to go in order to achieve the most optimal level of data management:

A hike gone awry as an analogy for Troubled Project Leadership

Post written by Jason Z., Project Manager at Ideaca. Read more about project management on his blog: Unnatural Leadership.

I have two fortune cookie fortunes on my desk at home – “promise only what you can deliver” and “now is a good time to finish up old tasks.” They are taped to the bottom of a picture frame with a picture of my wife and I from the day that she had a catastrophic accident hiking in the mountains. 

We were hiking “Bow Peak” with a friend in the middle of the summer and  decided to summit the peak. When we reached the peak, we realized that there were three paths down, but we had no guidance as to which path to take. Our goal was to get to the base of the summit, which connected to the safe path down. The first path was the one we came up, and it was all scree.  We didn’t really have the right gear to descend safely. The second path was down a rock chute to a wide open path, and we would have to walk an extra 15 km to get to the base of the summit. The third path was also down the rock chute, but it turned to a side of the mountain that we could not see.  However, we could see that it was a shorter route and would take us over some less dangerous scree to get us back to the base of the summit.

As a group, we identified the problem, recognized the constraints, discussed the risks of each path, and then chose the third path for our descent. Before we began, we agreed on some of our guiding principles for descending – such as only one person in the chute at a time so as to avoid being hit by tumbling rocks.

During our descent, we came across a part of the path that was previously unknown – we had to traverse and descend a small glacial formation. I went first, and having experience snowboarding, sat down on my heels and enjoyed the 60 foot slide. My wife (then girlfriend) went next, but did not have the same experience. She slid out of control and ended up breaking two teeth and puncturing her lip on a flat top rock. As she was sliding, I was trying to give her all of my knowledge of how to control the uncontrolled descent by yelling four words – “dig in your heels.” She yelled back “I am! I am!”  I didn’t offer her how to do so, or what else to do with her body when she was doing so. And so, as she was sliding, I was sprinting across the base of the ice to catch her before she hurt herself. I was too late, and we ended up spending the night in hospital followed by the next morning with a dentist to perform some emergency repairs.  The final result was that she endured 20 stitches in her lower lip and now has 2 false teeth.  While she will still come hiking, she is not as exuberant to go as she once was.



Our friend – who happened to have a med kit – ended up taking the 'safe' route down by walking down the side of the glacier where it was primarily slush.

I have never forgiven myself for that accident, but have never thought about the leadership lesson associated with it. In the short time span of her fall, I could not even begin to communicate the depth of my experience to ensure that she would be successful with her journey. I had just assumed that it would be fine if I showed her how to do it. When I saw she wasn’t getting it, I started yelling louder hoping she would get it.

By reflecting on this hiking experience, I now understand some of the more tumultuous projects that I have managed and participated on:

1. As we prepared for our descent, we discussed our guiding principles for descending. Creating a shared vision through detailed planning is the logical way to get a team ready to move forward with a project. However, I have observed, and participated on, project teams that just move forward without considering what could go wrong.
2. Project teams have disparate skill sets. While some members may be perfectly happy to “descend the glacier,” others may not be. Ensure that teams have a safe path to move forward without getting hurt.
3. When the pressure is high and time is short, yelling does not help anything.

My lesson to be applied is thus:

1. Be deliberate with your plans
2. Consider skill sets
3. Having someone with experience walking the path can provide helpful guidance, but they do not necessarily add value when they are “in charge.”

Long hikes are a lot like projects. To descend the mountain safely, you need to honestly assess team skills, previous experience, and know how to prevent catastrophic accidents. Never walk the path blindly.

Setting expectations with clients (part 2)

Post written by Steve J, Project Manager at Ideaca

Setting Expectations with Clients Part 2 is a two part blog series on managing client expectations. See here to read part 1.

In my previous blog post, I told a story about a project I was involved in that required expectations to be managed and the project to be pivoted. In this blog post, I will discuss four key factors in managing expectations that I have learned throughout my career.

1. Communication is a key factor when working with clients. No two clients are the same, some will hire a team to complete a specific task, while others will want to be fully engaged. Learning how and when to communicate with these different groups is crucial to project success. Fully engaged clients may require daily or even hourly updates on project status while more removed clients may only want occasional updates. Knowing your client and making decisions on when and how to communicate is an important way to manage expectations. Some of the effective communication mechanisms that we use on projects can include: daily stand-up meetings with the entire project team (including the client), weekly status reports with all items completed that week, any issues or decisions that arouse, and a budget burn down showing progress. Email is a very handy tool for communication, however if a portal (SharePoint or something similar) is available, this technology will allow for much tighter control over issues, questions, and decisions on the project, without the issues with email branching off into numerous threads and side conversations.

2. Building a relationship on honesty is necessary. Right from the first meeting, the client should understand what you can and cannot do for them. As much as we wish we could offer solutions to every problem, the reality is we can only do what is within scope and within our knowledge and skills. While digging into the details of what that desired end state will be, it is important to discuss what is possible and what is not. These discussions will affect the project and the decisions made will impact the deliverables, the timelines and especially the budget. The hope of this is to catch and identify any areas of the project that may lead to deliverables not being delivered, budgets and time frames expanding and missing expectations.

3. Deciding the roles and responsibilities for the project team (including the client team members as well) is a major task that should be completed at the start of the project. From the start of the project, setting up a project Governance is a great way to start. Project Governance will outline the relationships between all groups involved in the project, define the flow of information from the project to all key stakeholders, and ensure that there is appropriate reviews of all issues during the project. Another useful tool is to create a RACI Matrix. A RACI matrix describes the participation by various roles in completing tasks or deliverables for a project or business process. It is especially useful in clarifying roles and responsibilities in cross-functional/departmental projects and processes. Key contacts for the different areas of the project should be also defined. It’s important for everyone to know who to go to for questions and issues with certain aspects of the project and to keep these people consistent from kick off to go live. In projects I have been on, we start the project with an internal kick off meeting before we meet with the client. This ensures everyone is on the same page and has a shared understanding of the project and statement of work. On the first day of work with the client, we begin with a similar meeting. At this point we can assign the key contacts on my project team as well as the clients’ team. Everyone involved on both sides will have had chance to meet and put a name to the face and to their role.

 4. “When is the due date and when can we launch this project?” should be questions that are asked and answered early on. Making sure to develop a realistic project plan adds transparency to the project, shows when the milestones are due and provides everyone responsibilities and accountabilities. The project plan is a living document. It should exist to provide everyone an up to date snapshot of where things are in the project. If there are delays or changes made, the project plan should be updated and discussed with the client immediately. When it comes to managing expectations, establishing clear and consistent deadlines are a necessity. The sooner deadlines are set, the sooner the team can begin to work to ensure they meet them.

Every new project allows for lessons learned or growth, both for the team and the individual. Personally, I have learned so much about managing expectations from every project I have ever worked on. Every client is different and understanding how to work with them is part of having a successful project. Communications, honesty, assigning tasks and confirming deadlines are four key aspects of managing expectations that are part of my expectation management strategy for every project. Ultimately you want to start the project the same way that you finish it: with happy clients!

The data has the answers

Post written by Evan Hu, Co-founder of Ideaca. View his blog here: evanhu.wordpress.com

In a 2001 research report by META Group, Doug Laney laid the seeds of Big Data and defined data growth challenges and opportunities in a “3Vs” model. The elements of this 3Vs model include volume (the sheer, massive amount of data or the “Big” in Big Data), velocity (speed of data processed) and variety (breadth of data types and sources). Roger Magoulas of O’Reilly media popularized the term “Big Data” in 2005 by describing these challenges and opportunities. Presently Gartner defines Big Data as “high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.” Most recently IBM has added a fourth “V,” Veracity, as an “indication of data integrity and the ability for an organization to trust the data and be able to confidently use it to make crucial decisions.”

The volume of data being created in our world today is exploding exponentially. McKinsey’s 2012 paper “Big data: The next frontier for innovation, competition, and productivity” noted that:

• to buy a disk drive that can store all of the world’s music costs $600
• there were 5 billion mobile phones in use in 2010
• over 30 billion pieces of content shared on Facebook every month
• the projected growth in global data generated per year is 40% vs. a 5% growth in global IT spending
• 235 terabytes data was collected by the US Library of Congress by April 2011
• 15 out of 17 sectors in the United States have more data stored per company than the US Library of Congress

IBM has estimated that “Every day, we create 2.5 quintillion bytes (5 Exabyte) of data — so much that 90% of the data in the world today has been created in the last two years alone." In their book “Big Data, A Revolution That Will Transform How We Live, Work, And Think,” Viktor Mayer-Schonberger and Kenneth Cukier state that “In 2013 the amount of stored information in the world is estimated to be around 1,200 Exabytes, of which less than 2 percent is non-digital.” They describe an Exabyte of data if placed on CD-ROMs and stacked up, they would stretch to the moon in five separate piles.

This sheer volume of data presents huge challenges. For time-sensitive processes such as fraud detection, a quick response is critical. How does one find the signal in all that noise? The variety of both structured and unstructured data is ever expanding in forms: numeric file, text documents, audio, video, etc. And last, in a world where 1 in 3 business leaders lack trust in the information they use to make decisions, data veracity is a barrier to taking action.

The solution lays ever more inexpensive and accessible processing power and the nascent science of machine learning. While Abraham Kaplan (1964) principle of the drunkard’s search holds true: “There is the story of a drunkard, searching under a lamp for his house key, which he dropped some distance away. Asked why he didn’t look where he dropped it, he replied ‘It’s lighter here!’” A massive dataset that all has the same bias as a small dataset will only give you a more precise validate of a flawed answer, we are still in early days. Big Data is the opportunity to unlock answers to previously unanswerable questions and to uncover insights unseen. With it are new dangers as the NSA warrantless surveillance controversy clearly exposes.

I have had the privilege of listening to Clayton Christensen speak several times. In particular he has one common through line that stuck with me and forever embedded itself in my consciousness. “I don’t have an opinion. But I have a theory, and I think my theory has an opinion.” I believe the same for Big Data. The data has an opinion, the data has the answers.

Setting expectations with clients

Post written by Steve J, Project Manager at Ideaca

The Oxford Dictionary defines “managing expectations” as: “Seek to prevent disappointment by establishing in advance what can realistically be achieved or delivered by a project, undertaking, course of action, etc.”

Almost every part of our lives is surrounded by expectations, either ones we set for ourselves or ones that were set for us by others. When it comes to consulting and being on a project, every part of the project experience will be influenced by expectations. There will be expectations around the project as a whole, the deliverables, the time and especially the budget. It is the responsibility of the project team and Project Manager to ensure that the client has a clear and accurate understanding at all times. The overall success of any project will be linked to the expectations of the client, the understanding and efforts of the project team and how well these factors align. At the end of a project, the client’s satisfaction with the delivered project will determine its success.

I have worked on a variety of projects, including those with high expectations from the client. In one project the client as a whole had very little technical and user knowledge of the system that we were implementing for them. They were very much involved in the project and took on many tasks. One of the tasks that was completed by the client was the design and layout of the new system. This task was completed before the project team started on the project and with limited knowledge of the system. The designer was able to design the pages to match that of a SharePoint look and feel without any operational knowledge of the system as a whole. The project team was not a part of this design and the client wanted the end result to look and operate the same as they had designed it. When our project team realized this, we had to pivot our work to align with a more customized solution rather than an out of the box implementation as originally expected. Communication and managing expectations became a critical component of this project, especially since the plans and timelines shifted substantially. Working closely with the client, being honest about timelines and budget and reviewing changes before work continued were very important. Our open communication kept the client informed and the project team on the right track to meeting their needs. In conclusion, the client was very happy with the end product and assured us that we had exceeded their expectations.

Through my experience working with a variety of clients in different situations, I have developed an understanding of how to best manage client expectations. As in the example above, the situation could have turned out with one, or both parties upset about the changes in plans. But by effective expectation management, we were able to explain to the client why things needed to change and what exactly we were going to change. This turned a potentially problematic shift in work into a positive improvement of work.

In my next blog entry, I’ll cover the four key factors in managing client expectations that I have learned throughout my career. Check back next Tuesday, September 17 for more!