Post written by Jason Z., Project Manager at Ideaca. Read more about project management on his blog:
Unnatural Leadership.
As part of this month’s
Ideaca blogging network challenge, we were tasked with discussing our thoughts on Big Data.
This is going to be a 2 part post:
- The first part will cover how you, as a project manager, should approach a project that carries the mantle of “Big Data.”
- The second part will cover how you, as someone in a Project/Program Management Office, can use Big Data without getting snookered by the hype.
Part 1 – So you’ve been asked to “implement Big Data”… what now?
Defining Your Terms
I am going to assume that you – like me – tend to be baffled by the
marketing speak until you can speak with someone intelligently about a
topic. In the case of Big Data, I have heard a few definitions. The
one that seems to stick the most for me is the one from
Wikipedia:
- Data sets that are too big for traditional database management systems to handle
- Data sets that comprise information from multiple sources to try to infer correlation
Sounds easy enough, right?
Where it starts to get complicated (thanks
Wade!) is when you try to integrate “unstructured and semi-structured data with our 'traditional' structured data.”
You will never “implement Big Data”
When it comes to Big Data, you do not implement it. You may be
implementing a technology to support the analysis, but you will never
actually implement this “thing.” A project of this sort relies on
understanding the user requirements, selecting the right technology, and
taking an exploratory approach when developing reporting capabilities.
Understanding the User Requirements
In the case of a new process and technology, such as this, your user
requirements may be fairly light. "We want to correlate information
from disparate sources to identify predictive trends” or “I don’t know –
but I really want some cool looking reports” may be common lines that
you hear. Like all projects, the user requirements are your definition of
success. Because “Big Data” is still a technology in the exploratory
stage, though, expecting detailed requirements may be the wrong sorts of
requirements. The ones that you should be really focused on are the
data sources and ensuring that the information being presented is right.
To wit, if I were to ask you to present the information on the
average CEO compensation for the top 50 companies in North America, how
would you start? How would you define the Top 50? By Market
Capitalization? By Environmental Performance? By Stock Price? By
Revenue? What about getting access to private company information? All
of the sudden, a fairly simple question about the average CEO
compensation gets a little more complex.
The same will be true of your Big Data project. Start by
understanding that to present the information your users want, you will
either have to ask a whole lot of detailed questions, or provide a
platform to enable them to answer their own questions.
Understanding the available technology
As Project Managers, we know that when we are asked to Implement
something, it’s never that simple. Understanding what the technology
can and cannot do is critical to ensuring that your project can meet the
user’s definition of success.
One might want to satisfy the guiding principles of a company’s
Enterprise Architecture. A quick scan of the landscape will reveal that
tools like SAP HANA, Oracle’s Exadata, and Amazon’s AWS can all fulfill
the technology requirements quite nicely and potentially support a
company’s Enterprise Architecture. However, since this is a new
application of technology, fulfillment of requirements needs to trump
Enterprise Architecture.
Take an Exploratory and Iterative Approach to reporting
Some organizations will judge success of your project by its ability
to deliver a load of reports. If this sounds like your organization, be
realistic as to what can be delivered. Deliver a robust and reliable
dataset, some transactional reports, and one report that really helps
demonstrate the art of the possible.
Smarter organizations will judge the success of your project by its
ability to deliver analytic capabilities to the user base. The robust
and reliable dataset is still mandatory, but the ability for users to
generate their own reports will satisfy all of the “what about …?”
requirements that would blow your project budget and schedule out of the
water.
In the end… it’s the people that matter
If we believe all of the marketing hype, Big Data will help us
explore all the myriad of ways our world is constructed. But from the
perspective of a Big Data as a project, an empowered user base will
produce much more value than some canned reports.
Have you been asked to “implement big data”?
If so, what did your project look like? Let me know in the comments
down below. Stay tuned for another post on making the most of Big
Data in a PMO.
Special thanks to
Wade Walker and
Chris Sorensen for keeping me honest with this post.