We are in the midst of an analytic revolution, and "Big Data" has become the de facto term used to describe this sea change. According to a study by management consulting firm McKinsey & Company, big data will affect companies and global economies to such a degree that it "…will become a key basis of competition, underpinning new waves of productivity growth, innovation and consumer surplus." What is the impact of Big Data on HR organizations? Is this a new fad, or a profound change? To answer these questions, we need to first answer: What is Big Data?
What is Big Data?
Ironically, Big Data approaches do not necessarily imply large volumes of data. Big Data approaches look to solve, or answer, business questions that could not be solved previously because of technology limitations or prohibitive costs. These generally will fall into one or more of the following four categories:
1. Data complexity – combining multiple types and sources of data that may be changing rapidly
2. Data volumes – large data sets
3. Calculation complexity – mathematical, or computational, complexity applied across data sets
4. Unstructured data – often social data, or other human input like engagement responses or exit interviews
Why Big Data?
Traditionally, analytics have had a data warehouse as the core underpinning storage mechanism for the data that powers reports, dashboards and analysis. The structure of these warehouses, and limits on technology, meant some questions could be answered in seconds, some took hours to calculate, and others couldn’t be answered at all. During the creation of the data warehouse these trade-offs were built-in, and if the wrong assumption was used it was expensive to go back and retrofit changes. Or worse, assumptions would simply change as the business environment changed. For example, if the organizational structure of the organization changed, it would often take three months or more to update the warehouse so you could determine turnover rates for the new organizations. As data volumes increased, this challenge become greater as even the fast response questions were taking too long to calculate. Fortunately, technology vendors innovated a new approach that included combining new ways of storing data with advances in computer hardware that removed this need to bake-in the structure of the data. The result was a solution much more flexible to data that was changing or incomplete, and allowed any question to be answered with consistent sub-second response time. Even those questions that previously couldn’t be answered at all. With these innovations of "columnar" storage and "in-memory" technology Big Data was born.
Further advances have seen volumes of data increase as new sources of information external to a company were brought in - such as social data. Additionally, these new approaches allowed more complex calculations, which take advantage of statistical approaches, and can find results not previously possible. Some of these may be simpler like calculating span of control for every manager in a company, or more powerful such as predictive techniques that can find trends hidden in data such as what do top talent employees have in common. This last example has great potential to help inform how an organization can increase the quality of their talent.
The power for HR leaders in Big Data is it allows for greater speed of implementation, flexibility in a changing business environment, and the ability to answer questions that previously couldn’t be solved. And it is this last point that really determines whether Big Data will revolutionize HR. If the questions we ask, and the answers we determine lead to new ways to recruit, attract and retain the best and brightest, or find ways to improve engagement and productivity, then Big Data has the opportunity to revolutionize HR. This is something all of us will have to determine by exploring those questions, and it is something I am looking forward to exploring with other experts at the 9th HR Metrics Summit at the end of February in Orlando. I hope to see you there.
Written by Dave Weisbeck, Chief Strategy Officer for Visier
Dave Weisbeck leads the overall solutions success and strategy at Visier. Dave is a seasoned software executive who has experience ranging from building development teams to growing multi-billion dollar businesses as a General Manager. Prior to joining Visier, Dave spent more than 15 years in the information management and analytics industry, which included time-spent helping to build Crystal Decisions and Business Objects products and product strategy. Most recently Dave was the Senior Vice President and General Manager responsible for Business Intelligence, Enterprise Information Management and Data Warehousing at SAP.
Dave has recently been appointed to the HR.com Advisory Board.