In our last post we defined big data, explained why it poses many serious challenges and, most importantly, why it should matter to you. Our personal data is swimming in the big data pool whether we know it or not, whether we like it or not. And the waters will only get deeper.

With a historically unprecedented amount of data on our hands, it may be of no surprise that it has spawned a new field of academic study. Are you interested in a career in big data? There are graduate programs and PhD tracks in data science. Are you just hoping to develop basic acumen on the subject, and pick up some nomenclature? There are certificate programs and massively open online courses (MOOCs) that can get you up to speed. But as real world technological solutions evolve to manage big data, so will the courses that teach it. You can count on that.

And yet, despite all the courses and programs being offered, academia is unable to keep up with the demands for mining and intelligently collecting all that data. In the article “Data Science: The Science of Our Lives,” Claire Cain Miller explains that schools aren’t producing the volume of data savvy graduates quickly enough to meet current (and future) needs. “There will be almost half a million jobs in five years, and a shortage of up to 190,000 qualified data scientists, plus a need for 1.5 million executives and support staff who have an understanding of data.” According to Miller, the United States alone needs to grow the number of data scientists by 60% – and big data doesn’t recognize nation states or boundary lines. It’s a global problem.

In an All Things Digital article, Bill Wise, CEO of Mediaocean, sounds another cautionary note, one that reverberates particularly loudly at the moment given the recent debacle over the Reinhart-Rogoff paper. The paper, which shaped critical decisions on austerity measures worldwide, contained a simple Excel error in a very large data set. And the mistake – traced back to human error – has sparked a firestorm of debate, and is part of the reason the paper (and theory) is now somewhat discredited.

Wise wrote: “More human/data interaction means a lot more room for error […] We need to spend serious time around providing usability that guides human users in the best way to engage with the data, and automation that takes human interaction (and human error) out of the picture.”

In other words, as we learn to co-exist with more and more data, we need to stand guard against blithely and blindly accepting numbers at face value without a full vetting process – whether we become professionals in the field or aim to be conversant on the basics.

Back to the Reinhart-Rogoff paper for a moment – there has been some criticism that it was not peer reviewed. And there has also been some skepticism that a peer review process would have actually caught the Excel error (among others). But Wise is on to something, and it gets at the heart of what we do with big data: we can’t let computers and models do all the work. We can’t hang our hats on assumptions and assertions just because they sound or look good. Let’s hope that as the field of data science develops, a great deal of emphasis gets placed on accuracy and precision, inasmuch as that is possible.

Because ultimately, big data holds a promise that only a handful of visionaries would have thought possible. Wise explained in his article what this promise could look like: “It can make everyday processes – from critical analyses to mundane tasks – work smarter through data intelligence. Ultimately, all that data management translates into an economy and society that lets machines handle the minutiae as humans think through the larger picture.”

What does that mean for future or present business students? It means the “O” word – opportunity. In his article, “Big Data Gets Personal in US Cities,” Ben Hecht takes Wise’s version of the data promise one step farther: “the true promise is where the numbers and patterns from this data connect and become personal – enabling us to understand and to respond to humanity and the world in ways previously unimaginable.” We can’t forget that. And while business graduates may not be designing complex models to cull insights from large data sets, they will need to serve as modern-day data translators. They will need to collaborate with their scientist colleagues to communicate complex challenges, and deliver clear solutions that solve problems and make an impact. They are part of the big data promise, and they will play a crucial part in helping to deliver it. Does anyone out there envision a new business venture in big data, or have ideas for how it can solve a long-standing, perplexing issue? We hope you do.

Photo courtesy of Verbeeldingskr8.

Deep dive into analysis and data in the business world with Hult’s Masters in Business Analytics. To learn more, take a look at our blog Where statistical analysis and business knowledge meet: Boston’s Business Analytics Group, or get into broader business challenges with a Masters in International Business instead. Download a brochure or get in touch today to find out how Hult can help you to explore everything about the business world, the future, and yourself.