2012 will be remembered as the year big data began its momentum, but 2013 will likely be the year it takes off.
Though all the predictions for big data in 2013 aren't sunshine and roses, the growing pains should start spawning results. As we know just managing big data is a chore in itself never mind the ever expanding possibilities of what it will turn into. Here are the challenges and benefits predicted in the industry of big data for the year to come.
Hadoop does a lot for the companies utilizing it, including assisting in the running of massive data applications on huge hardware clusters and supporting the analytics and storage needs of these companies. IDC expects Hadoop to command $812.8 million in market shares by the year 2016. Some estimates are even higher, projecting the earnings to reach $2.2 billion by the year 2018.
For those who wish to tout their proprietary platforms, the growth of Apache Hadoop might not be the best news. This open source code platform is generally more desirable than their inflexible alternatives. So Hadoop is good news for most everyone, except the proprietors who'd like to hike their prices in response to higher demand for such products. Hadoop got a foothold in 2012, and will explode in 2013.
Off the shelf products for handling big data might not go the way of the dodo bird, but the concept of in-house development is certainly gaining momentum within businesses who see the potential for big data and want to implement plans to harness that power. Many are turning to their in house IT staff to develop these systems instead of opting for one size fits all systems which don't meet their particular needs.
This is good news for anyone looking to get involved in the IT industry in the near future, but bad news for companies who've depended on their product development to cash in and generate profits. Not that there won't be a market for off the shelf big data systems, but the trend is moving toward in-house efforts.
Image via Flickr by gailjadehamilton
PCs are making less of an impact on data accumulation as more people turn to smartphones, tablets, and laptop computers to get the majority of their daily work done. What exactly does this mean for big data? Well, it means developing ways to deal with touch screens, interfaces that work with smaller screens, and other issues associated with mobile devices.
Security is also a major factor in big data as mobile devices replace desktop computers. Since a single touch on most of these devices opens the user's access to enormous databases of highly sensitive information, security with mobile devices makes traditional desktop security look like child's play. As the need for development of big data systems grows, the need for keeping that data secure grows exponentially, due in part to mobile access.
Mobile devices aren't the only security issue faced by big data. The threats are endless. Hackers, identity thieves, competitive organizations, unfriendly world governments, and many others would love to get their grubby hands on the information contained in big databases.
These threats will likely increase in 2013, unfortunately. There's an ongoing debate on who should own big data, which poses another security threat. What people willingly shared with their mortgage lender, for instance, might not be something they want to pass on to the government. Similarly, the information they pass to their doctor isn't always information they want the companies they do business with to have privy to. All these issues spark debate within companies, on government legislative floors, and in the general population.
Experts estimate that the U.S. is currently short 140,000 to 190,000 workers in the field of big data, and 2013 won't see an improvement in those numbers. Data analysts don't become experts in the field of big data overnight, so the shortages range from front line workers all the way up to senior management positions.
It's going to be increasingly harder for companies to find qualified big data analysts, and even more challenging to keep their highly specialized and trained staff. Basically, anyone who has knowledge and experience in big data can cancel their plans to hit the unemployment line in the foreseeable future. Companies, however, are going to have to learn to lure these experts and work feverishly to keep their staff satisfied and on board.
Big data is an exciting prospect, but it's not fully realized yet. The potential is mind boggling, but only with the right systems in place and the proper personnel to handle the massive data can it be used effectively. But this won't stop companies from investing in these initiatives, because the word is out that companies who aren't harnessing the potential are doomed to lag behind.
For this reason, big data is going to gain ground in the market place in 2013 and beyond. Mobile devices and other rising technologies are playing a part in this momentum. Though humans are producing data at a staggering rate, most of the data we're gathering comes from devices. Sensors, counters, cameras and other devices are gathering data 24/7, just as humans are inputting information into systems faster than most people can comprehend. The $10 million big data is projected to generate in 2013 won't likely sound like a big number at all a few years from now.
The outlook for big data in 2013 may not be perfect, but it's certainly bright. Hopefully, this will be the year that companies realize the potential for big data and start to tap into the incredible power it surely holds. But the progress is not without hindrances. Foreseeing the potential problems for the industry and developing solid means for addressing those issues will be the critical factor in our progress with big data in 2013 and in the years to come.