SINGAPORE — Big Data is big business. This was shared by Managing Partner of Data Collective Matt Ocko, who spoke at the Vertex Innovation Forum 2013 yesterday.
Organised by Temasek-owned Vertex Venture Holdings, the forum detailed what Big Data is and why companies would benefit from jumping on this bandwagon. Data Collective is the world’s first Big Data-only early stage investment fund.
More information has been generated in the last handful of years than in all of human history. In the next two years, the immense volume of data will double.
Managing Partner, Data Collective
Mr Ocko defined Big Data with the three “V”s: Namely it is data that is too big (Volume), moves too fast (Velocity), and does not fit in the mould of existing data in storage (Variety or Variability). “Big Data is data that is growing so exponentially fast, and changing in its underlying content so fast, and it’s so varied in its origin and structure, that traditional IT systems and analytics and other processing systems are unable to cope with it.
“By that, I mean literally everything (computing systems) that was on the planet, up until a handful of years ago, was unable to cope with Big Data at the scale and speed that we’re talking about,” said Mr Ocko.
To give it perspective, Mr Ocko said more information has been generated in the last handful of years than in all of human history, by every single human being on the planet and by every adjunct, including computers, video cameras, etc. In the next two years, the immense volume of data will double again.
As a case sample, Mr Ocko said the Obama team put together an enterprise-grade infrastructure that processed 8.5 billion requests during the last United States Presidential election. This was done only 583 days before the election and the system was built from scratch and put together on-the-fly. It had processed, on average, 300 requests per second, 24 hours a day, across the entire course of the election. In many cases, there were 10,000 requests per second. This ad-hoc infrastructure, which was implemented largely in the Amazon Web Services platform, actually performed at a higher rate of reliability in speed than most major banks’ financial transaction processing systems.
It allowed the Obama administration to simulate the personal choice of 220 million US voters, hourly or better, and run their personal choices, including on a neighbourhood by neighbourhood basis, the entire US election 3,000 times per day. That insight allowed them to field volunteer time, advertisements, direct mail and email and tweets and Facebook postings in real time — down to an individual and neighbourhood by neighbourhood level — far better than the Romney team.
The mistake that the Romney team made was, while they knew this was important, they bought a bunch of very bloated software from some pretty old school guys, mostly their political cronies, and they built something very unwieldy that did not give them the real-time insights that the Obama team had.
The Obama team knew, based on their Big Data analytic, the morning of the election, that they were going to win. They literally all took the day off.
Essentially, Big Data will allow companies to sieve through data from customer feedback through social media or traditional survey forms to readings taken from backend systems such as the temperature gauge on a manufacturing machine, and collate them into information that companies can use to gauge customer demand and how their supply chain can meet the demand. Big Data management can take these data and translate them into real time reports that companies can react to: For example, a restaurant could gauge when a heat wave would begin and stock up on ice cream and beverages to meet demand.
But more than just businesses, Managing Partner, Data Collective Zachary Bogue said Big Data can also be used in agriculture or even healthcare. To prove his point, Mr Bogue showed his wristband that counts the number of steps that he’s walked, monitors his sleeping patterns and measures the intensity of his workouts. All the data collected can be used by medical professionals to mitigate health risks. “It becomes increasingly difficult for businesses, government, and even individuals to make ill-informed choices,” said Mr Ocko.
Singapore stands to benefit from the Big Data revolution with opportunity for telecom and datacentre revenue for countries that respect Intellectual Property and rule of law, said Mr Ocko.
“The fact is security in your software is only as good as the underlying software,” he said.
“If a rival company or a national intelligence agency is watching everything on the physical computer you’re running on, you’ve compromised your security. They know everything.
Global 2000 companies need a secure computing base to serve Asian customers, and Mr Ocko said his firm’s view is that “Singapore has a huge economic opportunity — the same way it’s transformed its banking industry from a regional powerhouse to a global powerhouse — to become the analog of a banking centre for secure computing”.