Unlock the creativity of your security team with big data | Media Network | Guardian Professional
Mark Seward | Monday 22 April 2013
When we talk about data security, there's an increasingly popular adage that there are two types of companies: those that have been breached and those that don't know they've been breached.
The reason that the latter type of company exists is because attacks on enterprise infrastructure have become both more persistent and more sophisticated. In other words, unless the company has bullet-proof security processes and technology in place, it's almost inevitable that some form of attack will have penetrated the enterprise perimeter, whether the company realises it or not.
Once an attacker or piece of malware is inside the network, it can often lurk unseen among the mass of data that enterprise systems generate and trying to locate it, even if you're aware that an attack has taken place is extremely difficult.
That's why the new frontier of enterprise security is big data and statistical analysis specifically in machine data. Every interaction with a 'machine' – whether it's a website, mobile device, application server, corporate network, sensor or electronic tag, and whether it's automatically generated or a manual transaction – leaves a trail and a record.
In this new world of security, chief security officers (CSOs) and IT teams have to unlearn their over-reliance on traditional data protection technologies such as anti-virus software, firewalls and security information and event-management (SIEM) systems. Instead, the tools which might be most useful in the future include a statistics textbook, a subscription to Psychology Today, and the security professional's own brain.
Over the past few years, three major issues have come to dominate the current enterprise security landscape. First, the enterprise is now under pretty much constant attack, with common sense dictating that this increases the likelihood of a breach taking place. The attacker only has to be right once, while the IT team has to be right all the time. This being the case, relying on an attacker to alert you to their presence by tripping a specific rule that you've set in a SIEM clearly isn't a sufficient or reliable strategy.
Second, attackers understand that the pressure on IT teams has increased enormously as a result and with much of their attention and resources dedicated to protecting the perimeter, companies don't tend to do a great job at monitoring what's happening inside the network. As such, if they make it past the barricades, attackers can become very difficult to spot and can act with impunity.
Finally, and perhaps most worrying for the security industry as a whole, this constant bombardment has turned security into a reactive, administrative role, where team members are just responding to systems alerts rather than thinking more laterally about threats. Security is an exciting industry to be in, but too often, both seasoned professionals and new entrants aren't being challenged to be creative and come up with new ways to defend the enterprise – they're just doing what they are told to do by the tools they use.
So what's the solution to these problems? It's about having much greater oversight of everything that's happening inside the enterprise and developing operational intelligence. It's also about giving security professionals the right tools to quickly analyse and sift through enterprise data sets that include data generated through normal interaction with IT systems – right down to the machine data – in order to identify unusual patterns and abnormal behaviour in that data which might indicate that an attack is taking place.
It's here that statistics, psychology and old-fashioned brainpower really come into their own.
Big data analysis technologies exist that can help to identify possible anomalies, but it still requires human insight and intelligence to interpret what they might mean.
For example, there are giveaways that malware might be trying to compromise your network available in the machine data logs – these include URL strings that might be two to three times longer than normal, indicating the possible presence of command and control instructions attempting to launch a web protocol attack, or it might be the 'tell' of a network access password being entered 10 times faster than it's possible for a human to type.
In addition to these web protocol attacks, there are other examples of data logs that track human interactions with IT systems and facilities that might raise the security teams' suspicions. Why is a user repeatedly trying to access a file they don't have permission to view, or why has their ID card been used to enter the office when they're meant to be on holiday in the Bahamas?
Achieving this level of operational intelligence not only opens up new possibilities for how companies defend themselves against the myriad security threats that they face, but also re-engages the interest and creativity of the IT teams entrusted with this vital task.
The days of rules-based security engines looking for known threats are drawing to a close, as they're simply not built to handle the volume and sophistication of attacks today. To truly understand the nature of the threats they face, companies need to move beyond traditional approaches to security and delve deeper into the machine data being generated every second of every day.
Mark Seward is senior director of security and compliance at Splunk