With so much data at hand, should cyber defenses be more effective?

These days there’s no shortage of threat data available to leverage in the fight against cybercriminals. With so much of it to call on, the uninitiated might be tempted to ask why security teams aren’t building better cyber defenses to ward off attacks. But, there’s a gaping divide between much of the threat ‘data’ that security analysts are given and the actionable threat ‘intelligence’ they need to make properly informed decisions and responses.

The difference between data and intelligence

What is often loosely referred to as ‘threat intelligence’ is in fact a vast array of information from disparate sources, including threat reports, email messages, vendor advisories, blogs, forums, articles, pdfs, and documents. Much of it is text-based, which is a major problem as it doesn’t have a predefined format, making it far more challenging to process and operationalize. While this data is essential for comprehensive cybersecurity, analysis is made even harder as it also lacks context and relevance. Trying to get this type of unstructured data into a usable, meaningful format eats into an analyst’s time, day in and day out. It is an arduous, lengthy, manual task.

Swamped by the sheer volume of data, security teams get trapped in a never-ending struggle to sift through it all, eliminate false positives, and confidently identify real indicators of compromise (IOC). Not only is this approach time-consuming and inefficient, its value is diminished by errors.

To reclaim precious time, analysts need a better and faster way of assimilating and structuring data for further investigation. This is where automated threat intelligence processing can be at its most effective, cutting out the onerous task of mass data crunching.

Starting with raw data

The first stage is to bring the raw data, whatever its type, into a threat intelligence platform (TIP) to restructure it into a standard format that analysts can query easily, plus integrate with existing security tools. STIX is an increasingly popular framework used to ensure that unstructured information can be consumed, as well as amalgamated with internal log files and other structured data. It enables organizations to share threat intelligence in a consistent, machine-readable manner. Each threat can be assigned multiple attributes covering: motivations, capabilities, and responses. Using the framework, a TIP automatically categorizes each piece of incoming threat information with its relevant attributes, eliminating hours of manual work and potential mistakes.

Typically, TIPs come with built-in agnostic conversion to a wide range of other formats. This also ensures each piece of threat information can be made available in the specific format required by every security tool and technology that depends on it.

Creating actionable intelligence

With all threat data now standardized and residing in a central platform, the next stage is to rationalize the information by removing duplication and assess the criticality of the different IoCs. Along with internal threat feeds, TIPs usually rely on trusted external search engine services such as Shodan, VirusTotal, and WHOIS, to examine and enrich hash values, IP addresses, domain names, network artifacts, tools, and tactics, techniques and procedures (TTPs), and host artifacts used by attackers.

As part of the normalization process, duplicate and irrelevant threat indicators are removed. By automatically correlating vast tracts of data, a TIP can uncover complex attacks or suspicious behaviors that may have gone unnoticed if analysts were examining individual data points in isolation. Instead, the data can be distilled to show which IOCs pose the greatest potential risk and given a confidence score to help assessment.

This is the point where security teams come into their own. They review the confidence score of the IOC, and based on their evaluation, the TIP can be set up to triage several response actions, such as blocking the IOC on internally-deployed security tools and adding it to the SIEM watchlist. With actionable intelligence at their fingertips, analysts can apply their skills and knowledge to make better informed, and faster decisions. No longer consigned to the threat processing treadmill, they can carry out advanced threat investigations to prioritize response and remediation, measurably improving the overall security posture of their organization.

Getting off the data processing treadmill

The threat intelligence lifecycle is an ongoing process that involves multiple stages to arrive at actionable intelligence, from collection, normalization, correlation, enrichment, analysis, to dissemination. However, meaningful threat intelligence with relevant context is a world away from the raw, noisy data often heaped on weary security analysts. The overwhelming volume and complexity of data generated by various security tools and sources can take its toll on even the most skilled professionals.

Organisations increasingly realise the need to liberate their security analysts from the burden of manual threat data processing. A threat intelligence platform offers an automated way of providing clear, context-rich information which can be acted on with confidence. These modern solutions enable security teams to put their energy and expertise into focusing on proactive threat detection and swift response, which is imperative for safeguarding an organisation against cyberattacks and helping to build stronger defences for the future.

We've featured the best  business VPN.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro

source https://www.techradar.com/pro/with-so-much-data-at-hand-should-cyber-defenses-be-more-effective