January 15, 2020 • The Recorded Future Team
Editor’s Note: Over the next several weeks, we’re sharing excerpts from the newly released second edition of our popular book, “The Threat Intelligence Handbook: Moving Toward a Security Intelligence Program.” Here, we’re looking at chapter two, “The Threat Intelligence Lifecycle.” To read the full chapter, download your free copy of the handbook.
On your journey to security intelligence, comprehensive, real-time intelligence must be woven tightly into your security processes, third-party risk management program, and brand protection strategy. But in order to get there, how can you develop threat intelligence that truly adds value to your organization? And how can you ensure the intelligence you deliver is actionable for teams across security functions?
To answer these questions, it’s important to view threat intelligence production as a multi-step, cyclical process — not a point-in-time task.
First, the goals of the intelligence cycle must be defined by key stakeholders. These objectives may vary widely from organization to organization depending on use cases, priorities, and risk. From there, data must be gathered from a range of sources — internal, technical, and human — to develop a complete picture of potential and actual threats. Then, this data must be processed and turned into actual intelligence that is timely, clear, and actionable for everyone — whether they’re staffing a SOC, responding to security incidents, managing vulnerabilities, analyzing third-party risk, protecting your digital brand, or making high-level security decisions. This finished intelligence output then goes back to key stakeholders, who can use it to continuously improve future intelligence cycles and hone their decision-making process.
The following excerpt from “The Threat Intelligence Handbook: Moving Toward a Security Intelligence Program” has been edited and condensed for clarity. In it, we’ll examine each of the six phases of the threat intelligence lifecycle, review sources of threat intelligence, and look at the roles of threat intelligence tools and human analysts.
The Threat Intelligence Lifecycle
Threat intelligence is built on analytic techniques honed over several decades by government and military agencies. Traditional intelligence focuses on six distinct phases that make up what is called the “intelligence cycle”: direction, collection, processing, analysis, dissemination, and feedback.
The direction phase of the lifecycle is when you set goals for the threat intelligence program. This involves understanding and articulating:
- The information assets and business processes that need to be protected
- The potential impacts of losing those assets or interrupting those processes
- The types of threat intelligence that the security organization requires to protect assets and respond to threats
- Priorities about what to protect
Once high-level intelligence needs are determined, an organization can formulate questions that channel the need for information into discrete requirements. For example, if a goal is to understand likely adversaries, one logical question would be, “Which actors on underground forums are actively soliciting data concerning our organization?”
Collection is the process of gathering information to address the most important intelligence requirements. Information gathering can occur organically through a variety of means, including:
- Pulling metadata and logs from internal networks and security devices
- Subscribing to threat data feeds from industry organizations and cybersecurity vendors
- Holding conversations and targeted interviews with knowledgeable sources
- Scanning open source news and blogs
- Scraping and harvesting websites and forums
- Infiltrating closed sources such as dark web forums
The data collected typically will be a combination of finished information, such as intelligence reports from cybersecurity experts and vendors, and raw data, like malware signatures or leaked credentials on a paste site.
Processing is the transformation of collected information into a format usable by the organization. Almost all raw data collected needs to be processed in some manner, whether by humans or machines. Different collection methods often require different means of processing. Human reports may need to be correlated and ranked, deconflicted, and checked.
An example might be extracting IP addresses from a security vendor’s report and adding them to a CSV file for importing to a security information and event management (SIEM) product. In a more technical area, processing might involve extracting indicators from an email, enriching them with other information, and then communicating with endpoint protection tools for automated blocking.
Analysis is a human process that turns processed information into intelligence that can inform decisions. Depending on the circumstances, the decisions might involve whether to investigate a potential threat, what actions to take immediately to block an attack, how to strengthen security controls, or how much investment in additional security resources is justified.
The form in which the information is presented is especially important. It is useless and wasteful to collect and process information and then deliver it in a form that can’t be understood and used by the decision maker. For example, if you want to communicate with non-technical leaders, your report must:
- Be concise (a one-page memo or a handful of slides)
- Avoid confusing and overly technical terms and jargon
- Articulate the issues in business terms (such as direct and indirect costs and impact on reputation)
- Include a recommended course of action
Some intelligence may need to be delivered in a variety of formats for different audiences, say, by a live video feed or a PowerPoint presentation. Not all intelligence needs to be digested via a formal report. Successful threat intelligence teams provide continual technical reporting to other security teams with external context around IOCs, malware, threat actors, vulnerabilities, and threat trends.
Dissemination involves getting the finished intelligence output to the places it needs to go.
Most cybersecurity organizations have at least six teams that can benefit from threat intelligence.
For each of these audiences, you need to ask:
- What threat intelligence do they need, and how can external information support their activities?
- How should the intelligence be presented to make it easily understandable and actionable for that audience?
- How often should we provide updates and other information?
- Through what media should the intelligence be disseminated?
- How should we follow up if they have questions?
We believe that it is critically important to understand your overall intelligence priorities and the requirements of the security teams that will be consuming the threat intelligence. Their needs guide all phases of the intelligence lifecycle and tell you:
- What types of data to collect
- How to process and enrich the data to turn it into useful information
- How to analyze the information and present it as actionable intelligence
- To whom each type of intelligence must be disseminated, how quickly it needs to be disseminated, and how fast to respond to questions
You need regular feedback to make sure you understand the requirements of each group, and to make adjustments as their requirements and priorities change.
Get The Threat Intelligence Handbook
There’s a lot more essential content in the full chapter of the book, including helpful diagrams, notes and tips on applying these explanations to your own threat intelligence program, and more detailed information on things like sources of threat intelligence.