I keep coming to the same topic over and over — why are we still bad at detecting threats?
I’ve lamented on this a few times, either touching on general difficulties with detection, its uncertainty or highlighting the fragile detections people write. I also noted the critical role of context in threat detection, which seems to imply that the best detections are written on-site by each team, and not by the vendors in their comfy little labs …
Here, I want to continue the conversation on detection quality. Also, I want to look for some ideas that can help everybody. How do we get the mainstream companies to improve their detection quality? What does it mean to have “good” detections? How do we get to more and better detections? More curiously, how do we get to better detections that are also developed rapidly (to avoid the “good/fast/cheap — pick any two” syndrome that led to many fragile IP- or hash-based detections)?
Now, some people will say that there are NO good answers. Hence, their only answer is to have organizations outsource detection to a quality MDR or an MSSP. Sure, that would solve the problem for them (It would, right? Right?!), but it would really just shift the bigger picture problem resolution to their MSSP. Specifically, how would many MSSP teams write good detections without access to the best of the best detection authoring talent? Worse, sometimes said MSSPs will try to “mass produce” detection rules hence losing local, on-site detection context. Even more worse, sometimes you can spot an MSSP that simply relies on commercial SIEM stock detection rules. Economies of scale here may translate into missed attackers …
Also, there will be a category of people that will say that the answer is …. drumroll … machine learning. However, guess what makes detection difficult for many security scenarios? The fact that we do not have reliable data on attackers and attacker behavior. Also, guess what is required for machine learning to work well? Just the data we just noted the lack of. No way, right?! So, ML unicorn cavalry is not coming.
Well, instead of just lamenting and highlighting the challenges, what can we do?
Let’s try to decompose the good detection into elements. What do you need to write a good detection? (Note that we kinda sorta punted the definition of “a good detection” to a later post…)
- You need some knowledge of the attacker; specifically, you need elements of such knowledge that you can latch onto in your detections. For example, if you somehow know what the attacker is thinking about, you can not necessarily write rules on that …
- Naturally, knowing the attacker goal is useful, because this may point at the types of telemetry you’d use to detect them and from what parts of the environment (Database logs? Application traces? Cloud authentication logs?)
- In addition to attacker knowledge, you need some useful telemetry that you can apply that knowledge to. Note that this equally applies to both rule-based and ML -based detections, but you will likely need more telemetry if you were to develop your own ML detections.
- Knowledge that separates good from bad, attacker knowledge alone won’t cut it as there are legitimate activities that look the same in some regards. Unless you know “good” well, your detections will become “false positive” machines.
- Naturally, some way to express or declare detections, a language (like say YARA-L or Sigma) or a method at least. Further, it goes without saying that you would need some kind of tool to actually implement that method.
- Finally, it is fair to say that you would need some test data or a test environment, or at least a testing approach, to verify that what you created actually does what you wanted to do (and does not do a lot of other unneeded things).
- Anything else I missed?
Now, reading the above surely does not mean that you will write good detections, but IMHO you will have a fair shot at that. Frankly, this is much more nuanced IRL.
For example, during a recent SANS webcast we went through a bunch of rules and derived these additional lessons.
By the way, this post ended up being more of “an incomplete thought” then a solid framework, but I hope you forgive me. More to come!
Related blog posts:
*** This is a Security Bloggers Network syndicated blog from Stories by Anton Chuvakin on Medium authored by Anton Chuvakin. Read the original post at: https://medium.com/anton-on-security/how-to-make-threat-detection-better-c38f1758b842?source=rss-11065c9e943e——2