A metric ton of... fun?

In my last post I discussed why SOC metrics fail us: A Metric Ton of Problems: Why SOC Metrics Fail Us. I talked about how metrics mean different things to different SOCs, how we often overemphasize speed and volume, mix in incident response metrics with regular, old SOC metrics and generally speaking why the traditional SOC metrics are not all that useful in 2026

This time I’m talking about how »I« think we should approach SOC metrics so they’re meaningful and measure the right things.

BIG DISCLAIMER: you do you. Meaning, define what success looks like and choose whatever metrics can tell you whether you’re on the right track. I’m a big believer that for internal purposes you can measure what you want, as long as it gives you a good idea of where your SOC is and doesn’t demand impossible from the team.

This means that once a week, bi-weekly, or monthly, one of your best analysts samples alerts handled by the team to verify they were done correctly. Correctly here means a couple things:

As far as specific approaches, I know SOCs who have found good success in their Quality Assurance efforts by following ISO 2859-1:2026 - it lays out a framework for QA and it’s solid.

Or, here’s what you can do instead:

Why do we start with QA, you might ask? First of all, as AI is increasingly baked into threat detection tools, chances are your analysts will end up doing more QA than ever before. Good to start them early.

Then, and more importantly, without QA your metrics are useless. What good is MTTA / MTTR in green if alerts aren’t resolved properly or what does the false positive rate matter if alerts are categorized wrong? And so on.

Quality Assurance unlocks the value of your metrics and should always be the starting point. Now - how many SOCs start establishing solid QA fundamentals before worrying about MTT-X? In my experience, not many.

A boring, yet 100% true answer, is most of them once you've set up Quality Assurance. Suddenly the volume numbers, TP rates, MTT-X mean something.

Ok, we can’t leave it here or it will be anticlimactic. Here are 3 SOC metrics that »I« have been playing with recently and either found good success with or think those are interesting as concepts:

SIEM TCO is a big number. Often the single biggest item in your security budget or close to it. It’s difficult to discuss without a ton of context, and I found senior leaders to be often either unwilling or unable to grasp it.

And yes, we could do the usual money go up, risk go down. But what if instead we broke the big number down into multiple, individual line items?

And there are so many ways of doing that.

You could first work to understand capabilities of adversaries likely to attack your company based on your individual threat model - can start with your industry, geography, size. List their techniques, see which ones repeat the most, outline top 5-10 and make them your threat detection targets.

Calculate how much achieving each of them costs and track that.

Or, just do a flat $ per data source versus how many detections you have for it. Then maybe add context about whether those have ever fired and if you’re feeling fancy talk about the TP / FP rate.

That’s a lot of words, so let me explain with a scenario:

Your SIEM budget is $20,000 per month. You’re collecting $5,000 worth of firewall logs, you have 5 detection rules that use those logs, and you generated 2 firewall related alerts this month, both false positives. Firewalls are not your critical assets. This means that:

Is this cost effective? I’d argue it is not.

Obviously, define what cost effective is for you and track against that. I once assumed that $1000 per detection rule, $500 per alert is the limit, went over all our detections rules and had interesting findings.

Yeah, I know not all rules are meant to fire regularly, but you get me.

And if you can add your own risk context to all of that and say that, for example, to write detections around your crown jewel with an ALE of X$ you need Y$? That’s a very mature way of communicating your budget needs.

Trust me it might work.

In my LinkedIn posts I often ridicule the well established practice of preparing fake SOC workload reports with inflated numbers, to then use them as a marketing prop. I do it so much that if you close your eyes and throw a stone at my content, chances are you’ll hit a post about this very topic.

Alert volume obviously matters only after finetuning and automation. Personally, I like to calculate it per analyst because I know roughly how many alerts an analyst can handle in a day (in-house analyst ~25, MSSP analyst ~50 btw.). But what does it matter if volume goes up one month, and down on another?

Is it good if it goes down? Did we finetune our detections, or lose visibility somewhere? And if it goes up? Did we cover a detection gap, or did a random rule just fire a lot?

The volume itself is not an amazing metric, but what if we measured it in relation to quality? Not only does it make for a more complete metric, I believe this is also one of the better ways to understand your SOCs workload ceiling.

Let me explain.

There’s a breaking point where your analysts will start missing alerts entirely due to high workload. But before that point, there’s a whole range of alert volume that won’t cause analysts to miss alerts, but will impact investigation quality. That’s the danger zone you’re trying to avoid and want to be aware of.

In other words - workload goes up and quality stays intact? That’s fine. Workload keeps going up and quality starts going down? That’s where you probably want to stop.

(Small disclaimer - I mention % of alerts with quality issues as a shorthand to represent a QA number. For some SOCs it will be % of critical defects, defect rate, alerts that passed QA etc. - many ways to present the same idea.)

(Another small disclaimer - obviously quality issues are not there just due to the workload, but, for example, insufficient training. Up to SOC leads to figure that out)

Back when I was still a young Rafal I remember hearing SOC leaders talk about how many alerts their SOAR closes automatically and I thought to myself - “there is no way anyone cares about that”.

Turns out this is a metric that for some reason lands extremely well with senior stakeholders.

No idea why, but here’s what happened to me on multiple occasions in my days as a Security Architect: during a leadership meeting I’d throw out a % figure of how many alerts we’re closing automatically before they reach the SOC. Thought nothing of it. Then a couple weeks after I’d hear from people who were not part of those meetings about how our SOC (or managed SOC) closes 40% of alerts automatically. It was either other leaders or customer success people when I was with an MSSP.

???

Lessons learned I guess. If that figure stays with people, might as well start measuring it. Some ways I’ve done it:

Automation Rate = (Alerts Closed By Automation / Total Alerts) × 100

or

Automation Rate = [((Alerts Closed By Automation / Total Alerts) × 100) + ((Alerts Enriched By Automation / Total Alerts) × 100)] / 2

or

Automation Rate = (Alerts Where SOAR Automatically Kicked In / Total Alerts) x 100

Or any other way. The possibilities are endless, especially with AI increasingly doing triage and assisting in investigations. I’m sure people writing the checks to bring in that technology are interested in knowing how well it works.

And with Quality Assurance established % of alerts handled “autonomously” by AI is a good metric to track.

One important caveat though. Automation is not the same as just auto closing your all DLP alert because someone decided to enable Microsoft Purview and you got 40.000 alerts in your queue the next day. The definition of what constitutes automation is up to you, but I believe at minimum it requires more logic than:

if alert.title == "XYZ":
alert.status = "closed"

Those were the more “interesting” metrics, not the entirety of what I consider worth tracking.

If I were to give a simple summary of what comes to mind here’s how it would look like:

I don’t obsess over (or don’t see much value in tracking):

I like tracking:

I’m sure I missed a ton.

Some people say that security metrics don’t matter at all and what matters is that the business can keep achieving its objectives without third party interruptions. Some say the real metric is the friends we’ve made along the way. I believe metrics can be useful, but you need to be selective with what you track.

And most importantly, metrics don’t matter if you don’t have a solid Quality Assurance process in place.