When I imagine the region’s CISOs staring unblinkingly at their ceiling as they try to get to sleep at night, I imagine the questions burning in their brains, robbing them of rest. Where are our vulnerabilities? Are we making progress on risk? What is the ROI of our cybersecurity suite? These and other questions are not answered by sleepless rumination, no matter how hard we try. No, they are answered through measurement. But therein lies the challenge. How do we extract the right metrics from the fog of complexity that is the modern IT suite? How do we supply tactical thinkers with accurate information at the right time?
“On-premises data centers and cloud platforms vary greatly in risk profile.”
Hadi Jaafarawi, Managing Director, Middle East at Qualys
Setting the table
We must begin by focusing on outcomes. If we ingest too many telemetry feeds, we over-encumber the SOC and defeat our purpose before we even begin. If you define what you want to achieve, then it becomes easier to choose those metrics that illustrate delivery or shortfall. After you have determined what good and bad scores look like for your metric, think in terms of a narrative that will resonate with the relevant audience — end users, senior stakeholders, and GRC (governance, risk, and compliance) teams. While telling your story, be sure to include the limitations of current measurement and how to improve on this, once buy-in from your audience is achieved, and more budget is allocated. Also be ready to answer questions such as “Why now?” and “Why not sooner?”. Let your narrative paint a picture of an ongoing process that has room to mature.
When you start out, it will be necessary to baseline everything to summarize your organization’s cyber-risk posture, from the perimeter (however you choose to define it) to the personal endpoints of remote-working employees. From this, you can gauge the best preventative, detective, and corrective controls to introduce. As part of baselining, perform an audit of data-retention standards and measure the organization’s adherence to them.
Bread and butter
After having established your baselines, you are ready to select Bread and Butter (B&B) metrics. For the purposes of storytelling, B&B metrics should be split into two groups — Below the Line (BTL) for tactical and operations teams and Above the Line (ATL) for leadership audiences. For the former, metrics must provide insights into network environments and arm the SOC with the right information at the right time to allow rapid and effective decisions that can mitigate risk.
The baseline for BTL metrics will have established a picture of hardware distribution and associated risks. For example, mobile devices like laptops are viewed with more suspicion than desktops that are permanently stationed within controlled premises. By distinguishing between assets like this, we can identify vulnerabilities and target mitigations more intelligently. We can also establish better controls. Reviews of software will also be necessary.
Yes, examine servers and cloud-hosting environments, but do not neglect the errant personal device used by a WFH employee. It may be riddled with unsanctioned software and may be missing mandated applications and critical updates. Also missing may be the software agents that gather the very information needed by security teams. The percentage of assets that are missing such agents is another critical BTL metric because not only are these assets at risk but so are any assets with which they share dependencies.
When considering ATL metrics, we need to provide leadership teams with the right information to craft value-adding risk-management strategies. These metrics center on strategy and the building of a more resilient security posture, so they often involve organization-wide measurement and risk scoring. Cumulative risk scoring can be helpful in giving a bird’s-eye view of vulnerabilities and their associated risks. A risk heatmap allows leaders to see at a glance where they need to direct resources. Sometimes that can be done through instructing teams to refocus their efforts. In other instances, investment in a new tool or platform may be required. Perhaps the problem can even be addressed through a new policy or awareness training for end users. Care should be taken, however, not to be misled by high-level metrics when vulnerabilities in low-risk assets may obfuscate the organization’s averages.
It may be useful to separate ATL risk metrics into device types such as clients, servers, network infrastructure, and so on, together with device roles. Metrics such as these present decision makers with a view of elevated risk areas, which allows them to target cybersecurity investments. Further segmentation is advised between ATL metrics that cover internal networks and those that measure risk from external domains. As IT environments become more of a mixture of cloud and on-premises, understanding these risks separately helps leaders make better decisions about the source of risk and the extent to which they can control their exposure.
On-premises data centers and cloud platforms vary greatly in risk profile. On-premises facilities are physically more secure but challenges such as scaling, resources, and hardware maintenance remain. Cloud platforms solve these problems and present the added value of managed services and predictable costs. But when it comes to security, the shared responsibility model is a work in progress and the distribution of obligations between provider and customer represents many risks, including those associated with potential misconfigurations. It is important to get to grips with these details and balance the architecture of the environment with the organization’s risk appetite and business requirements.
Measure who you are
Organization-wide risk scoring may be critical, but that does not mean that we abandon more granular risk assessment. Segmentation can be done by geography, subsidiary, or business unit. Metrics that are tailored to these categories will lead to more actionable information. Again, stratification allows more efficient allocation of resources as teams deploy security measures intelligently to address unique challenges.
Measurement must never stop, because threats never stop.
