A wise man once said, “to measure is to know…if you cannot measure it, you cannot improve it.” When it comes to application security, measurements are crucial to the success of your program. But determining how to best combine your measurements into metrics which show your programs value is much more important.
As a CISO or the like, you lead a team that the business absolutely depends on. Unfortunately, information security in general and application security in specific have a hard time gaining support, even if the latest Verizon Data Breach Investigation Report noted that 75% of web app attacks are financially motivated, and that application security falls “squarely under ‘the cost of doing business.’
It’s up to you to prove that your application security activities are valuable to the company – and that’s where metrics come in. Metrics are the data points for you and any other security stakeholders that help facilitate the decision making process and improve accountability and performance of your application security practices.
Application Security Metrics: Why?
Metrics offer a practical approach to helping make decisions about which parts of your program are working and which need to be fine-tuned or replaced – all with the business goals at top of mind. With an application security program in place, the only way to keep the continued support of upper management is by offering up quantifiable numbers and data tied into the goals and objectives of your organization as a whole.
The right metrics will help your organization:
- Fully understand the security risks of core business processes and applications
- Verify your compliance and that the correct security controls are in place
- Identify points of strength and points of improvement in your program
- Decide which issues need to be addressed and how to best resolve them
Determining the correct measurements for your business will take time to perfect. With so much data at your fingertips, mining through and filtering out the most important metrics will be the next challenge. But by identifying what you and your program stakeholders value highest in the business and what applications are most business-critical, you’ve already taken a big step towards proving yourself to the board.
The great news is that the information is there, sitting in your dashboards and on your and your teams minds. Now you just need to know what you’re looking for and how to weave it together into something that makes sense to yourself and the board.
Without measuring your security activities, you’re not looking at the big picture. Metrics are important because the quantify the otherwise unquantifiable practice of securing your organization’s applications. They help justify your application security program, including the people and the solutions working for it.
The Challenge of Metrics in Application Security
The rapid rate of application development, paired with the unreasonable amount of inconsistent data, does make it difficult to know if what you measure today will help you modify your security program or practices for the better.
It’s precisely because of the difficulty in measuring your activities that you need to do so.
Without being able to communicate your success in solid numbers to the business, you could find yourself hard pressed in receiving continued internal support and budget for your application security programs.
With a data-based understanding of what overarching trends and specific program configurations and activities are lowering the application risk to your organization (and which are not), you’re enabling the business and doing your duty in helping your organization overcome aversion to agility.
What Do Effective Metrics Look Like?
The best types of metrics in any type of assessment are those that are SMART – Specific, Measurable, Achievable, Realistic, and Timely. They will answer questions like: How do our numbers compare to similar organizations? Is my spending justified? Is the training I’m investing in paying off? Are we more secure this year than last?
OWASP has noted three focus areas for application security metrics: Application security process metrics, including on what level the business is meeting security standards and policies; Application security risk metrics that link vulnerability, threat and incident measurements to the businesses risk objectives; and lastly, metrics surrounding security in the SDLC – metrics showing value in the tools and resources in identifying root causes of vulnerabilities and mitigating their risk.
Many organizations have employed best practices, but best practices are enough only in organizations just starting with application security. Best practices don’t scale, are essentially impossible to enforce, and most importantly can’t be measured, especially as organizations grow.
Metrics are the only true way to show improvement to your application security program by offering real numbers and quantifiable data which they can then use to make strategic business decisions.
Application security metrics are like snowflakes: No two sets of metrics will be the same at different organizations. So while we can’t tell you exactly what metrics to use, we can send you off into the right direction.
Once you’ve realized the need for strong metrics in your application security program, it’s time to speak with the board. Only after a thorough understanding of what is expected of your application security program – and what you expect of the board – can you come to an agreement on what metrics you should be determining and the best data and tools for the job.
Where to go from here:
- Check out the OWASP CISO Guide section on Application Security Metrics.
- The Center for Internet Security offers an extensive guide to building a security metrics program including and well beyond application security.
- Carnegie Mellon’s Software Engineering Institute had developed two projects dedicated to helping develop risk-based approaches to monitoring software security:
- Ponemon’s survey, ‘Security Metrics to Manage Change’ offers a look at what other CISOs and security professionals determined to be the most important metrics in place in their respective organizations. (Page 15 of the PDF)
Latest posts by Sarah Vonnegut (see all)
- How Secure is Your Online Banking App? - February 26, 2018
- Top 5 OWASP Resources No Developer Should Be Without - January 9, 2018
- Smart Cities: Can My City be Hacked? - December 11, 2017