The adage “you can’t manage what you can’t measure” is true in business and equally true for information security. How else will you know if you’re doing better and how will you know if your investments are providing any return? The simple answer is you won’t. Measurement is critical to managing everything, and in our case, managing an information security program.
Information security
As important as the measurement itself is what we’re trying to measure. We measure information security, and we define information security as:
Managing risk to unauthorized disclosure, alteration, and destruction of information using administrative, physical, and technical controls.
If we assume that this is a valid definition, which it is, then information security starts with managing risk. Measure risk and you measure information security. Simple as that.
Measurements
There are numerous ways to measure most things, and information security risk is no different. For the measurement we define (S2SCORE) to be valid it must meet these three criteria:
It must be objective. Objective criteria are well-defined and easily validated, subjective criteria are reliant on perspective, interpretation, and/or opinion. The more objective measurement is, the more valid it is.
It must be consistent. A measurement must be the same every time it is used. If the criteria don’t change, nor should the measurement. Consistent application allows for the determination of changes or deltas in the state of something, in our case an information security program. When the state of our information security program changes, we can determine the impact of the change based on previous measurements.
It must be relevant. If a measurement of something isn’t comprised of relevant elements, criteria, and/or characteristics, then it’s useless. Distance is an easy measurement for most people to relate. In 1150 AD, King David I of Scotland defined the old English ynce as the breadth of a man’s thumb at the base of the nail. The measurement was objective and relevant, but it wasn’t consistent because different men have different thumb sizes. Recognizing this issue, the King decided to apply the average of three men’s thumbs, one small, one medium, and one large. Thus, the inch became a valid measurement of distance. SecurityStudio defined our information security inch. It’s important to note, just like the inch isn’t the only valid measurement unit for distance, the S2Score probably isn’t the only valid measurement unit for information security risk.
Distance is an easy measurement for most people to relate. In 1150 AD, King David I of Scotland defined the old English ynce as the breadth of a man’s thumb at the base of the nail. The measurement was objective and relevant, but it wasn’t consistent because different men have different thumb sizes. Recognizing this issue, the King decided to apply the average of three men’s thumbs, one small, one medium, and one large. Thus, the inch became a valid measurement of distance.
SecurityStudio defined our information security inch.
It’s important to note, just like the inch isn’t the only valid measurement unit for distance, the S2Score probably isn’t the only valid measurement unit for information security risk
Maturity and S2SCORE
Maturity and S2SCORE are both measurements; however, they don’t measure the same thing and one is more valid than the other.
Maturity
Arguably the most common reference for maturity is the Capability Maturity Model Integration (or “CMMI”) administered by the CMMI Institute, a subsidiary of the Information Systems Audit and Control Association (or “ISACA”). The model was originally developed by Carnegie Mellon University in 1986 as a process level improvement training and appraisal program and has since been modified to fit many different applications, including information security.
There are five levels defined in the CMMI:
• Level 1 – Initial; processes are unpredictable, poorly controlled and reactive. • Level 2 – Managed; processes are characterized for projects and are often reactive. • Level 3 – Defined; processes are characterized for the organization and are proactive. • Level 4 – Quantitatively Managed; processes are measured and controlled. • Level 5 – Optimizing; focus is on process improvement.
Maturity is a common metric, used by many within the information security industry as a measurement of information security program performance and risk. The problems with using maturity as a measurement are:
Maturity, by itself, is an indicator of information security program performance and risk, but it is not a valid measurement. There’s more to information security than maturity, especially when we consider our definition (above).
Maturity is most often determined and applied subjectively. Subjective measurements are valid ones. Objectivity is the first measurement validity requirement.
The application of maturity as a metric is often inconsistent between organizations and even within an organization from assessment (or measurement) to assessment. This violates our consistency requirement.
SecurityStudio recognizes the value in the CMMI and in using maturity as a risk indicator but does not use the CMMI as a measurement of information security program performance or risk.
Caveat
For organizations who use maturity as a measurement, SecurityStudio has vastly improved its validity. Within SecurityStudio, maturity is measured objectively and consistently, which makes maturity a valid metric for measuring program maturity; however; it still fails for relevance because of the definition between maturity and information security.
S2SCORE
The S2SCORE was built as a measurement of information security program performance and risk. The metric fits our three criteria for measurement (above) and is compatible with traditional maturity measurements
S2SCORE uses maturity as it’s intended, as an indicator of performance and risk; however, the S2SCORE also accounts for our definition of risk:
The likelihood of something bad happening and impact if it did.
Likelihood and impact are derived from threats and vulnerabilities. In this context, maturity applies more to control weakness (or vulnerabilities) as much (or more) than anything else.
Applying our measurement validity test to S2SCORE:
The measurement must be objective. The S2Score is based on binary data and it is what it is. The measurement cannot be manipulated except through deception (or lying).
The measurement must be consistent. Application of the S2Score is the same from one organization to the next allowing for comparison between organizations, and it’s applied consistently over time allowing for state changes within an organization (improvements, digressions, and external threat factors).
The measurement must be relevant. One part of the assessment (the part the users see) is metrics related to information security control weaknesses (or vulnerabilities), while the other part of the assessment (the part that users don’t see) is related to threats. The result of applying threats to weaknesses is our definition of risk and by proxy information security. The S 2Score is completely relevant to what it is we’re attempting to measure.
NOTE: S2SCORE is not the only measurement any more than an inch is the only measurement of distance. It is a valid measurement, and it’s obviously one we use and highly recommend.
Summary
As you can see, maturity and information security risk measurement are two different, but related things. The S2SCORE is a measurement of information security risk; whereas, maturity is an indicator of information security risk. SecurityStudio’s S2Org measures both but uses them in a more accurate context.
If you’re used to using maturity as an information security measurement, we designed the S2Org to help you transition to the S2SCORE. For more information about the S2Score, see: About the S2SCORE or contact SecurityStudio
SecurityStudio (or S2) is a community and mission-driven information security solutions company dedicated to simplifying information security management and compliance. We help people and organizations in all industries (public and private) master information security fundamentals by providing practical tools on our best-in-class SaaS platform and through our trusted service partners.
The S2 platform is the premier risk and digital safety assessment tool in the world. Driven through our easy to use interface, information security risks can be assessed and managed for individuals (consumers and employees/personnel), the organizations they work for (public and private sector), and their vendors. With more than 3,000 assessments completed, our platform has been proven to be successful in simplifying and improving information security for hundreds of thousands of people.
Our tools: • S2SCORE – our quantitative scoring metric, plotted on a scale between 300-850. • S2Org – our organizational information security risk assessment tool. • S2Vendor – our third-party information security risk assessment tool. • S2Team – our team/personnel information security risk assessment tool. • S2Me – our personal information security risk assessment tool.
As an Information Security (“Cybersecurity”) consultant Risk Assessments are essential; providing a quick and efficient method for me to get a view of what security measures your organization has adopted well, and what it has not.
For some of my customers, the Risk Assessment is the first step to building an effective Information Security or Risk Management program from the ground up – for others, it’s a way to validate or identify improvements to the program they already have in place. Regardless of your organization’s maturity or size, performing a Risk Assessment is extremely valuable – and you don’t have to hire an Information Security consultant to do one. This, with the insufficient number of InfoSec professionals available in the market today, is essential!
In case you’re new to Cybersecurity Risk Assessments, I’ll cover the basics first: why these cyber risk assessments are important, and how they can help you. Then I’ll review some of the free Cybersecurity Risk Assessment tools that are available out there today.
Why Cyber Risk Assessments are Important
We live in a connected world, and threats to your organization’s Information Resources and technology are imminent. Attackers have built a $1.5 trillion-dollar business identifying internet-accessible vulnerabilities and if you don’t actively secure your technology, they will find you and take advantage of your systems. This is no longer a question, but the reality – if you have an organization with systems connected to the internet, you MUST actively work to secure them, and that’s what “cybersecurity” is all about.
Cybersecurity is essentially managing risks to your organization’s Information Resources; whether these consist of protected patient data, sensitive banking transactions, or the network infrastructure that allows your personnel to keep working effectively.
Peter Drucker has said “You can’t manage what you don’t measure.” It’s largely true, but there are some that would object. Instead, I prefer this quote from Evan Francen, CEO of FRSecure, “You can’t manage what you don’t know you have.” And that includes risk – you need to know what your risks are before you can figure out what to do about them.
Every piece of technology introduces some degree of risk, especially if it is connected to the internet. Conducting a Risk Assessment allows you to identify the threat sources to these technologies and the likelihood and impact to your organization associated with these threats. As such, risk is commonly calculated like so:
Likelihood X Impact = Risk
That seems simple enough, but how do you assign a value to likelihood and impact to make that calculation? How do you efficiently identify all the thousands of potential threats to the technology within your organization? And how do you use that information to actually reduce the risks your organization faces? That’s where a Risk Assessment is invaluable.
So, if you want to improve the security of the technology and data within your organization, you need to know what the risks are to your technology and data. And the best way to identify risk, is by conducting a Risk Assessment.
How a Cybersecurity Risk Assessment Tool Can Help You and Your Organization
Besides knowing what your risks are, you also need to understand the severity of the risks posed to your Information Resources, and whether they are high or low risk. Trust me, presenting a large list of “risks” to your CEO to “do something about” isn’t going to go over well. You need to be smart about it, and a Risk Assessment can help you do just that. Most Risk Assessments will not only give you some sense of priority (high risk vs low risk), but also give you an idea of what you can do to address those risks.
If you’re an IT Manager (of managing the information technology risk assessment tools) you may even already have a wish list of security improvements you’d like to implement within your organization. And if you are struggling to get approval for these projects, a Risk Assessment may make the difference.
The Risk Assessment will help you to communicate which of those security improvements will make the most difference to the security of the organization and help you explain to the CEO what they will get out of the additional money and resources you’re asking them to spend on the problem.
A Cyber Risk Assessment helps to break down the risks to your organization’s Information Resources in a way that they can be prioritized, are actionable, and that executives can understand.
Cybersecurity Risk Assessment Tools
Above I touched on why risk assessments are important and how they can help your organization. To break it down, a good risk assessment will help you do the following:
Identify threats to your organization’s Information Resources
Objectively evaluate the level of risk posed by these threats
Provide an action plan for effective risk reduction
Provide cybersecurity risk assessment tools to aid in communicating the organization’s risks to executive management
So, let’s take a look at some of the free Cybersecurity Risk Assessment Tools out there that can help you achieve these things.
Security Risk Assessment Tool (SRA Tool)
The SRA Tool is very popular because it is provided by the U.S ONC in collaboration with the HHS Office for Civil Rights (OCR) to help healthcare providers conduct a security risk assessment as required by the HIPAA Security Rule. OCR does not require that you use their tool, nor do they ensure that use of their tool will make you compliant with HIPAA, but it’s a free tool, and a good starting point. So, let’s dive in.
List the location(s) you’re assessing and any vendors and assets to include (optional)
Complete the assessment (required)
Complete the multiple-choice questions
Select the vulnerabilities that apply to your organization
Assess the threats associated with each vulnerability
View/output your report (PDF)
Firstly, about the locations, information assets listing, and vendor listing (3). These are entirely optional in the tool; however, you do need to know and track your assets and vendors (BAAs) to be compliant with HIPAA (164.310(d)).
Overall, the tool is very easy to use for someone already familiar with risk management practices and processes. The assessment is divided into seven sections, and each section is split into three parts: multiple choice, vulnerability selection, and a vulnerability risk assessment. Most of the multiple-choice questions are fairly easy to understand, your organization is either following the requirement, or not. There are some questions that are more complex, however; with multiple answers that require deeper reading, and this is where the on-screen tips provided in the tool are helpful.
secsecur
The multiple-choice questions are used to determine which vulnerabilities may apply to your organization; but since the questions alone may be insufficient to determine whether a vulnerability applies to your organization, you are required to choose those that apply. I really wish these selections defaulted to “selected” rather than not, as I believe this would be less confusing.
Once you select your vulnerabilities, then you are asked to assess the likelihood and impact of each of the threats associated with the selected vulnerabilities. An inexperienced person may be tempted to click “Next” past the vulnerability selection; however, this is the most valuable aspect of the reports produced at the end, so it really must be completed in full for any vulnerability that applies. The challenge with the risk assessment, if you’ve never done one, is accurately determining whether your organization has a Low, Moderate, or High likelihood for experiencing the threat, and the level of impact (Low, Moderate, High) if it did. At this point you may need to reach out to an experienced risk management consultant.
The reports shown at the end only reflect the results of the risk assessments completed for each of the applicable vulnerabilities. For example, in my test I completed the multiple-choice questions but only selected a couple of vulnerabilities in each section to do the full risk assessment on. And although all of my questions are listed in the full risk assessment report, only the vulnerabilities I did the assessment on are included in the risk chart at the top.
Overall, while easy to use, I feel that the SRA Tool has two problems:
It doesn’t provide enough guidance for organizations new to performing risk assessments in determining the likelihood and potential impact of threats
It doesn’t allow the user to update or change any of their responses once a section or the full assessment is complete. Meaning that if you want to change any of your responses later on, you’ll have to start at the beginning.
TraceSRA
TraceInsight has developed an online-based version of the SRA Tool. Their revision to the controls provides you with an easy user interface and some nice-looking reports. While their tool is setup in the same way as the SRA Tool, I prefer how they divided the multiple-choice questions and the threat assessment phases.
I also appreciate that when selecting the identified threats, they’ve used the statement “This threat is applicable to my organization”, making it much clearer what you’re doing in this phase. The same challenge exists; however, in that you probably need some prior experience with risk management to confidently assign the correct values to the likelihood and impact for each threat identified.
Like the SRA Tool, you’re able to view your results on-screen or export a PDF.
Compared with the SRA Tool, TraceSRA is very similar with a slightly nicer interface for identifying the risks and performing the cyber risk assessment. One advantage is that you’re able to update your answers even after the assessment is complete; which is extremely handy for any risk manager. Since the tool is cloud-based, you do sacrifice the privacy that the format of the locally-installed SRA Tool provides, but since TraceSRA does not provide an interface for listing your locations, assets, and vendors, this information isn’t terribly sensitive.
From nist.gov, “The PRAM is a tool that applies the risk model from NISTIR 8062 and helps organizations analyze, assess, and prioritize privacy risks to determine how to respond and select appropriate solutions. The PRAM can help drive collaboration and communication between various components of an organization, including privacy, cybersecurity, business, and IT personnel.”
It helps organizations meet obligations of Circular A-130 and other relevant policies.
The NIST PRAM tool is a combination of documentation and spreadsheets (XML format) designed to help organize and direct a cyber risk assessment to your organization based on NISTIR 8062. This tool is probably the most time-intensive of the tools I’m reviewing, but also allows you to thoroughly assess each aspect of your organization. As such, I’ll just provide a high-level description of each of the parts included in the tool.
Worksheet 1: Framing Business Objectives and Organizational Privacy Governance
Describes two tasks for you to complete, used to describe and define your organization and its privacy needs.
Worksheet 2: Assessing System Design; Supporting Data Map
Provides instructions and a spreadsheet for you to determine the threats to your organization’s data actions that should be assessed. This is done by defining the organization’s privacy capabilities and related systems, products, and services; and determining other factors for consideration.
Worksheet 3: Prioritizing Risk
Builds on the work done in Worksheet 2 to assess the risk to your organization by determining the likelihood and impact of the threat to the organization. Impact is split into Noncompliance, direct business, reputational, and cultural, and other potential costs. Values assigned to the likelihood and impact are used to calculate the risk to each data action. Risks are then prioritized based on those posing the greatest risk to the organization.
Worksheet 4: Selecting Controls
Building on the work done in Worksheets 2 and 3, you’ll identify improvements to the highest-risk data actions to create your remediation action plan, and determine the residual risk remaining.
Catalog of Problematic Data Actions and Problems
Provides a list of common problematic data actions for use in your risk assessment.
FSSCC’s Automated version of the FFIEC Cybersecurity Assessment Tool
About FFIEC CAT: “In light of the increasing volume and sophistication of cyber threats, the Federal Financial Institutions Examination Council (FFIEC) developed the Cybersecurity Assessment Tool (Assessment) to help institutions identify their risks and determine their cybersecurity preparedness. The Assessment provides a repeatable and measurable process for financial institutions to measure their cybersecurity preparedness over time.”
On its own, the FFIEC Assessment is very difficult to use as they don’t provide a questionnaire or any fillable anything to complete and use to generate reports; luckily, the Financial Services Sector Coordinating Council (FSSCC) has created an automated spreadsheet with these features to help you out. However, you may still want to visit the FFIEC website to reference their user guide and other useful information and tools such as the mapping to NIST (National Institute of Standards) CSF.
If you’re in the financial sector, this is the assessment tool you should start with. The assessment consists of two parts: determining your organization’s “Inherent Risk Profile” and assessing the maturity of your organization’s security program.
To determine your Inherent Risk Profile, you’ll select a risk level for a variety of criteria across five domains:
Cyber Risk Management & Oversight
Threat Intelligence & Collaboration
Cyber security Controls
External Dependency Management
Cyber Incident Management and Resilience
I found selecting the risk level to be very easy in this tool, as they’ve provided clear and objective categories for each of the selections available.
From this information, the tool will recommend a maturity level for your organization, it may consist of several levels, such as in my example:
Next, you’ll complete the assessment to determine your organization’s current maturity level in each of the five domains. The assessment consists of nearly 500 statements which you must determine whether your organization complies with, does not comply with, or does not apply to your organization. The scale and complexity of the assessment may prove to be an obstacle for some people and organizations; in which case you may wish to hire an audit or consulting organization to help you interpret and complete this part of the assessment.
Once the cyber risk assessment is complete, there are several useful charts to view your results in.
The Assessment Factor charts allow you to select a desired target for each domain, here you have the option to set it to whatever target you like, but the tool will remind you of it’s recommendation based on your completion of the Inherent Risk Profile.
The level you select is used to place the red-dotted line on the charts for each domain, showing you how your current maturity level compares. In my example, there’s still a lot of work to do!
The Component charts break each domain down by component instead. The red-dotted line will be placed in the same location that you previously selected.
Overall, if you’re a financial services organization, this is probably the best self-assessment for you. Due to the length and complexity of the assessment; however, you’ll probably need a risk management professional to complete it effectively.
About: “The CRR is a no-cost, voluntary, non-technical assessment to evaluate an organization’s operational resilience and cybersecurity practices… The assessment is designed to measure existing organizational resilience as well as provide a gap analysis for improvement based on recognized best practices.”
The Cyber Resilience Review (CRR) is derived from the CERT Resilience Management Model (CERT-RMM), developed by Carnegie Mellon University’s Software Engineering Institute. The CRR is targeted at service organizations; that is, organizations which utilize their assets (people, information, technology, and facilities) to provide specific critical services or products.
The CRR Self-Assessment consists of a 41-page questionnaire (fillable PDF), which is then used to generate a detailed report of about 150 pages (PDF). The questions are divided into ten (10) domains:
Asset Management
Controls Management
Configuration and Change Management
Vulnerability Management
Incident Management
Service Continuity Management
Risk Management
External Dependencies Management
Training and Awareness
Situational Awareness
Each domain consists of a collection of questions related to required activities, followed by maturity ranking. To achieve a maturity level greater than MIL1, the organization must be performing all listed practices and all maturity level requirements for that level and any level below it.
Here’s a brief description of the Maturity Indication Levels (MIL):
MIL0 – Incomplete: none or some of the requirements are followed in practice
MIL1 – Performed: all of the requirements are followed in practice
MIL2 – Planned: practice and policies are documented and supported by stakeholders
MIL3 – Managed: practices are adequately staffed and funded, overseen by management, and periodically reviewed for risk
MIL4 – Measured: practices are periodically evaluated and monitored for effectiveness
MIL5 – Defined: practices are consistently defined and practiced across all organization units
To complete the assessment, DHS (U.S. Department of Homeland Security) recommends that you involve multiple people from across your organization. These members should represent a variety of different functional areas, including: business, operations, security, technology, and maintenance.
The assessment is designed to be completed in a single day; but the PDF will allow you to save your progress, so it certainly doesn’t have to be.
Overall, I really like how this assessment is laid out. The questions are simple to answer (yes/incomplete/no), and each question offers tooltips to help clarify the question and clearly lay out the requirements for a “yes” answer – so there’s not much left to interpretation. With a general understanding of how the maturity levels work I think most organizations should be able to complete this assessment and benefit from the reports with little-to-no outside support.
Questionnaire:
Tooltips:
Also, the report and visuals generated from the assessment are overall clear and effective. They’ve used an intuitive color scheme that very quickly gives you a good idea of how you scored and what areas your organization needs to work on.
The CRR report is mapped to NIST CSF and provides a gap analysis chart for NIST Cyber Security Framework (NIST CSF) as well.
So, if you have a service organization looking to increase your cybersecurity and improve your organization’s resilience, I would definitely recommend checking out this assessment.
About: Developed by SecurityStudio, the S2 Org assessment is a fit for any organization, regardless of size or industry. The cyber risk assessment was originally developed by FRSecure’s founder, Evan Francen, and used for FRSecure’s S2SCORE assessments for 10 years. SecurityStudio has now branched off to develop products that any organization can use to improve the information security of their organization.
The S2SCORE assessment is designed to assess the cyber risk to all aspects of Information Security within your organization. The S2SCORE score is based on a scale of 300-850 (modeled after the credit score), with 300 being rated as Very Poor (High Risk) and 850 as Excellent (Low Risk). The assessment is divided into four phases: Administrative, Physical, Internal Technical, and External Technical. While primarily question-based, the assessment also gives you the option to import Internal and External vulnerability scan results for a more accurate picture of your current security profile.
Each phase can be completed and scored independently of the others, enabling you to assess all or a subset of controls to your organization. Each phase is additionally broken up into control groups, allowing you to easily see where your organization is strong or weak at a glance.
For each control statement, you simply select True, False, or N/A. The developers have even built in a quick switch to mark all questions relevant to the group with the same answer in a simple click; making the 600+ statements surprisingly quick to get through. Additionally, you can create accounts for multiple team members to work on the assessment together. When any person updates the assessment, their name and a date stamp will be recorded on screen.
You can really tell that the team at SecurityStudio has put a lot of effort into making the assessment easy to navigate, by adding features such as filters for incomplete questions, and multiple methods of navigating this otherwise very in-depth assessment.
Once the assessment is complete, you’re provided with a variety of report options, downloadable in PDF format. Additionally, if you continue to update your responses over time, you can measure your organization’s progress over time on the Organization Dashboard.
In addition to the basic reports, the team at SecurityStudio is working to make additional reports available with mapping to other common standards such as HIPAA, NIST CSF, ISO 27000-1, and others, for a fee.
S2Org also includes the ability to add multiple physical locations or sub-entities to your organization, and the ability to manage the risks identified from the assessment in an interactive Security Roadmap.
The Security Roadmap is one of the features I’m the most excited about. As you complete the assessment the tool automatically creates a remediation roadmap customized to your organization. As a free member you are limited to view and export the roadmap, while the pro membership will allow you to actively update and modify the Roadmap within the tool. Currently, you can customize your Roadmap by defining priority levels, target dates, tags, and work effort for each recommendation; additionally, you can update your progress and make notes relevant to your remediation activities. You can effectively use this tool as a risk register or action plan for your organization, updating it as each task is completed.
Despite the length of the cyber risk assessment overall, the S2 Org is still my favorite tool to use. The simplified questions, ease of use interface, variety of reports, and customized roadmap really make this tool a great way to manage your security program.
We are committed to providing free resources to help keep you, your business or organization, safe. We protect your information and never give it out to vendors.
Please also follow us on Linkedin to catch our latest updates. If you found this information helpful, please share with your community.
