Criteria to measure CISO success

Credit: Manage Engine Blog. click here to read it

According to Marlin Hawk’s Global Snapshot: The CISO in 2021 report, 53% of CISOs have been in their roles for two years or less, and 64% were hired from another organization. It is crucial to consider that many security specialists working for smaller businesses do the duties of a CISO without having the title. Organizations hiring a CISO for the first time find it challenging to define the role.

An organization’s information security depends on every individual’s contribution, but the responsibility for it is placed on the CISO’s shoulders. CISOs work on protecting organizations from new, unknown threats daily. However, their expertise goes beyond overseeing how security teams manage and mitigate security incidents.

Below are some top operational areas that CISOs oversee. Their involvement levels may vary depending on their organization’s size, industry, and security maturity.

• Security operations center
  • CISOs manage a tireless group of analysts constantly working on anomaly detection, threat mitigation, and incident management.
• Third-party security risk management
  • CISOs oversee various vendors and third-party providers for digital risks, PII, and privacy.
• Product security
  • CISOs ensure security through design. Along with the company’s security architecture, the CISO also significantly influences the creation of secure software building architecture that is impenetrable.
• Pen testing and red teaming
  • The CISO often has direct responsibility for penetration testing and red teaming exercises that continuously test and bolster the organization’s security posture.
• Compliance regulations
  • CISOs manage the legal ramifications of breaches, including privacy concerns, and maintain compliance with government and other cybersecurity regulations.
• Disaster recovery planning
  • A successful data breach can halt the business’s operations, damage its reputation, and incur regulatory penalties. A crucial responsibility of a CISO is disaster recovery planning, which enables business continuity and impact reduction.

In addition to short-term KPIs, such as the MTTD, MTTR, number of incidents mitigated, and breach costs, the success of a CISO should also be measured as part of your security strategy. On LinkedIn, we looked for job posts for CISO positions from well-known firms, like Cisco, Barclays, DBS, and Citi, and consolidated the top requirements listed in the job descriptions.

Here are the criteria you should weigh when assessing and measuring the long-term success of a CISO:

Cybersecurity framework development: The CISO must develop and establish a successful cybersecurity management program that aligns with the organization’s strategic vision over the years. This includes revamping organizational policies and building appropriate teams and processes.

Cybersecurity culture cultivation: The CISO must cultivate a strong cybersecurity environment where managers and end users are well-trained and aware of digital risks. This culture had to be developed via innovative employee training, workshops, benchmarking, and periodic assessments.

Stakeholder management: A CISO should be an effective liaison who communicates information and ideas about cyber risk management, the business vision, and security requirements with management, internal teams, and business partners. This also entails maintaining solid relationships with different security vendors.

Security budgeting: This criterion focuses on the budget that the CISO manages during their tenure and the workforce they directly manage. Delivering a return on investment for cybersecurity investments is a crucial goal. Success also depends on the CISO’s ability to evaluate numerous vendors and negotiate contracts with them for different business functions to align with organizational needs.

Global exposure: A CISO must manage an international team across geographies. A successful CISO should not be confined by the organization’s security needs. They should keep up to date with global trends and practices in cybersecurity.

In most cases, a breach of an organization can be the result of the previous CISO. It is thus vital to give a new CISO time and measure their success in the longer term. CISOs are participating in board meetings more frequently than ever before. However, it’s critical to understand that the CISO should be an integral component of your organization’s security culture from the beginning, not just when things become grim.

How to evaluate CISOs

Credit: Dan Lohrmann, based on his LinkedIn Post, click here to read

What sets a security leader apart? or, more broadly, how can we differentiate one tech CxO from another? Here is my ‘easy-to-use’ but ‘hard-to-excel-at’ formula for grading security and technology leadership performance.

Great organizations have great leaders. Leaders who surround themselves with top talent. Their teams work well together, using the best technology and efficient processes.

Leaders who inspire others, who motivate, who lead by example. Those who properly reward staff and who get things done the right way. Leaders who get results, with projects delivered on-time and on-budget, but without constant staff burnout to get there.

Leaders with skill, honesty and integrity. Men and women who are kind and yet relentlessly pursue excellence. People who are thrilled when their own staff members surpass them in their career.

Leaders who leave a positive legacy long after moving to another role or another organization or even retiring.

Good Questions to Ponder

But you may be thinking, is such leadership even possible? How do you really know who will stand the test of time?

No doubt, this discourse raises many questions: Who’s really good at security and/or technology leadership?

Also, inquiring minds ask introspective questions of themselves. Questions like: How am I doing in my leadership role? Am I seeing progress over time? Where should I focus to improve my organizational people/process/technology skills?

These topics typically come up when I am mentoring a mid-career chief information security officer (CISO), or talking with a group of trusted executive friends at a reception during a national or international event, or speaking to a top journalist who is looking for someone to interview for a cyberstory.

Also, sometimes a top-level elected official or career government leader is conducting a national search and wants to find the best and brightest, the age-old question comes up: Who would I recommend?

With so many tech and cyberawards being given out and with the rapid turnover in public- and private-sector leaders, are kudos on LinkedIn profiles even trustworthy? Aren’t there award-winners who seem to receive their recognition through office politics or from less-than-average work or are known to have poor ethics?

Despite confusion, one thing is absolutely clear. Study after study has shown that the overall performance of leaders such as CIOs, CISOs and CTOs is vital to organizational success. It fact, most experts say an organization’s leadership is the No. 1 predictor of future organizational success.

I really like John Maxwell’s books on leadership, and here are some of Maxwell’s inspiring quotes:

“Great leaders always seem to embody two seemingly disparate qualities. They are both highly visionary and highly practical.”

“The higher you want to climb, the more you need leadership. The greater the impact you want to make, the greater your influence needs to be.”

“If you can’t influence people, then they will not follow you. And if people won’t follow, you are not a leader. That’s the Law of Influence.”

“Anyone can steer the ship, but it takes a leader to chart the course. Leaders who are good navigators are capable of taking their people just about anywhere.”

“If you are a leader, the true measure of your success is not getting people to work. It’s not getting people to work hard. It is getting people to work hard together. That takes commitment.”

Note: While we can make similar points for most CxO roles, I will focus on the CISO role in this piece.

For good articles on the need for CISOs, see:

• Security Intelligence said this on why the role of the CISO is vital.
• CIO magazine described why you need a CSO.
• McAfee described the changing role of a CISO.
• Wall Street Journal (WSJ) on the changing reporting ties of a CISO.
• My blog on why CISOs need a mentor — and how to get one in government

How Do You Measure? Typical Responses

OK, so enough background on the vital importance of security and technology leadership. How can we evaluate CISO or chief security officer (CSO) performance?

Typical responses I hear fall into several categories

First, there are many “hard” metrics used to measure security leaders such as audit findings addressed, trouble tickets opened or closed, breaches reported, budget items managed well, staff vacancies filled (or other staff growth or decline numbers), degrees or certifications obtained, specific project success, outcome focus on top executive initiatives or even bottom-line company returns (may include legislative budgets approved for government entities).

Many CISOs spend the majority of their time putting out cyberfires or dealing with never-ending security incidents. These leaders often struggle with their wider CISO responsibilities or perhaps fail to get the needed resources or support to be effective in the long run. They never seem to gain more executive support and/or improved their security culture.

And while these (and other) metrics are important to consider and certainly help in the evaluation over time, I also see operational metrics being misused on a regular basis.

How?

Some CISOs coast on a past leader’s success in these areas for months and even years. For example, a current CISO’s lack of breaches on their watch may be the result of years of work by the former team. Conversely, I have seen a few outstanding CISOs hit by a data breach that has more to do with the inaction of their predecessor.

Closing audit findings is necessary — even essential for most leaders. But in what order? Are these CISOs leading the organization, or just responding to audit reports or regulations issued by others?

Also, are you closing the barn door after the horses have escaped? What about hot new cyberthreats that are not in those internal or external audits? Bottom line: Good metric baselines are hard to obtain and rarely are put in place for new CISOs.

Benchmarking security or technology against other best practice organizations in your industry can help in many important ways, but with CISOs turning over an average of every 17 months (this number seems too low to me for government), it is very hard to gauge the true impact of a single leader on an organization using this data alone.

In some cases, whatever numbers are used to measure performance are clear-cut and directly impact bonuses or incentive plans and are way too simplistic — such as a single number on incidents reported or resolved. Many of these “one and done” metrics are actually not in the control of the CISO. (For example: You can’t control how often you are attacked.)

On the sad side of things, I have seen CISOs fired because of a breach that was largely outside of their control, as they were outmatched by the outside organization that hacked them — or an employee clicked on a link somewhere.

Second, there are many lists of “soft” performance measures that are used to judge performance. These soft skills include items like this list from Inc.:

Good leaders have a mission and inspire others to join them.
Good leaders create strong organizations.
Good leaders have strong interpersonal skills.
Good leaders are good motivators.
Here are some other good lists of important skills and success factors to help from Habott.com and Kinhave.org.

There are hundreds of such character trait and leadership lists that are used by management to evaluate people in role-specific situations. In many circumstances security leaders are judged by the perceptions from someone higher in management (often the CIO or department director in government.) Seldom do metrics surrounding these soft-skill topics include a 360-degree feedback evaluation that is really fair, in my experience. When management does get input from others, it often includes only colleagues who provide a similar viewpoint as the evaluator.

I have also seen 360-degree performance evaluations go painfully wrong based on popularity contests or office politics, but that discussion is for another day. On the other hand, 360-degree evaluations implemented well, can certainly be an effective way to measure performance.

Lohrmann’s CISO Grading Tool: Are You a Trusted Adviser to the Enterprise?

Which brings me to a third approach that I use. My analysis uses elements of the first two methods and a few twists. It focuses on becoming a trusted adviser within your specific situation.

I generally evaluate CISO effectiveness in five core areas (Note: sometimes I add a sixth area, which I will describe at the end).

These relationship areas listed below reflect the level of trust, respect, project results, communications skills and overall competence with the different groups that CISOs generally interact with on a regular basis. It also reflects an ability to lead and inspire greatness in others.

Relationships with your internal security team. Staff under your control who report to you.
Relationships with internal organizational peers. Similar level of business and technology professionals throughout your organization. This includes internal customers who you are working with and protecting.
Relationships with your management. This includes your boss(es) and other senior executives that are higher up (your boss’s peers or above.) Of course, many view your boss as the most important evaluator. Certainly, achieving stated duties and core functions is essential. And yet, I know some hands-off tech bosses that do not provide much guidance or direction to their CISO.
Vendor relationships. How do you work effectively with security providers, managing contracts, contract staff acquisition processes, technology providers, new technology acquisition and effectiveness, etc.

External customers. This includes relationships with your wider organizational clients. People using your business partner’s products and services. For example, in Michigan government, I would meet with the Treasury Department’s and the Office of Retirement’s (and many other organizations’) customers and partners.

I realize that CISOs have differing levels of involvement and relationships in each of these five areas, so the importance of each item can vary depending on a variety of factors. (These are not equally weighted in every organization.) Also, how you grade each area (as a yes or no) can vary based on the items above under hard and soft metrics and skill sets. Nevertheless, I maintain that almost all CISOs have some responsibility in each of these five areas. See below for more on this.

How to Grade to Stand the Test of Time

The main goal is to keep grading simple in my view. This is meant to be a high-level assessment and not a detailed performance review with organizational specifics or required cybersecurity objectives.

Answer this question: Does the CISO have “good” (or even “very good” or better yet “great”) relationship with this group. I’m not talking about personal likes and dislikes, but professional rapport. Does this group respect and trust the CISO as their security adviser?

(Note: Sometimes this trust shows as fear of breaches, but they still keep following and abiding-by policies, procedures and supporting the security organization.) No doubt, this trust will be influenced by security events, incidents, data breaches, project success, audits, security competencies, communication skills, etc.

Here’s how to evaluate a security leader (or yourself) over time:

Only one (1) group says CISO is doing a very good job and trusts CISO — Won’t last long unless your boss loves and protects you. Basically the CISO is in trouble.
Any two (2) groups show trust and support and respect — Basic competence — but average CISO at best
Any three (3) groups — Doing well — Keep striving for more
Any four (4) groups — Above average CISO
All five groups trust CISO, respect you. They will follow you through cyber good times and bad — Truly exceptional CISO
One Caution: There is a wide variance in CISO roles in both the public and private sectors. Some have many staff, some have none. Some are board-level execs, others are buried in the org chart. Authority and budgets vary widely, and governance models are all over the map. Some leaders have operational security authority and accountability 24/7, while others have matrix control and others are advisory-only roles.

Also, if you are a one-person CISO team on security in your small government, your internal team may also be measured by your own technical skills to program a firewall or some other security task.

This grading approach can work for all types of CISOs, but it requires flexibility in implementation. At first, just ask, how is he/she doing with these five groups? Be sure to get a diverse range of inputs in each category. Sometimes, a single person in a group can throw off the evaluation. A special effort may need to be made to obtain balanced input.

For example, if 20 business-side execs trust the CISO, but still one business person (who doesn’t get along with anyone) doesn’t respect the CISO and opposes you, don’t count that person/situation. However, I have also seen one vocal executive critic — who has the ear of the wider group and substantial influence — sink a CISO.

This simple, high level approach provides a sound foundation for a viable working assessment, strengthened further to stand the test of time by considering the priorities among the relationship areas for the leadership candidate within her or his current employment context; unique strengths and weaknesses; longer term needs and challenges within the future employment arena; and most importantly flexibility balanced by rigor and resilience, the capacity for change, growth.

For related final thoughts, a potential sixth category to measure for security leaders and additional food-for-your-thoughts on this topic, please visit the original blog posting by Dan Lohrmann for Government Technology Magazine at:

http://www.govtech.com/blogs/lohrmann-on-cybersecurity/evaluating-technology-and-security-leaders.html

You can follow Dan Lohrmann on Twitter at: @govcso -or-

Connect on LinkedIn at: https://www.linkedin.com/in/danlohrmann/

For more Security leadership articles click here 

Keywords

measure CISO success

CISO Grading Tool