How to assess a candidate’s technical skills with HTB

Having worked for over 13 years in information security consulting, I’ve had my fair share of experience hiring and being involved in the interviewing process. All of those years of experience have taught me one key thing about hiring: 

Candidate assessments are an excellent tool to assess whether an individual would be a good fit on the team and able to perform the duties required for the role.

But, these assessments need to be done right. Each one should be relevant to the security role, aligned with your hiring goals, and have the right level of difficulty for the candidate. 

The power of skill assessments for technical security roles

Knowing whether someone can do the job for a technical role, like penetration testing, is essential. Resumes and typical interviews are simply not enough to determine a candidate’s practical skillset or their approach to problem-solving. 

I remember a time when I was asking candidates basic questions about Active Directory (AD), and I could just tell that they were pausing to Google the answers. We also got people with fantastic-looking resumes, but when we tried to dig into them, it became clear that they were bluffing or fluffing up their resume. 

As a result of that process, we ended up hiring someone who didn’t work out well due to lack of experience. I had been trying to push for a candidate assessment, finally after that experience, my management team actually let me do it!

We built a customized lab that modeled a client network (albeit smaller) with various vulnerabilities and misconfigurations throughout. When we reinterviewed for that job a second time, we required that everyone being considered for the role take it. The lab was designed in a way that increased difficulty as a candidate progressed deeper into the network. This helped us gauge an individual’s fit for a junior, mid-level, senior, or principal role.

It was a great way to weed out candidates, and to not waste the hiring manager’s time.

The next step, after the hiring manager interview, for them to get to the actual technical interview was to do the lab. Around 70% of people ghosted us after that! This meant that the candidate skills assessment was effectively filtering out candidates who weren’t serious about the role or who didn’t have the required technical skills. 

As a hiring manager on a busy consulting team, this saved me a lot of effort and many billable hours.

Smart strategies for candidate skills assessments

Before conducting any type of skills assessment, you should set some clear goals so you can determine whether candidates are performing well or not. This will also help you to ensure that the assessment is fair. 

Set candidate performance thresholds 

Hiring for multiple roles? That doesn’t mean that you need to set up multiple skill assessments! 

We had junior consultant, staff consultant, senior consultant, and principal consultant roles. So we set thresholds in the lab for where each individual should be able to get to based on their seniority and overall work or technical experience.

Setting these thresholds made it much easier for us to accurately assess how well someone was performing, depending on the job they applied for. It also saved us time by eliminating the need to set up multiple skills assessments simultaneously. 

If someone was applying for a principal consultant role, and they could barely finish the first part, that would be a red flag. But if they were a junior and they finished the whole thing, then we definitely needed to talk to that person, as they could be a “unicorn” candidate!

Make sure it’s relevant to the role 

While this may sound obvious, the assessment must focus on the main functions of the role. Which is why CTFs aren’t the best way to assess candidates. 

For example, suppose someone’s amazing at Active Directory (AD) and reverse engineering, but they can’t do a crypto challenge. In that case, that doesn’t really matter to me as you won’t be doing that during a pentest.

If you’re hiring a pentester that’s going to be doing 90% AD pentests, make sure you give them an AD lab.

Find HTB labs relevant to any skill using Academy X HTB 💡

HTB has a variety of labs tailored to any skill level. To find the right labs for your assessment needs:

• Select any Academy topic by difficulty level.

• Get a list of all the HTB Labs and Challenges linked to the topic.

• Choose the lab that’s right for the candidate or job role you’re hiring for. 

Let’s say you’re hiring for a junior pentester who has a good basic understanding of AD environments and Kerberos: 

1. Go to Academy X HTB Labs and click Modules. 

2. Then choose the skill you’re testing for. In our case, we were looking to test for someone who has skills with Kerberos Attacks. So I chose the Kerberos Attacks module.

3. Look through the list of all the HTB content related to Kerberos attacks & mapped by difficulty levels. This means we can filter for junior-senior candidates based on difficulty levels of the Machine. 

Keep things fair

At their core, all skills assessments should be relevant to the role and consider the level at which you’re hiring. You can’t give a one-size-fits-all lab to all your candidates, and say “you have to be able to complete this”. Otherwise, you might be hiring someone for something they either can’t do, or that’s too easy for them.

Conducting the skills assessment 

Now that you’ve got your assessment set up and have determined the thresholds each role must fulfill, you can move forward to sharing the task with candidates. 

Step 1: Assign the assessment and deadlines 

Once the initial screening interview has been conducted, you can share the technical assessment with potential candidates. For a penetration tester role, candidates were usually given 3 to 5 days to complete the assessment. 

We wanted to be respectful of people’s time, so we made the lab take only a few hours to complete in total, across those days. We’d then have them prepare a short report, we provided them with a more stripped-down template, as we weren’t asking people to complete a 40-hour assessment.

Step 2: Monitor actions & methodology with logs 

We also had logging and monitoring going on, so we could see the types of things the candidates were doing, and how much time they spent on them. It was a two-fold way of screening people, as you could see their testing methodology as well as the results. 

For example, we told them to treat it like a real client network, and asked them to perform no brute-force attacks. But some people still did brute-force, as we could see in the logs. This would be a concern as brute force attacks are unstable, and we wouldn’t want our penetration testers to use this method on client networks. 

Step 3: Have the technical interview 

Once the assessment has been completed and logs analyzed, you can invite candidates back for a technical debriefing and interview. The goal here is to determine whether they fully understood the task and if they can talk through their findings. 

The technical interview should first be a debrief. 

During our interviews, we’d ask the candidate to walk us through the lab as if they had performed a pentest on a real company. Some people smashed it, but for some it was obvious they didn’t do it themselves, because they didn’t know what they were talking about.

While the technical debrief gives you an insight into a candidate’s presenting and practical skills, it also gives you unique (and valuable) insight into their personality and penetration testing behaviors:

It showed us their temperament and how much risk they’d take during an assessment.

There was one candidate who even emailed us as if we were the client and asked if he could try a particular exploit as it might be unstable, which we really appreciated!

How to assess candidate results 

So, you’ve had several candidates complete the skills assessment, and they’ve all performed well. How do you actually go about deciding who excelled and who deserves the role? 

Assessing the task

We would assess them on three things: 

1. Their technical ability.

2. How do they approach problems (are they throwing automated tools at the problem, or are they being more careful?)

3. Their write-up (how did they write up their attack chain, was it clear, easily reproducible, did it appeal to the less technical people?

Assessing the findings

Within the report, candidates should also share some of their findings, as this provides a unique insight into how they’d share these with the client or less technical people within the company. 

We also have them write up one or two sample findings. When reviewing these findings we’d ask ourselves:

• Was it clear?

• Were reproduction steps there? 

• Was remediation advice actionable?

Assessing the technical debrief

The technical debrief provides hiring managers with plenty of priceless information about a candidate. I’d suggest considering the following:

• Could they answer the questions about the lab? 

• Could they explain something specific that they did?

• Could they walk us through the attack chain in their own words? 

That way, you could judge whether they did the assessment themselves, and their level of knowledge and experience against the requirements of the role.

Taking a 360-degree approach 

The insights candidate skill assessments provide are an essential part of the hiring process for cybersecurity roles, because you get a 360 view on how they worked technically, and how they presented their findings. 

During interviews, we would treat the assessment as though they were presenting results to a client first, before we had a technical discussion where we’d ask specific questions. This holistic approach ensured we found the right candidate for the job as often as possible.

Conduct skills assessments with HTB

The importance of skills assessments is clear and at Hack The Box, we have ready-made scenarios on our Enterprise Platform that serve as a fantastic candidate assessment tool. 

Here’s how:

By using Spaces, companies can create sub-labs within HTB Enterprise Platform and use them for candidate assessment purposes in just some simple steps:

1. Inviting the candidate as Guest User to the enterprise environment.

2. Select a scenario and assign it to the candidate.

3. Toggle “Mask Mode”, a feature that hides all the scenario information and restricts access to write-ups or other guided settings for maximum integrity.

You can then gain full visibility of a candidate’s performance and ask key questions during the technical interview. 

💡For more information, read our full blog on: 4 steps to implement a skill-based candidate assessment strategy