Creating review guides for sift questions

Skip ahead to:

1. Best practices for review guides
2. When to add a review guide
3. Walkthrough an example review guide

While you’re thinking up some insightful sift questions - have a think about what a great answer might look like.

Review guides are intended to help your reviewers score answers to sift question and are displayed alongside candidates answers.  

People who are not generally involved in the hiring process may not know what a good or bad answer to a question looks like. Review guides make it easier for you to convey what you're looking for, so it's easier for reviewers to jump right into the scoring process. If you’ve never worked in Marketing, for example, you might find it difficult to know what a good answer to a question about Marketing looks like. A good review guide will briefly highlight things a non-expert should look out for.

Best practices for review guides

•  Highlight key traits of a good/ bad answer to a question. Your reviewers might not have experience in the same role that you’re hiring for.
  
  
• Remind reviewers that you’re looking for content, not style. Candidates might not excel at written communication for a whole host of reasons. Unless you’re hiring for an Editor or for a role where writing is critical, explicitly mention in your review guide that spelling, grammar, form etc. don’t matter or be specific on what does matter.
  
  
• Be brief and to the point. Your reviewers will be skimming the guide as they review answers, and over-complicating the guide will make their lives difficult.
  
  
• Be realistic about the constraints of a 250 word answer. Candidates only have so much space to impress you - and a good review guide acknowledges this by being realistic about what differentiates a 5-star from a 4-star answer.
  
  
• Structure it on a 5 point scale. i.e. mentions what would constitute a 1 star, 3 star and 5 star answer. This is so it's aligned with how reviewers will score candidates on the platform.

When to add a review guide

We recommend adding a review guide as you're creating questions. This is because you'll have all the relevant information to hand, and an idea of the rationale behind questions.

However -  we know you're pushed to advertise roles as soon as possible and you may not have time to create review guides at this point. 

You can add review guides up until you send applications to be reviewed. Once reviewers start scoring candidates, review guides become locked, and you won't be able to edit them further. 

This is so that you're scoring all candidates on the same criteria, and don't start reviewing candidates differently part way through the process.

Walkthrough an example review guide

Below is an example question and review guide taken from our Applied library.

Question title: Actions when managing an underperforming team

Skills tested: Teamwork, Complex problem solving, Leadership

Question Text: You have been asked to manage an underperforming team, who have had three different managers in the past year. What actions would you take in the first 3 months as their new manager?

Review guide

1 Star

• Does not seek to understand the team’s situation or history

• Offers little to no evidence that they will provide encouragement or empower team members

• Makes no attempt to define key responsibilities of team members

3 Star

• Recognises the importance of providing guidance to their team, but is somewhat unclear on how this will be done

• Identifies ways to encourage and improve current team performance, but this could be clearer or more detailed

5 Star

• Provides detail on how they would seek to understand the team’s current situation; using this to inform their approach as a manager

• Outlines how they would provide clear guidance on the requirements of the team’s roles

• Identifies ways to encourage more effective performance (e.g. constructive feedback, training), and suggests ways that they could monitor this going forward

What makes this a good review guide?

As you can see, the review guide makes it absolutely clear what an answer would touch upon in order to be considered a 5 star answer. The review guide above does not offer exact examples of what a candidate would say, since that would be very difficult to do, but rather provides guidance on what key things should be present in an answer in some shape or form in order to be considered a 1, 3 or 5 star answer.

In the above example, if a candidate did not identify in their answer that they would seek to understand the team's current situation in order to inform their approach, then that answer could not be considered a 5 star answer. The reasoning behind this is that in order to effectively solve complex problems like this, one of the first things a qualified candidate would think to do is to spend some time with their team to identify the current issues. This would allow them to understand what steps need to be taken to solve the issues they are specifically facing. 
 

If anything in this article piques your interest or isn’t clear, don’t hesitate to get in touch at [e-mail], where a member of our team will be on hand to give you support.