What to expect as a reviewer of short-answers to Sift questions

If your organisation is using Applied, there is good chance that at some point you'll become (or already are) a reviewer of candidate's Sift question answers.

Here you'll find:

• the behavioural design principles underlying Applied's signature Sift; and,
• the main steps that you can expect as a reviewer.

 

Behavioural design principles underlying Applied's Signature Sift 

Principle	Why?	What it means for you
Anonymisation	Names and other socioeconomic details can be distracting and result in inadvertent bias.	We remove names and all candidate personal details so you just focus on who's got the goods.
Chunking	It's hard to simultaneously compare candidates in multiples areas at once, and that leads to cognitive overload. Reviewing candidates top to bottom also means we can fall prey to the 'halo effect', where it a candidate starts of strong (or weak), that affects everything we read thereafter.	Instead of reviewing candidates' application in full vertically, we make it easy for you to do direct, horizontal comparisons of candidates - reviewing a batch of answers to question 1, then to question 2 and so on.
Randomisation	Our brains are heavily affected by 'ordering effects' and small contextual factors around us -we tend to be kinder to those at the beginning, or those just after a poor response. If we're hungry or tired, our scores are less reliable. All of this leads to lots of noise in the reviews and a lack of objectivity about who's really good.	We randomise the order of all candidate responses. That way, no candidate is disproportionately advantaged or disadvantaged by where they show up in the pile.
Crowdsourcing	We all have a slightly different way of seeing the world, and that means we rarely agree completely on what 'good' looks like. That means hiring decisions left to one person can end up being skewed by their personal perspective. Alternatively, hiring decisions made in open committee can result in groupthink or social hierarchy biases where our true opinions don't surface.	We allow you to gather views from multiple members of your team simultaneously so that a candidate's overall score is a more holistic measure of their quality. We do this independently, so no-one's affected by anyone else's perspectives.

Main steps as a Reviewer

1. You will get an email from HR inviting you to anonymously review some applications. This email contains a unique personal link to those answers that were allocated to you.

2. Click on the link and follow the instructions there (or you can view assigned tasks to you by logging into your dashboard here).

3. The core task you'll do through the link is to score each answer on a 1-to-5 scale. Your role as a Reviewer is to score the candidate answers using the skills tagged to questions and the Review Guide. The Review Guide is the measure of what "Good" looks like - i.e. each answer should be scored in isolation and on its own merits, using the guide to qualify the quality of answers submitted.

4. Once you've sifted all the answers you can check how you scored them compared to the others in your team. You can return to this last page anytime you'd like, but remember that you'll not be able to change the scores you gave once you have completed all your reviews.