A Reviewer Link
When you are allocated Sift answers, CV's or written tasks to review, you will be sent an email from your Job Admin. Your email will contain a unique url link that allows you to access the answers that you need to score.
Please note - if you are unable to locate your original email, you can also view your reviewing links in your 'My Tasks' area.
What do I see once I click into the link?
You'll be asked if you'd like a quick reminder of how Applied works, if so then just click on the blue type and you'll go straight into a slideshow. Otherwise, just click onto 'Get Started!'
Next, you'll see:
- The total number of answers that you'll review
- An approximate time estimate for how long this will take
- A reminder of the job description and the main skills that your team are scoring
- FAQs that cover most areas of interest for first time reviewers
To move forwards click on the 'Review your first question' button.
Your first Question page
Running along the top of the page you'll see a tally of answers that you have been allocated as well as forwards and backwards arrows that can take you to the previous or next question. In the middle of the page will be a written reminder of the question itself. Below this will be a copy of the Review Guide that you will be expected to score by as well as an example AI answer provided by our new referent AI feature.
When you are ready click 'Ready To Review'
On this new page you will be presented by a Candidate's response and a star scoring system above, you simply need to click into the stars that you wish to award and the star rating will be saved against that candidates' answer. Again at the top of the page is the tally bar showing how many answers you are yet to review and those that have been completed.
There is also a reference area on the right hand again giving the scoring guide and an example of an AI generated answer so that you can compare this with the submitted by the candidate.
Simply progress through the answers that you have been allocated, taking regular breaks to prevent cognitive overload.
What happens next?
Your scores will be saved on the platform and accessed by the Job Admin who will take the next steps as agreed by your hiring team, in rejecting or moving forward each application.
Candidates who are rejected at this point can be sent feedback along with their rejection email.
Candidates who are amongst the higher scorers can be invited to interview and or to answer further questions or produce a new piece of work.
What are Review Stats?
Once you have completed reviewing you will be taken to a Review Stats page which shows how your scoring measured up against scores submitted by your hiring team members. You won't see how each person scored each answer, but you'll see anonymised averages, as well as an indication of whether your scoring pattern was higher, below or similar to the average.
Can I change my scores at any point?
If you feel that you need to revisit your scoring and alter any of the scores that you originally awarded, simply use the scoring links that were emailed out to you, they will remain active until the candidate has been moved into the next phase of the pipeline i.e. rejected, invited to interview or hired etc.
If you are going to alter your scores and there has been a few days in between your original scoring point and now, it's probably a good idea to alert your Admin, just in case they are poised to react to the original scores.
Behavioural science in the reviewer flow Rules for Scoring, Anonymisation and Randomisation are behavioural design principles that come into action at this point of the reviewer flow: Rules of Scoring. Your Review Guide will help you to be objective and consistent when identifying good, not so good and bad answers (click here to learn more about a typical structure of a Review Guide). Anonymisation. Names and other socio-demographic details can be distracting and result in inadvertent bias. That's why the review flow removes names and all candidate personal details so you just focus on who's given good answers. Randomisation. Our brains are heavily affected by ordering effects and small contextual factors around us. If we're hungry or tired, our scores are less reliable. All of this leads to lots of noise in the reviews, and a lack of objectivity about who's really good. That's why the review flow randomises the order of all candidate responses everytime you start a scoring cycle. So answer 1 from question 1 and answer 1 question 2 are not necessarily from the same candidate. That way, no candidate is disproportionately advantaged or disadvantaged by where they show up in the pile. |
If you have any questions at all on the above, please email into us on Support, at hello@beapplied.com
Comments
0 comments
Article is closed for comments.