Statistics about Sift and Interview Questions

At Applied we want to partner with you to build good quality questions with the help of data

That's why you'll notice different statistics about questions when you're building a role and are exploring your library of questions or checking out the reports section of a particular role.

 

Statistics on the Library of Questions

All the statistics that you see on the library of questions correspond to a 0-10 score. The closer a score is to 10, the better the quality of the question on that dimension.

We've created a traffic light system to make it easy for you to interpret the score given to a question on a particular dimension. Below is a summary of each dimension and the thresholds that make it a green, yellow or red label.

Dimension	What does it capture?	Thresholds for traffic light system
		New or N/A	Red	Yellow	Green
Maturity	It captures how many jobs have used the question and how many answers the question has received. The more mature the question, the better.	The question hasn't received any answers, therefore we cannot show you any scores.	N/A	From 0.0 to 7.4	From 7.5 to 10
Agreement	For each answer, it captures the similarity of the scores among reviewers. The overall agreement score aggregates the levels of agreement at an answer level. The more agreement, the less subjective the question is.		From 0.0 to 4.0	From 4.1 to 6.5	From 6.6 to 10
Spread	It captures how much a question separates candidates through the average scores they get. The more spread of average scores, the better.		From 0.0 to 5.9	From 6 to 7.4	From 7.5 to 10

Statistics on each role's report section

Average score 

After your hiring team reviewed all questions for a job, you'll see what was the overall average that candidates got. The higher the average score, the less likely that a question has helped you to separate the field.

Reviewer agreement

For each answer, it captures the similarity of the scores among reviewers. The overall agreement score aggregates the levels of agreement at an answer level. The more agreement, the less subjective the question is.

We're always open to feedback! Building these statistics is an ongoing and fascinating process at Applied, so we'd love to get your feedback about any bit that can improve your experience when interpreting and using these statistics.