How does the AI Detection Tool work?

In a world where AI is giving immeasurable benefits to our day to day workspace, we have also recognised the need for a tool, that can identify with confidence when a candidate has used AI to respond to application assessments.

Background

Which is why we have partnered with Pangram Labs - a company of AI experts working to provide high quality, accurate detection for use in Education and other areas where accuracy is of utmost importance, to introduce the Applied AI Detection Tool.

We take the detection of AI very seriously, which is why from the beginning of our research and development into this area we set a near-zero false positive rate requirement as standard. 

How does it work?

Every answer that Applied runs through the AI Detection Tool is classified based on the likelihood generated by a large language model. Applied flags and displays the answers that are very highly likely AI, i.e. those answers for which the AI Detection Tool has given a very high probability of being AI. These answers flagged as AI consistently exhibit stylistic choices that AI uses.

What answers will the AI Detection Tool check?

It will check text answers over 50 words in length. If the AI Detection Tool recognises an answer that is less than 50 words, the result will be flagged as being too short to analyse.

These are  scenarios of text answers that will be flagged:

Scenarios: Use of AI -Large Language Models (LLM)-	Potentially flagged as highly likely AI?
Candidate copies and pastes an answer directly from AI	Yes
Candidate writes text initially, then asks the AI to rewrite it	Yes
Candidate writes text originally, and then asks the AI to rewrite it and instructs it to not be detectable	Yes
Candidate uses AI text and then does quick edits before submitting answer (e.g. 1-2 minutes rewrite)	Yes
Candidate uses AI text and then does more thorough edits before submiting answer (e.g. 5-10 minutes of rewriting 250 words)	Sometimes flagged, sometimes not flagged
Candidate puts AI text through 'humanisers', i.e. techniques that try to modify, distor and obfuscate AI text to hide its true autorship	Yes

Accuracy 

The AI Detection Tool has been measured to have a detection rate of 99.99%+. This compares to a 90 -98% accuracy rate seen with other detectors on the market (for example, GPTZero, Originality, DetectGPT).

How do I enable this feature?

Any member of your team with ‘org admin’ permission level can switch on the AI Detector in your Organisation’s Settings page. Select the ‘AI Detector’ features and toggle on to enable. Once you do this, all roles will have this written AI feature detector enabled.

Who can see these AI Detection Flags?

Only org admins or hiring team members with access to the Candidate List View on a role will be able to see the flags and which candidates had potential AI answers. 

Why can't my hiring team see these flags in review?

The only place to view these hiring flags is before the candidates are sent to Sift review. This is to ensure that your team can decide who goes to review and when but the reviewers aren't biased when reviewing. This is also to decrease the cognitive load when reviewing.

Can I disable the AI Detection Tool for specific roles?

The initial release of the AI checker will be enabled at an Organisation level only. This means that the AI Detection Tool can't be disabled for specific roles, but even if you have it turned on for your org but for specific roles, if you do not want to use this feature, you can completely ignore these flags as they will not have any impact on hiring or reviewing and won't be seen by any reviewers. 

Is there a charge for using this feature?

If you are a paid Applied customer, the AI Detection Tool will be made available to you at no extra charge.

If you would like more information on this, please contact your Customer Success Manager or Applied Support at [e-mail]