A few months ago, our contact form inexplicably stopped working. The page itself would load fine, the fields would behave normally, and the submit button would do what it was supposed to – but actually, submissions were coming in at only a third of the previous rate. It took a while for anyone to notice, but when they did, it turned out that we had been silently rejecting a considerable amount of users for the entire time.
It took one particularly annoyed reader to report the issue to us before we could actually get anywhere close to diagnosing it. As it turned out, our CAPTCHA was not working correctly for users behind a VPN or a privacy-focused browser, and rejecting them without telling them so. They would get no indication of what had happened, leaving them thinking that they had made some sort of mistake.
This is the point where I have to admit that, prior to this experience, I actually hadn’t known precisely what a CAPTCHA challenge response was. Most resources I could find online described the same vague general idea, but none of them went into detail. What is the best beginner-friendly explanation for what a CAPTCHA challenge response is, then? Glad you asked.
The short version
A CAPTCHA challenge response is essentially the authentication token, or verification response, that gets sent to a website upon successful completion of a CAPTCHA test. The challenge itself is whatever the user has to type or click in order to pass the test, whereas the response is the information that gets sent to the website’s server to indicate that the test has been passed.
The long version – separating the terms
The reason why most descriptions for the term “CAPTCHA challenge response” are so vague is because there isn’t really one set standard for how CAPTCHA systems work. CAPTCHA itself is an acronym for Completely Automated Public Turing test to tell Computers and Humans Apart – a name which reflects its primary purpose rather well. Individual CAPTCHA implementations, on the other hand, are based around the same concept of “a test that only humans can pass” and little else.
Challenge-response authentication itself is actually a type of security protocol wherein one party asks the other to prove their identity by answering a question, usually by means of cryptographic functions. Put the two together, and you get “a public Turing test to tell computers and humans apart, presented through a cryptographic challenge-response protocol.”
As you can probably guess, the phrase itself refers to the information sent back to the website after the CAPTCHA has been completed. This can be the string of text a user has entered if they are using a text-based CAPTCHA, or the information sent back by an invisible CAPTCHA implementation. Either way, this information is used to verify the user’s identity – and the next time someone asks you “what is a CAPTCHA challenge response?” – you can explain it to them in detail.
What actually happens behind the scenes
To put it all into perspective, I think it might be useful to actually outline the process that takes place in a CAPTCHA system. It’s rather simple, really – only three things need to happen in order to complete what we commonly know as a CAPTCHA challenge response:
Challenge – The website injects the CAPTCHA script onto the webpage, which either asks the user to perform some action or silently collects information about their browser.
Response – After the user completes the CAPTCHA, the script sends back a response, usually in the form of an authentication token.
Verification – The website then uses the response to verify the authenticity of the user’s request.
It’s the third step that many people seem to misunderstand, including us. Most people are aware of the fact that CAPTCHA responses can be denied, but very few of them are actually aware of what happens to a request after it has failed CAPTCHA verification. The truth is, it all depends on the website. Some sites will immediately block the request, whereas others may ask for more information or prompt the user to try a different CAPTCHA type. In our case, however, it appears that we simply failed to deliver the information to the user in time.
What different kinds of CAPTCHA are there?
There are a number of different ways that CAPTCHA implementations can appear to the user, depending on the specific technologies they use. At present, most of the CAPTCHA systems that are actually used in practice fall under one of the following categories:
TypeDescriptionWhat constitutes the response
Classic CAPTCHAAsks the user to type a scrambled word or phraseThe string of text entered by the user
CheckboxAsks the user to click a box indicating that they are not a robotInformation about the user’s behavior leading up to clicking the box
Image CAPTCHAAsks the user to click on a set of matching objects in an imageInformation about the user’s behavior, combined with the selected objects
Score-based CAPTCHA (invisible)Does not ask the user to do anythingVisible: nothing; Invisible: information about the user’s behavior
Managed CAPTCHAAsks the user to click on an image or solve a CAPTCHA puzzleInformation about the user’s behavior
The most troublesome CAPTCHA type by far, however, is score-based CAPTCHA. Invisible CAPTCHA implementations, especially the versions that use the “risk score” system, are notoriously difficult to troubleshoot due to the lack of on-page feedback. This is the reason why many users who encounter invisible CAPTCHA challenges report that they have “passed CAPTCHA verification but got blocked anyway.” The truth is that, in invisible CAPTCHA implementations, “passing CAPTCHA verification” does not actually mean “passing CAPTCHA scoring.”
Troubleshooting CAPTCHA problems, from a first-time perspective
After I got a basic understanding of what the problem was, I went back to the support ticket and analyzed it from a troubleshooting perspective. In short, here are the things that I checked, and the things that I saw:
User trust score cutoff was too high – If the website used reCAPTCHA v3 or a similar score-based CAPTCHA system, I checked the cutoff score for “trustworthy user” and compared it to the traffic statistics. For many reCAPTCHA implementations, the default cutoff is “0.5 or higher.” A reasonable-sounding number, if you ask me, that hides a particularly nasty detail – 50% of traffic failing CAPTCHA verification simply because, for whatever reason, their behavior did not match the expected CAPTCHA user behavior. We had to adjust the cutoff score afterwards, as well as add a few additional rules for traffic that scored “medium.”
Failed CAPTCHA
Failed CAPTCHA verifications are invisible to the user – This was the root cause of the issue, and the one that ultimately led to it. The majority of CAPTCHA implementations do not actually provide any kind of on-page feedback when the user fails CAPTCHA verification, invisible to them. This means that there is no way for the user to know what has happened, leaving them to believe that they have made a mistake when they have not. If you are the website owner, you can and should test this for yourself by attempting to fail CAPTCHA verification on a page. If you do not see any kind of on-page message telling you that you have failed CAPTCHA verification, chances are that your CAPTCHA implementation is broken.
Shared IP address ranges may have too many users with low trust scores – This one is primarily relevant for large-scale organizations, but it is worth mentioning regardless. If you are using a cellular network or a company-owned IP address, your IP address range may be shared with a number of other users, some of whom may be bots or malicious attackers. This can result in a significantly lower trust score for everyone on that IP range.
Mismatched or outdated server keys – On the website server side, there are a number of possible causes for CAPTCHA verification failures, including mismatched or expired keys. If you are the website admin, make sure to double-check your server time, as well as your reCAPTCHA API keys, if this issue started happening suddenly and recently.
On the user side, common causes include cache issues, browser extensions, and the use of a VPN. Clearing the browser cache, disabling browser extensions, and disabling the VPN altogether should rule out most common issues.
Misconceptions and things that the CAPTCHA “response” actually does not verify
One of the things I learned from this experience is that a CAPTCHA response actually does not verify anything really. It merely says, “this particular request passed all of the CAPTCHA-related checks at this particular moment.” If you are looking for an authentication method that never fails and has virtually no effect on user experience, then CAPTCHA is most definitely not for you.
Some of the things to consider when evaluating CAPTCHA as an authentication method include the following:
The CAPTCHA response itself says practically nothing about the user and their request beyond “this particular request passed CAPTCHA checks at this particular moment.”
Some CAPTCHA implementations, such as Google’s reCAPTCHA, can be bypassed using third-party services.
CAPTCHA can significantly hurt the conversion rate of your website, depending on the implementation and the expected user traffic.
CAPTCHA is not accessible to people with disabilities by default, and requires additional effort to make it work for them. Other, more modern CAPTCHA implementations, such as the score-based ones described above, are even more difficult to use for people with learning disabilities.
Most CAPTCHA implementations only provide “application-layer” security. In other words, they only affect web requests sent directly to your website. They do nothing to protect your DNS records, TLS certificates, or anything else lower in the stack.
The most common FAQs
The next set of questions, as can be expected, concern the most common misunderstandings surrounding the phrase “CAPTCHA challenge response:”
Is a CAPTCHA challenge the same thing as a CAPTCHA code?
No, a CAPTCHA challenge is the instruction that tells the user what to do in order to prove that they are not a robot. A CAPTCHA code, on the other hand, is the information that the user enters in response to the challenge.
Why do I keep getting CAPTCHA challenges, even when I answer them correctly?
There are a number of possible reasons for this, but incorrect or low trust scores are by far the most likely ones. In many CAPTCHA implementations, the user has to pass a CAPTCHA test twice before they are actually allowed to proceed.
Can I disable CAPTCHA and use something else to stop bots?
Yes, there are a number of CAPTCHA alternatives that can be used to stop bots without requiring the user to pass extra tests every time they visit your website. Some of them include honeypot fields, rate limiting, server-side request filtering, and CAPTCHA alternatives like Turnstile. It is also common for websites to use a combination of these methods in tandem with CAPTCHA to block bots.
Does failing a CAPTCHA test mean that I am registered as a bot?
Not really. Failing a CAPTCHA test only means that your request failed CAPTCHA verification. This can happen for a number of reasons, and it does not necessarily mean that you are a bot. If you want to avoid this happening to you in the future, consider disabling any unnecessary privacy-related browser extensions or using a different network.
Conclusion
A CAPTCHA challenge response is not one of those concepts that most website owners or operators are likely to think about, unless they happen to run into serious problems with their CAPTCHA implementation. As such, most of them are not familiar with the precise details of what a CAPTCHA response actually is, or what goes into troubleshooting a failed CAPTCHA response. Hopefully, this guide has helped you understand these concepts better, so that the next time you encounter a CAPTCHA-related issue, you know how to analyze and resolve it.
If you are a website admin or operator, I encourage you to spend some time reviewing your CAPTCHA-related settings and making sure that your CAPTCHA implementation is working correctly. It can take as little as five minutes to verify that your CAPTCHA implementation does not reject any of your traffic due to low trust scores, and it can save you hours of headaches down the road.
