Update: Credential Stuffing Prevention Cheat Sheet #1920
Description
What is missing or needs to be updated?
Updates in #1871 should be reviewed for commercial COI and factual accuracy and potentially reverted.
How should this be resolved?
The changes should be reverted to the original generic wording , noting that:
- The entire piece reads like an advertorial to a specific service's offering than generic recommendations and encourage due diligence.
- The PR is opened by someone who is a Member of the "FriendlyCaptcha" GitHub organization (presumably an employee or manager), recommending themselves as the first option (updated to the last when challenged by the reviewer). Yet they failed to bring any factual basis (audits, benchmarks, etc) to their claims except suggest themselves as a better alternative. Clear conflict of interest.
- I reviewed the original PR review process, It asserts:
Ideally, prefer modern CAPTCHA services that ...
use cryptographic or [proof-of-work](https://en.wikipedia.org/wiki/Proof_of_work) challenges as they make automation economically impractical while requiring little to no input from legitimate users;
and retracts:
However, CAPTCHAs are not perfect, and in many cases tools or services exist that can be used to break them with a reasonably high success rate.
Such editorial movements appears to lack factual basis and the use of terms like "cryptographic" could lead to a false sense of security. We should not make a blanket assertion that PoW “makes automation economically impractical” without very specific, evidence-backed parameters (like Password Hashing Tutorial); based on my experience most sites cannot set difficulty high enough without resorting to tracking user activity or hurting user conversion rates.
I have done my own empirical research on the only open source option they recommended and both the implementation quality and the real economic resistance to automation appeared questionable. I also brought this issue to the attention of one of the open source maintainer mentioned in the PR months ago, they did not contest any of my findings.
In my opinion, this is a very new technology with limited adoption, and the current evidence is lacking and we should refrain from making explicit value judgements that compares these captchas with "traditional" ones and explicitly say one is to be preferred than the other.
I had a chance to look at FC's front end package, it uses BLAKE3 which is one of the supported algorithms in my research. I cannot legally wire it up to their closed source SaaS backend, but I believe given the comparison to open source analogues it is reasonable to assume should one do a benchmark the throughput outcome would be comparable.