this post was submitted on 04 Mar 2025
517 points (99.1% liked)
Technology
63897 readers
5058 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Disgusting and unsurprising.
Most web admins do not care. I've lost count of how many sites make me jump through CAPTCHAS or outright block me in private browsing or on VPN. Most of these sites have no sensitive information, or already know exactly who I am because I am already authenticating with my username and password. It's not something the actual site admins even think about. They click the button, say "it works on my machine!" and will happily blame any user whose client is not dead-center average.
Enter username, but first pass this CAPTCHA.
Enter password, but first pass this second CAPTCHA.
Here's another CAPTCHA because lol why not?
Some sites even have their RSS feed behind Cloudflare. And guess what that means? It means you can't fucking load it in a typical RSS reader. Good job!
The web is broken. JavaScript was a mistake. Return to ~~monke~~ gopher.
Fuck Cloudflare.
Ever been down the gemini rabbit hole? It's not perfect, but quite interesting.
I get why you're frustrated and you have every right to be. I'm going to preface what I'm going to say next by saying I work in this industry. I'm not at Cloudflare but I am at a company that provides bot protection. I analyze and block bots for a living. Again, your frustrations are warranted.
Even if a site doesn't have sensitive information, it likely serves a captcha because of the amount of bots that do make requests that are scraping related. The volume of these requests can effectively DDoS them. If they're selling something, it can disrupt sales. So they lose money on sales and eat the load costs.
With more and more username and password leaks, credential stuffing is getting to be a bigger issue than anyone actually realizes. There aren't really good ways of pinpointing you vs someone that has somehow stolen your credentials. Bots are increasingly more and more sophisticated. Meaning, we see bots using aged sessions which is more in line with human behavior. Most of the companies implementing captcha on login segments do so to try and protect your data and financials.
The rise in unique, privacy based browsers is great and it's also hard to keep up with. It's been more than six months, but I've fingerprinted Pale Moon and, if I recall correctly, it has just enough red flags to be hard to discern between a human and a poorly configured bot.
Ok, enough apologetics. This is a cat and mouse game that the rest of us are being drug into. Sometimes I feel like this is a made up problem. Ultimately, I think this type of thing should be legislated. And before the bot bros jump in and say it's their right to scrape and take data it's not. Terms of use are plainly stated by these sites. They consider it stealing.
Thank you for coming to my Tedx Talk on bots.
Edit: I just want to say that allowing any user agent with "Pale Moon" or "Goanna" isn't the answer. It's trivially easy to spoof a user agent which is why I worked on fingerprinting it. Changing Pale Moon's user agent to Firefox is likely to cause you problems too. The fork they are using has different fingerprints than an up to date Firefox browser.
You're definitely right that it's a game of one-upping each other. Unfortunately, it's now directed in a path that infringes on privacy of the users it aims to serve.
Since you're working in the internet security industry, what's your take on something like Altcha as opposed to more invasive means of protecting against both attacks?
Trust me, my team and I often feel at odds with the part that infringes on privacy. As someone that enjoys and wants more privacy, I wish there were other solutions that didn't create a type of dragnet. If it assuages some of your fears, I've never heard of the fingerprinting being sold or used outside of detections.
Emphasis are mine. I honestly do not know how this statement is possible. Captcha-less, proof-of-work solutions have to fingerprint on some level. It's essentially having the browser prove it is what it claims to be. I get what they're trying to say but it's marketing. That said, I don't know everything and maybe they have some method I'm not aware of. Grains of salt all around.
Thanks for sharing!
Thanks for reading and commenting!
During my first (shitty) job as a dev outta school, they had me writing scrapers. I was actually able to subvert it pretty easily using this package that doesn't appear to be maintained anymore https://github.com/VeNoMouS/cloudscraper
Was pretty surprised to learn that, at the time, they were only checking if JS was enabled, especially since CF is the gold standard for this sort of stuff. I'm sure this has changed?
Given that the last updates to this repo were five years ago, I'm not too sure if it's still valid. I don't follow Cloudflare bypasses but I am fairly certain there are more successful frameworks and services now. The landscape is evolving quickly. We are seeing a proliferation of "bot as a service", captcha passing farms, dedicated browsers for botting, newsletters, substacks, Discord servers, you name it. Then there are the methods you don't readily find much talk on like custom modified Chrome browsers. It's fascinating how much effort is being funneled into this field.
But captchas have now proven useless, since bots are better at solving them now than humans?
Welcome to bot detection. It's a cat and mouse game, an ever changing battle where each side makes moves and counter moves. You can see this with the creation of captcha-less challenges.
But to say captcha are useless because bots can pass them is somewhat similar to saying your antivirus is useless because certain malware and ransomware can bypass it.
Dude, thank you for this context. I was already aware of these considerations but just wanted to thank you for sharing this with everyone. Its participation like this that makes the internet a better place. 🍻
That's very kind of you. Thank you for the kind words. 🍻
Thank you for that info, very helpful.
Thank you for reading and considering the information.
Also Cloudflare adds a caching layer, often physically closer to users. Increasing speed of delivery and reducing server costs. It's a no-brainer for server admins.
Also, I don't work for Cloudflare either. The animosity is new to me, and certainly something I'll look into.
Ever heard of counting attempts? Log the IP, present a CAPTCHA after 100 requests in a minute.
Besides, if I wrote a bot I would run a browser dialer from Chrome. It would request your site in a Chrome tab and appear completely legitimate to your stupid fingerprinting scripts
Ever heard of IP rotation? This is one malicious source rotating through IPs over the course of 24 hours. They're attempting to credential stuff my logins ( on a production service ).
Yes, the industry is well aware of this. We do behavioral detection on both sessions and IPs. This is fairly basic.
Yeah, it's fine as long as you don't block legitimate users. For example, when I use a VPN a lot of sites block me. Even when my actual IP is banned when I'm in China (4chan range bans Chinese IPs) or the website is blocked in China.
LibreWolf is next, and it's not exactly niche. I'm seeing it more and more, and LW defaults, even dropping resist settings, gets bounced by CloudFlare every time.
Fire dragon here and yeah, sometimes Google won't even let me log in either.
https://tildeverse.org/
Tilde.teams and tilde.club even have outwardly facing email accounts.
We have a newsgroup server.
We have a dedicated irc server.
Member gopher/https/gemini pages.
And other services.
And each tilde has it's own focus.
Be kind. Contribute as you can to discussions.
What is gemini
https://tilvids.com/videos/watch/e1d6ed23-315a-4fc6-8d5b-6d96d51e4819
Rocking the web bloat.
https://media.ccc.de/v/mch2022-83-rocking-the-web-bloat-modern-gopher-gemini-and-the-small-internet
Be Free.
So cute :)
It's not much, but it's home. :)