This is a most excellent place for technology news and articles.
cross-posted from: https://programming.dev/post/35852706
Tech bros just actively making the internet worse for everyone.
Tech bros just actively making ~~the internet~~ society worse for everyone.
FTFY.
We had a trust based system for so long. No one is forced to honor robots.txt, but most big players did. Almost restores my faith in humanity a little bit. And then AI companies came and destroyed everything. This is why we can't have nice things.
Big players are the ones behind most AIs though.
I know this is the most ridiculous idea, but we need to pack our bags and make a new internet protocol, to separate us from the rest, at least for a while. Either way, most “modern” internet things (looking at you, JavaScript) are not modern at all, and starting over might help more than any of us could imagine.
Like Gemini?
From official Website:
Gemini is a new internet technology supporting an electronic library of interconnected text documents. That's not a new idea, but it's not old fashioned either. It's timeless, and deserves tools which treat it as a first class concept, not a vestigial corner case. Gemini isn't about innovation or disruption, it's about providing some respite for those who feel the internet has been disrupted enough already. We're not out to change the world or destroy other technologies. We are out to build a lightweight online space where documents are just documents, in the interests of every reader's privacy, attention and bandwidth.
Yep! That was exactly the protocol on my mind. One thing, though, is that the Fediverse would need to be ported to Gemini, or at least for a new protocol to be created for Gemini.
If it becomes popular enough that it's used by a lot of people then the bots will move over there too.
They are after data, so they will go where it is.
One of the reasons that all of the bots are suddenly interested in this site is that everyone's moving away from GitHub, suddenly there's lots of appealing tasty data for them to gobble up.
This is how you get bots, Lana
Won't the bots just adapt and move there too?
It's not the most well thought-out, from a technical perspective, but it's pretty damn cool. Gemini pods are a freakin' rabbi hole.
reminder to donate to codeberg and forgejo :)
I use Anubis on my personal website, not because I think anything I’ve written is important enough that companies would want to scrape it, but as a “fuck you” to those companies regardless
That the bots are learning to get around it is disheartening, Anubis was a pain to setup and get running
Is there nightshade but for text and code? Maybe my source headers should include a bunch of special characters that then give a prompt injection. And sprinkle some nonsensical code comments before the real code comment.
Okay what about...what about uhhh... Static site builders that render the whole page out as an image map, making it visible for humans but useless for crawlers 🤔🤔🤔
Accessibility gets throw out the window?
I wasn't being totally serious, but also, I do think that while accessibility concerns come from a good place, there is some practical limitation that must be accepted when building fringe and counter-cultural things. Like, my hidden rebel base can't have a wheelchair accessible ramp at the entrance, because then my base isn't hidden anymore. It sucks that some solutions can't work for everyone, but if we just throw them out because it won't work for 5% of people, we end up with nothing. I'd rather have a solution that works for 95% of people than no solution at all. I'm not saying that people who use screen readers are second-class citizens. If crawlers were vision-based then I might suggest matching text to background colors so that only screen readers work to understand the site. Because something that works for 5% of people is also better than no solution at all. We need to tolerate having imperfect first attempts and understand that more sophisticated infrastructure comes later.
But yes my image map idea is pretty much a joke nonetheless
AI is pretty good at OCR now. I think that would just make it worse for humans while making very little difference to the AI.
The crawlers are likely not AI though, but yes OCR could be done effectively without AI anyways. This idea ultimately boils down to the same hope Anubis had of making the processing costs large enough to not be worth it.
OCR could be done effectively without AI
OCR has been neural nets even before convolutional networks emerged in the 2010s
AI these days reads text from images better than humans can
Is there a migration tool? If not would be awesome to migrate everything including issues and stuff. Bet even more people would move.
Codeberg has very good migration tools built in. You need to do one repo at a time, but it can move issues, releases, and everything.
Increasingly, I’m reminded of this: Paul Bunyan vs. the spam bot (or how Paul Bunyan triggered the singularity to win a bet). It’s a medium-length read from the old internet, but fun.
