is anyone really surprised?
this post was submitted on 10 Oct 2025
203 points (97.2% liked)
Technology
76040 readers
3240 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
Oh no, not information that's already available online, whatever will we do.
If you need AI to tell you how to build weapon system you're not going to build the weapon system anybody who's an actual threat already has this information. This is just nonsense pearl clutching to sell a story, there's nothing actually here though.
Remember kids, if you want to look up something that you don't want the government to know about, don't use the internet to do it.
Also, LLMs are not the best source for asking about how to make things that explode.
Uhm, why not go to true and trusted Wikipedia? TM 31-210 Improvised Munitions Handbook
The TM 31-210 manual appeared as an "Easter egg" in the 1995 CGI animated film, Toy Story. In the scene where Woody is trapped under a blue plastic box in Sid's bedroom, it's possible to see behind him a document titled "TM 31-210 Improvised Interrogation Handbook", a clear reference to the actual document.
Like, every search engine would yield the exact same results. It doesn't mean the average person would have the means or necessary requirements to develop it.
Do these morons think that because someone uses ChatGPT it magically gives access to those materials to make a bomb?
This is actually a marketing approach.
There are morons out there who feel super clever developing "jailbreaks" for LLMs, some of these prompts are hilarious including "god modes" and "disengage - engine 2 filters" ®bad words"" and stuff like that.
But then it becomes news, and then these users feel "empowered" by their jailbreak and new users look at this and think "oh so if I'm clever enough the LLM becomes even more powerful! I'm clever, so I'm going to try it!" which is ultimately what OpenAI wants.
You can't "bypass the system prompt" because that's not how it works. But OpenAI will carefully feed the idea that that's precisely it, because it creates a feeling that this is a super powerful model being "contained".
Again, it's marketing. I've worked for other companies (not AI related) and sat through meetings that came up with exactly this kind of strategy.
Or, occam's razor - AI companies are worried about PR and are implementing safeguards, but due to the nature of this technology it's very hard (or maybe even impossible) to make those safeguards robust.
Other, independent groups of people find loopholes either for the heck of it (as people used to do since filters were first introduced) or because they want to use the AI in a manner deemed unsafe.
Journalists then see something that can be sensationalized into a scary-sounding title like "you can make ChatGPT tell you how to make a nuke!!" or "you can make ChatGPT encourage suicide!!" and they run with it because it makes people click.
Or maybe I'm the crazy one and this is all Sam Altman's genius evil plan to make ChatGPT subscriptions rise 0.2% per quarter. Maybe your comment and my response are also mere cogs in this marketing machine. We will never know.
Yea but its not end uses being targeted, its investors.
Damn that makes a lot of sense. Thx!
I made a kilo of black powder a couple of years ago for my old-school guns. Sulfer, charcoal and stump killer is not exactly hard to come by. Neither is fertilizer and diesel fuel.
Biggest domestic terror attack in US history used a truck full of the later.
As much as I don't want chatbots to explain to morons how to harm people, I don't like that this just seems to be a form of censorship. If it's not illegal to publish this information, why should it be censored via a chatbot interface?
It's irrelevant anyway because the sorts of people who would want to make a bomb to harm others are not the sort of people that would be able to follow the instructions.
It is more likely than anything else that they would blow themselves up with some nitroglycerin. Even professionals used to do that back in the day because it was so unstable. I can imagine that a MAGA would be able to top 1900s scientists.
What about iron 2 oxide and aluminium powder? Seems simple enough to get.
Spicy k-cups are available commercially
Did you actually try that?
Lol, yeah. The anarchists handbook has been in public domain longer than most people in this thread have been alive. Yeah it's absolutely available on a search engine you could have got it on alta vista.
How do you think people figure out how to make IEDs do you think it's some secret knowledge pass down from father to son, no, they get it online or they just working out from basic principles of scientific understanding. Trying to contain knowledge never works.
I didn't ask if it was available, I asked if a typical search engine would lead you to it. Because it won't.
It's literally on Amazon.
I downloaded local Llama Uncensored and it easily teaches me how to make a home made bomb, suicide methods etc...
This isn't news anymore, anyone can have access to such things.
You don’t even need an LLM, just an internet connected browser.
Or literally just buy some fertiliser. We've all seen what happens when some ammonium nitrate catches fire, if you have enough of it in one place it's practically a nuclear bomb level detonation.
You don't need a browser just use cURL
When I first got internet in 95, it was easy to find stuff like that. I even made a website about making explosives for my computer class. Got a good grade for it and everything. Nobody said anything. Kind of weird if I think of it now. Anyway, making explosives as a hobby is a real bad decision. Most people understand that. The ones that don't are not smart enough to make them. The ones that are smart enough and still want to make them, would not use chatgpt.
It's really easy to make explosives. Making them stable and reliable is the hard part.
Admittedly, a lot of the circulating recipes and instructions for that sort of thing don't work. The infamous Anarchist's Cookbook is full of incorrect recipes. The problem might come from a LLM filtering out debunked information.
Id still want to double check😀.