this post was submitted on 22 Jul 2023

2318 points (98.8% liked)

Lemmy.World Announcements

30121 readers

15 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages 🔥

https://status.lemmy.world/

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to info@lemmy.world e-mail.

Report contact

DM https://lemmy.world/u/lwreport
Email report@lemmy.world (PGP Supported)

Donations 💗

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Join the team

founded 2 years ago

MODERATORS

ruud@lemmy.world

lwadmin@lemmy.world

lwCET@lemmy.world

jelloeater85@lemmy.world

Serinus@lemmy.world

lw_mod_notification@lemmy.world

Thekingoflorda@lemmy.world

2318

Lemmy.world update: Downtime today / Cloudflare (lemmy.world)

submitted 2 years ago by ruud@lemmy.world to c/lemmyworld@lemmy.world

440 comments fedilink hide all child comments

Today, like the past few days, we have had some downtime. Apparently some script kids are enjoying themselves by targeting our server (and others). Sorry for the inconvenience.

Most of these 'attacks' are targeted at the database, but some are more ddos-like and can be mitigated by using a CDN. Some other Lemmy servers are using Cloudflare, so we know that works. Therefore we have chosen Cloudflare as CDN / DDOS protection platform for now. We will look into other options, but we needed something to be implemented asap.

For the other attacks, we are using them to investigate and implement measures like rate limiting etc.

(page 2) 50 comments

sorted by: hot top controversial new old

[–] Anaralah_Belore223@lemmy.world 20 points 2 years ago* (last edited 2 years ago) (1 children)

Benefit of using Cloudflare CDN:

Commenting and editing took about 0.5 seconds!

Also, ping is now from 200-300 miliseconds to just between 50 and 60 (depending on your ISP):

64 bytes from 172.67.218.212: icmp_seq=1 ttl=64 time=56.2 ms
64 bytes from 172.67.218.212: icmp_seq=2 ttl=64 time=60.2 ms
64 bytes from 172.67.218.212: icmp_seq=3 ttl=64 time=55.8 ms
64 bytes from 172.67.218.212: icmp_seq=4 ttl=64 time=58.9 ms
64 bytes from 172.67.218.212: icmp_seq=5 ttl=64 time=60.6 ms
64 bytes from 172.67.218.212: icmp_seq=6 ttl=64 time=60.5 ms
64 bytes from 172.67.218.212: icmp_seq=7 ttl=64 time=60.1 ms
64 bytes from 172.67.218.212: icmp_seq=8 ttl=64 time=55.0 ms
64 bytes from 172.67.218.212: icmp_seq=9 ttl=64 time=60.0 ms
64 bytes from 172.67.218.212: icmp_seq=10 ttl=64 time=61.4 ms
64 bytes from 172.67.218.212: icmp_seq=11 ttl=64 time=59.3 ms
64 bytes from 172.67.218.212: icmp_seq=12 ttl=64 time=58.5 ms
64 bytes from 172.67.218.212: icmp_seq=13 ttl=64 time=56.0 ms
64 bytes from 172.67.218.212: icmp_seq=14 ttl=64 time=60.6 ms
64 bytes from 172.67.218.212: icmp_seq=15 ttl=64 time=58.7 ms

[–] BitOneZero@lemmy.world 18 points 2 years ago* (last edited 2 years ago)

The bugs in Lemmy are such that you don't even need to touch a server for it to be vulnerable. Cloudflare does not defend against such mistakes. Other servers can trigger deep PostgreSQL logic problems within Lemmy. Growing pains, a lot of the federation code was never tested, and today's crash is due to a logic issue with lemmy_server mistakenly updating 1700 servers it knows of through federation for a delete instead of the 1 local server.

[–] mordred@lemmy.world 19 points 2 years ago (1 children)

Thank you for your efforts, work and results. Those "attackers" only deserve disgust.

load more comments (1 replies)

[–] itadakimasu@lemmy.world 18 points 2 years ago

I'm learning a lot by following lemmy.worlds actions. Appreciate the transparency!

[–] fence_prude@lemmy.world 18 points 2 years ago (4 children)

Where can we donate toward server costs?

load more comments (4 replies)

[–] Tag365@lemmy.world 17 points 2 years ago (1 children)

Why is Cloudflare so bad?

load more comments (1 replies)

[–] Resol@lemmy.world 17 points 2 years ago (3 children)

What doesn't kill you makes you stronger

load more comments (3 replies)

[–] drmoose@lemmy.world 17 points 2 years ago (3 children)

I hope lemmy.world can avoid using Cloudflare which goes against the spirit of Fediverse as it's just an objectively evil company.

[–] phar@lemmy.ml 16 points 2 years ago (11 children)

Can you give some insight to this?

load more comments (11 replies)

load more comments (2 replies)

[–] free@lemmy.world 16 points 2 years ago

ty for all ur hard work ♥️

[–] AphoticDev@lemmy.dbzer0.com 16 points 2 years ago (1 children)

It's not ideal, but there's not a whole lot of options out there for DDoS mitigation.

load more comments (1 replies)

[–] Anaralah_Belore223@lemmy.world 16 points 2 years ago* (last edited 2 years ago) (1 children)

The more you attack a Lemmy instance, the more stronger it gets.

It's like tempering iron or steel.

load more comments (1 replies)

[–] zazaserty@discuss.tchncs.de 14 points 2 years ago (3 children)

Damn these script kiddies.. I don't like Cloudflare at all but it does its job well. It may just be my paranoia, but putting a single entity in control of so many websites seems dangerous. I think we have all learned about the intentions of big corporations. But hey, it's better than being taken down tbf.

load more comments (3 replies)

[–] chipamogli@lemmy.world 14 points 2 years ago (1 children)

Maybe is that Reddit dude, jealous of Lemmy's increasing popularity.

[–] TerryMathews@lemmy.world 13 points 2 years ago (1 children)

It seems like you made this comment in jest, but I wouldn't say it's outside the realm of possibility. We can't fly off the handle and lob accusations absent any sort of proof, but it would hardly be the first example of a corporation targeting an up-and-coming disruptive service run by amateurs.

load more comments (1 replies)

[–] boulderly@lemmyadmin.site 14 points 2 years ago (2 children)

I put this site behind cloudflare in response to this post. Other than having to change SSL/TLS encryption mode to Full, it seemed easy. I turned on bot fight mode and I'm using the managed WAF ruleset that comes with the free tier. Any configuration recommendations anywhere in the panel?

load more comments (2 replies)

[–] db0@lemmy.dbzer0.com 13 points 2 years ago (2 children)

I wonder now with the semi-adversarial/semi-cooperative nature between lemmy instances, if wer'e not going to see more DDOS and other types of raids happening because a different instance has an ax to grind against yours. Say between you defederated them, or they consider your instance too big etc.

load more comments (2 replies)

[–] michael@lemmy.perthchat.org 13 points 2 years ago (5 children)

Hmmm, we're getting a fuckload of web requests on our Lemmy too... I think I'll enable CloudFlare too! :)

load more comments (5 replies)

[–] stevedidWHAT@lemmy.world 13 points 2 years ago

Welcome to the internet, glad there’s a working solution

[–] grandkaiser@lemmy.world 12 points 2 years ago

If you decide to use Akamai, hmu. I'm not an Alamai guru, but I do it professionally.

[–] thatguydude@lemmy.world 12 points 2 years ago

I joined July 1 for obvious reasons, I love it here.

I'm asking how many times has lemmy had to deal with these kinds of attacks prior to the "Date shall not be named"

Cause it kinda seems pretty coincidental for the amount of times I've been forced to be "offline" on this platform is gd laughable.

I thumbs up good content always.

load more comments