So, docker is a viable solution, but since you're a fullstack and will likely add more shit than you can imagine in the future, you might as well setup a proper solution.
Check out Proxmox. It's a management platform that allows you to run containers and just about everything else you need for self-host. In addition to that, I recommend getting a very small VPS with a domain to reverse proxy your services if you want. I highly recommend caddy2 for this as it does rproxy and even ssl seamlessly.
I’m on a shitty 5G internet at home, so VPS seems like the way to go but with who?
Considering you have a poor internet connection, you'd want to keep as much locally as possible. You're not going to be able to stream HD movies with shitty internet if you host your media on a remote server, but if you rely on a local wifi network, it's fine. You won't have remote access to your movies (I mean you can, but like you said, shitty internet) it's not going to be awesome. Other services like your matrix server would be fine, but since you're self-hosting, might as well host them at home, too. Matrix isn't exactly resource heavy and doesn't require a shit ton of upload to make usable.
If I’m torrenting, do I need to be careful which hosts I choose so I don’t get copyright pinged?
If you're on 5G, and you torrent, you'll be found out almost immediately, even with a VPN. I highly recommend a seedbox. Download to the seedbox, then use rclone or something to grab the files to your local NAS cluster (in proxmox) then stream the video's locally.
Is there a good guide for securing and hardening my server?
I always recommend 2 things when dealing with *nix servers;
- Run SSH from a non-standard port and drop connections on port 22.
- Only open ports you're using.
IMO this is really the only hardening you need, especially if you're working with rproxy and the ports only have to be opened locally or tunneled.