I've setup Lemmy on a VPS with LetsEncrypt. When navigating to my domain, Lemmy just says "There was an error on the server. Try refreshing your browser. If that doesn't work, come back at a later time. If the problem persists, you can seek help in the Lemmy support community or Lemmy Matrix room."
I've checked both the Lemmy API and WUI instances, but no errors are reported:
lemmy@ubuntu-s-1vcpu-2gb-sfo3-01:~$ docker compose logs
postgres |
postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres |
postgres | 2025-02-17 18:21:24.644 UTC [1] LOG: starting PostgreSQL 17.3 (Debian 17.3-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
postgres | 2025-02-17 18:21:24.646 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres | 2025-02-17 18:21:24.647 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres | 2025-02-17 18:21:24.652 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres | 2025-02-17 18:21:24.685 UTC [27] LOG: database system was shut down at 2025-02-17 18:21:15 UTC
postgres | 2025-02-17 18:21:24.729 UTC [1] LOG: database system is ready to accept connections
lemmy-ui-1 | Lemmy-ui v0.19.9 started listening on http://0.0.0.0:1234/
lemmy-ui-1 | 31 translation imports verified.
lemmy-ui-1 | date-fns "zh" failed: unexpected format
lemmy-ui-1 | 1 out of 94 date-fns imports failed.
lemmy-ui-1 | 4 highlight.js imports verified. (Only testing 4 samples.)
nginx-certs-1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx-certs-1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx-certs-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx-certs-1 | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
nginx-certs-1 | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
nginx-certs-1 | /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
nginx-certs-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx-certs-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx-certs-1 | /docker-entrypoint.sh: Configuration complete; ready for start up
nginx-certs-1 | 106.75.133.150 - - [17/Feb/2025:18:21:43 +0000] "GET / HTTP/1.1" 301 169 "http://my.server.ip.addr/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36"
nginx-certs-1 | 106.75.133.150 - - [17/Feb/2025:18:21:45 +0000] "GET /favicon.ico HTTP/1.1" 301 169 "http://my.server.ip.addr/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36"
proxy-1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
proxy-1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
proxy-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
proxy-1 | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
proxy-1 | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
proxy-1 | /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
proxy-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
proxy-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
proxy-1 | /docker-entrypoint.sh: Configuration complete; ready for start up
proxy-1 | 75.154.243.157 - - [17/Feb/2025:18:21:47 +0000] "GET / HTTP/2.0" 500 5284 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
proxy-1 | 75.154.243.157 - - [17/Feb/2025:18:21:47 +0000] "GET /manifest.webmanifest HTTP/2.0" 500 21 "https://mydomain.xyz/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
proxy-1 | 75.154.243.157 - - [17/Feb/2025:18:21:48 +0000] "GET / HTTP/2.0" 500 5284 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
proxy-1 | 75.154.243.157 - - [17/Feb/2025:18:21:50 +0000] "GET /service-worker.js HTTP/2.0" 304 0 "https://mydomain.xyz/service-worker.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
certbot-1 | Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot-1 | Certbot doesn't know how to automatically configure the web server on this system. However, it can still get a certificate for you. Please run "certbot certonly" to do so. You'll need to manually configure your web server to use the resulting certificate.
pictrs-1 | 2025-02-17T18:21:23.900781Z INFO pict_rs: Starting pict-rs on 0.0.0.0:8080
pictrs-1 | 2025-02-17T18:21:23.901363Z INFO actix_server::builder: starting 1 workers
pictrs-1 | 2025-02-17T18:21:23.901410Z INFO actix_server::server: Tokio runtime found; starting in existing Tokio runtime
lemmy-1 | Lemmy v0.19.9
lemmy-1 | 2025-02-17T18:21:25.386757Z INFO lemmy_db_schema::schema_setup: Running Database migrations (This may take a long time)...
lemmy-1 | 2025-02-17T18:21:25.408878Z INFO lemmy_db_schema::schema_setup: Database migrations complete.
lemmy-1 | 2025-02-17T18:21:25.493688Z INFO lemmy_server::code_migrations: Running user_updates_2020_04_02
lemmy-1 | 2025-02-17T18:21:25.515048Z INFO lemmy_server::code_migrations: 0 person rows updated.
lemmy-1 | 2025-02-17T18:21:25.516901Z INFO lemmy_server::code_migrations: Running community_updates_2020_04_02
lemmy-1 | 2025-02-17T18:21:25.540687Z INFO lemmy_server::code_migrations: 0 community rows updated.
lemmy-1 | 2025-02-17T18:21:25.541173Z INFO lemmy_server::code_migrations: Running post_updates_2020_04_03
lemmy-1 | 2025-02-17T18:21:25.551391Z INFO lemmy_server::code_migrations: 0 post rows updated.
lemmy-1 | 2025-02-17T18:21:25.553517Z INFO lemmy_server::code_migrations: Running comment_updates_2020_04_03
lemmy-1 | 2025-02-17T18:21:25.571059Z INFO lemmy_server::code_migrations: 0 comment rows updated.
lemmy-1 | 2025-02-17T18:21:25.572187Z INFO lemmy_server::code_migrations: Running private_message_updates_2020_05_05
lemmy-1 | 2025-02-17T18:21:25.576403Z INFO lemmy_server::code_migrations: 0 private message rows updated.
lemmy-1 | 2025-02-17T18:21:25.578895Z INFO lemmy_server::code_migrations: Running post_thumbnail_url_updates_2020_07_27
lemmy-1 | 2025-02-17T18:21:25.681807Z INFO lemmy_server::code_migrations: 0 Post thumbnail_url rows updated.
lemmy-1 | 2025-02-17T18:21:25.684992Z INFO lemmy_server::code_migrations: Running apub_columns_2021_02_02
lemmy-1 | 2025-02-17T18:21:25.693262Z INFO lemmy_server::code_migrations: Running instance_actor_2021_09_29
lemmy-1 | 2025-02-17T18:21:25.734984Z INFO lemmy_server::code_migrations: Running regenerate_public_keys_2022_07_05
lemmy-1 | 2025-02-17T18:21:25.743673Z INFO lemmy_server::code_migrations: Running initialize_local_site_2022_10_10
lemmy-1 | Federation enabled, host is mydomain.xyz
lemmy-1 | Starting HTTP server at 0.0.0.0:8536
lemmy-1 | 2025-02-17T18:21:26.062209Z INFO lemmy_server::scheduled_tasks: Updating active site and community aggregates ...
lemmy-1 | 2025-02-17T18:21:26.233668Z INFO lemmy_server::scheduled_tasks: Done.
lemmy-1 | 2025-02-17T18:21:26.233702Z INFO lemmy_server::scheduled_tasks: Updating hot ranks for all history...
lemmy-1 | 2025-02-17T18:21:26.263454Z INFO lemmy_server::scheduled_tasks: Finished process_hot_ranks_in_batches execution for post_aggregates (processed 0 rows)
lemmy-1 | 2025-02-17T18:21:26.281667Z INFO lemmy_server::scheduled_tasks: Finished process_hot_ranks_in_batches execution for comment (processed 0 rows)
lemmy-1 | 2025-02-17T18:21:26.289482Z INFO lemmy_server::scheduled_tasks: Finished process_hot_ranks_in_batches execution for community (processed 0 rows)
lemmy-1 | 2025-02-17T18:21:26.289521Z INFO lemmy_server::scheduled_tasks: Finished hot ranks update!
lemmy-1 | 2025-02-17T18:21:26.289527Z INFO lemmy_server::scheduled_tasks: Updating banned column if it expires ...
lemmy-1 | 2025-02-17T18:21:26.310614Z INFO lemmy_server::scheduled_tasks: Clearing old activities...
lemmy-1 | 2025-02-17T18:21:26.329883Z INFO lemmy_server::scheduled_tasks: Done.
lemmy-1 | 2025-02-17T18:21:26.329909Z INFO lemmy_server::scheduled_tasks: Overwriting deleted posts...
lemmy-1 | 2025-02-17T18:21:26.345925Z INFO lemmy_server::scheduled_tasks: Done.
lemmy-1 | 2025-02-17T18:21:26.345960Z INFO lemmy_server::scheduled_tasks: Overwriting deleted comments...
lemmy-1 | 2025-02-17T18:21:26.388678Z INFO lemmy_server::scheduled_tasks: Done.
lemmy-1 | 2025-02-17T18:21:26.408661Z INFO lemmy_server::scheduled_tasks: Done.
Due to no errors showing up in logs, I'm unsure where to begin troubleshooting.
Here's my Nginx config:
events {
}
http {
limit_req_zone $binary_remote_addr zone=mydomain.xyz_ratelimit:10m rate=1r/s;
upstream lemmy {
server "lemmy:8536";
}
upstream lemmy-ui {
server "lemmy-ui:1234";
}
server {
listen 80;
listen [::]:80;
server_name mydomain.xyz;
location /.well-known/acme-challenge/ {
root /var/www/certbot;
}
location / {
return 301 https://$host$request_uri;
}
}
server {
http2 on;
listen 443 ssl;
listen [::]:443 ssl;
server_name mydomain.xyz;
ssl_certificate /etc/letsencrypt/live/mydomain.xyz/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mydomain.xyz/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
ssl_session_timeout 10m;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets on;
ssl_stapling on;
ssl_stapling_verify on;
server_tokens off;
gzip on;
gzip_types text/css application/javascript image/svg+xml;
gzip_vary on;
add_header Strict-Transport-Security "max-age=63072000";
add_header Referrer-Policy "same-origin";
add_header X-Content-Type-Options "nosniff";
add_header X-Frame-Options "DENY";
add_header X-XSS-Protection "1; mode=block";
client_max_body_size 20M;
location / {
set $proxpass "http://lemmy-ui/";
if ($http_accept = "application/activity+json") {
set $proxpass "http://lemmy/";
}
if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams/"") {
set $proxpass "http://lemmy/";
}
if ($request_method = POST) {
set $proxpass "http://lemmy/";
}
proxy_pass $proxpass;
rewrite ^(.+)/+$ $1 permanent;
# Send actual client IP upstream
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location ~ ^/(api|feeds|nodeinfo|.well-known) {
proxy_pass "http://lemmy/";
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
limit_req zone=mydomain.xyz_ratelimit burst=30 nodelay;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location ~ ^/(pictrs) {
# allow browser cache, images never update, we can apply long term cache
expires 120d;
add_header Pragma "public";
add_header Cache-Control "public";
proxy_pass "http://lemmy/";
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
limit_req zone=mydomain.xyz_ratelimit burst=30 nodelay;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location ~ /pictshare/(.*)$ {
return 301 /pictrs/image/$1;
}
}
map $remote_addr $remote_addr_anon {
~(?P<ip>\d+\.\d+\.\d+)\. $ip.0;
~(?P<ip>[^:]+:[^:]+): $ip::;
127.0.0.1 $remote_addr;
::1 $remote_addr;
default 0.0.0.0;
}
access_log /var/log/nginx/access.log combined;
}
Docker-compose:
networks:
lemmyexternalproxy:
lemmyinternal:
driver: bridge
internal: true
services:
proxy:
image: nginx:1.27.4-alpine
networks:
- lemmyinternal
- lemmyexternalproxy
ports:
- 443:443
volumes:
- ./volumes/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./volumes/certbot:/var/www/certbot:ro
- ./volumes/letsencrypt:/etc/letsencrypt:ro
restart: unless-stopped
depends_on:
- pictrs
- lemmy-ui
nginx-certs:
image: nginx:1.27.4-alpine
ports:
- 80:80
volumes:
- ./volumes/nginx_certs/nginx.conf:/etc/nginx/nginx.conf:ro
- ./volumes/certbot:/var/www/certbot:ro
- ./volumes/letsencrypt:/etc/letsencrypt:ro
restart: always
lemmy:
image: dessalines/lemmy:0.19.9
hostname: lemmy
networks:
- lemmyinternal
- lemmyexternalproxy
restart: unless-stopped
environment:
- RUST_LOG=verbose,lemmy_server=info,lemmy_api=info,lemmy_api_common=info,lemmy_api_crud=info,lemmy_apub=info,lemmy_db_schema=info,lemmy_db_views=info,lemmy_db_views_actor=info,lemmy_db_views_moderator=info,lemmy_routes=info,lemmy_utils=info,lemmy_websocket=info
volumes:
- ./volumes/lemmy-api/lemmy.hjson:/config/config.hjson
depends_on:
- pictrs
- postgres
lemmy-ui:
image: dessalines/lemmy-ui:0.19.9
networks:
- lemmyinternal
environment:
- LEMMY_INTERNAL_HOST=lemmy:8536
- LEMMY_EXTERNAL_HOST=mydomain.xyz
- LEMMY_HTTPS=true
depends_on:
- lemmy
restart: unless-stopped
postgres:
image: postgres:17.3
networks:
- lemmyinternal
container_name: postgres
restart: unless-stopped
environment:
POSTGRES_USER: lemmy
POSTGRES_PASSWORD: XXXXXXXXXXX
POSTGRES_DB: lemmy
volumes:
- ./volumes/postgres:/var/lib/postgresql/data
pictrs:
image: asonix/pictrs:0.5.16
hostname: pictrs
networks:
- lemmyinternal
environment:
- PICTRS__API_KEY=j9Dj3FxLxWL8mxMF
user: 0:0
volumes:
- ./volumes/pictrs:/mnt
restart: unless-stopped
certbot:
image: certbot/certbot:v3.2.0
volumes:
- ./volumes/certbot:/var/www/certbot/:rw
- ./volumes/letsencrypt:/etc/letsencrypt/:rw
And finally, Lemmy's config:
{
setup: {
admin_username: "lemmy"
admin_password: "XXXXXXXXXXXXXXXX"
site_name: "MYDOMAIN"
}
hostname: "mydomain.xyz"
bind: "0.0.0.0"
port: 8536
tls_enabled: true
pictrs_url: "http://pictrs:8080/"
database: {
database: "lemmy"
user: "lemmy"
password: "XXXXXXXXXXXXXXXXXXXX"
host: "postgres"
port: 5432
pool_size: 5
}
}
