Docker docs:
Docker routes container traffic in the nat table, which means that packets are diverted before it reaches the INPUT and OUTPUT chains that ufw uses. Packets are routed before the firewall rules can be applied, effectively ignoring your firewall configuration.
I assume portainer communicates via the docker socket? If so, couldn’t you just point portainer to the podman socket?
Portainer Docs | Install Portainer CE with Podman on Linux The official docs also mention doing that.