It could probably work but would quickly turn into a mess of custom extensions.
For example, ActivityPub has no concept of sorting by hot or active or new. ActivityPub also doesn't specify how a client would authenticate to a server to post on your behalf. There's definitely no ActivityPub message for registering a user account.
So it makes sense that S2S only does the bare minimum for the purpose of federation of content, while instances with varying implementations can implement whatever C2S protocol makes the most sense.
For example, should ActivityPub expose a Lemmy post as a nested thread, or a Mastodon microblogging-like format and let the clients reassemble the thread? How should a Mastodon client present a Lemmy community and threads? How about a Lemmy client connecting to a Mastodon server?
If we put that in ActivityPub, you're pretty much bound to supporting it forever because other servers will eventually expect those protocol extensions, whereas it's much safer to change a C2S protocol.
Keeping the ActivityPub simple has a lot of benefits if we don't want the fediverse to remain really interoperable. A Lemmy client can reasonably expect that a given server supports a given set of features, providing a much more reliable experience than basically spaguetti of supporting every possible features and presenting the data weirdly.