Actually, no sooner than I finished writing this post did I find a solution that works 90% of the way there. I can bring up the go to folder prompt with cmd+shift+G already, but I realised if I want it to tell me the current folder I can just type "." and it will display the current path and you can also just cmd+c to copy that to the clipboard. I should have realised that the go to folder bar behaves just like terminal in many ways so there'd be common commands for both.
I guess this isn't a complete solution because if I wanted the filepath of a selected object in Finder it wouldn't work as it would only give you the parent directory, but that's actually what I need probably more than 90% of the time and given the rest of the time where I need a full filepath for one specific object is so rare, it's really not a problem to have to occasionally use the mouse.