So switching from iOS to Mac almost works but its bugged. It doesnt stop the iPhone sound but it connects back to the Mac anyway. Weird right?

So the temporary fix:

1.) Play something on your iPhone.

2.) Then to get your Mac and play something there (youtube video etc).

3.) Take one of the airpods out of your ear. This will pause the iPhone sound.

4.) Put it back in and sound will move from iPhone to Mac.

A bit annoying you have to do this but hey, it works.