Phrenology circa 2020.
If there's a "right" way to do the audio thing, systemd/logind/polkit/pam/whatever make it too difficult if not impossible to figure out. This is its serious design flaw that I don't think can be fixed.
The solution for now:
sudo setfacl -m u:woz:rw /dev/snd/*
systemctl --user restart pulseaudio
I will have to do that on a per-session basis, but that's fine since I plan on keeping sessions around for the long term.
A bunch of technomancers in the fediverse. Keep it fairly clean please. This arcology is for all who wash up upon it's digital shore.