[ALSA] Fix possible deadlocks in sequencer at removal of ports