Your audio hardware (codec) probably has powersaving enabled – the kernel powers it off after no program is using it anymore, and it takes a while for the hardware to properly turn on once it's needed again (sometimes intentionally, so that loud power-on pops could be avoided).

To disable this temporarily (assuming the system is using Intel HDA, which many do):

echo 0 > /sys/modules/snd_hda_intel/parameters/power_save

Permanently:

1. Add options snd_hda_intel power_save=0 to /etc/modprobe.d/modprobe.conf
2. Rebuild the initramfs and reboot

Your audio hardware (codec) probably has power saving enabled – the kernel powers it off after no program is using it anymore, and it takes a while for the hardware to properly turn on once it's needed again (sometimes intentionally, so that loud power-on pops could be avoided).

To temporarily lengthen the powersave timeout or even completely disable this feature, run (assuming the system is using Intel HDA, which many do):

echo 0 > /sys/module/snd_hda_intel/parameters/power_save

Permanently:

1. Add options snd_hda_intel power_save=0 to /etc/modprobe.d/modprobe.conf
2. Rebuild the initramfs and reboot

The number indicates how many seconds the device can remain idle, but 0 disables power-off.