This is a hardware problem and so will not be easily solved in any way but with a change in hardware. You need to put physical and electrical distance between the mouse and the speaker output. How easily this can be done depends on what you already own.
The easiest way I found to fix this is with a USB audio adapter. They can be bought for something like $20 and they are about as "plug-n-play" as devices can get. You may already have one and don't realize it. If you have a smartphone with a USB-C headset, and a USB-C port on your computer, then you can at least test this theory by plugging in the phone headset to the computer. It is theoretically possible for this noise to find it's way back out the USB audio adapter but so far this worked for me for the dozen or so times I came across this issue.
Because the mouse noise is entering from the mouse into the USB port there's no fixing this with a new mouse. Anything that looks like a mouse to the computer will generate the same noise, because all mice will be "speaking the same language", and peaking this into the same USB bus.
Another thing to try would be using Bluetooth, for the mouse or the audio out. If you have Bluetooth on the computer and a Bluetooth headset or speaker for your phone then you can test this as a means to get audio out without buying anything. I know Bluetooth mice are a bit hard to find, and some so-called "Bluetooth mice" are actually using something very different, something that will not solve the problem because they still send a mouse signal into the computer, not a Bluetooth signal.
If you have PS/2 or serial ports then maybe you can try those for connecting the mouse. This sometimes work, sometimes makes it worse. If you have a serial mouse in a spare parts drawer then try it, if not then I would not suggest going out to buy one as they will be rare, expensive, and often junky. All versions of Windows I've seen will handle a serial mouse, and do it plug-n-play style. Plug it in, wiggle the mouse a bit to create traffic on the port, and then the OS detects it, installs drivers, and off you go.
There's likely many other things to try beyond what I listed but I'm already getting into some crazy stuff to try. Start with whatever bits you have on hand to isolate the problem, and if that doesn't work then try to keep the extra hardware spending to a minimum.
I realize this is an old question but I still see things like this come up once in a while.
