Either your headset volume control is defective, or the design of the headset volume control is defective.
The volume control options in the OS is software controlled (no wires, knobs, ect). The volume control on our headset should be hardware controlled, that is physical items that adjust the volume (knobs, resistors, potentiometers, etc) which is completely independent of your computer. Normally, turning the volume down on a hardware type device attenuates the signal to 0db, which is 0volts and therein 0 current. If you can still hear sound, then you volume control does not attenuate the signal like it should. I can't imagine a volume control designed to only attenuate from loud to soft, most go to off. Any thing other than this is a defect.
Try plugging your headset into the output of another device like a stereo receiver. And see if it exhibits the same behavior; it should.
Talk to your supplier; ask if the other headsets of the same make and model do this as well. If they do, buy a different one; if they do not, get a replacement.