You could, perhaps, use an SSRSSR (Solid State Relay), but it would need a drive circuit that detects battery voltage dropping below a threshold and switching off the control LED.
The one I've linked to needs less than 1mA to stay off. However that will cut into battery life. It's a changeover (Form C) type.
Analog switches are possible too, but they usually need some voltage to turn on at all.
All in all, I think a mechanical switch would be your best bet. Quick and easy to flip and generally reliable.