The major difference is potential capability.
Holographic AIs are hardware limited. You don't want them to interact with something, you don't put any emitters around and isolate them from networks. You could make a power-mad AI and leave them in a box with a speaker, and they could rant away all they wanted and perhaps try and annoy you to death, but that's about it as long as you aren't an idiot and allow them voice access to another computer. As demonstrated in various episodes, you can alter their psychology, delete specific memories, fool around with skill sets by allowing them access or not to subroutines and information, basically treat them like computer programs. One starts getting big for its britches, turn off the computer and reboot them and they're back to their pleasant personality before they went insane.
The Soong-type androids however, are something different. By design, they've got at least the same physical capabilities as humans or others, usually more in terms of physical and mental abilities. They're not as easy contain, and, as shown in various episodes, it's not that easy to simply mentally alter them; despite Lore being a threat, Soong wasn't able to make him moral or erase the memory of what he'd done. One of the major issues in the episode Clues was that Data couldn't simply have the memory of the last 24 hours erased like everyone else, and that powering down and rebooting him didn't reset him.
So given an AI that is physically limited in its capabilities and can be easily controlled and prevented from becoming dangerous (despite the fact they keep making that stupid mistake, see M-5, Control, Moriarty, etc), versus one that by design is not limited in the same way, there's an obvious reason why people would be blasé about holographic AIs versus the androids. If the AI butler in the holodeck plans to kill you, it's trivial to deal with. If your android butler wants to kill you, you're screwed.