I'd go for the jugular in non-empathetics: sociopaths and psychopaths, people who clinically lack the ability to empathize. There is a difference in their brains.
You might be interested in the story of the neuroscientist Dr. James Fallon..
Fallon wanted to identify lack of empathy as a brain patern. He had PET scans of serial killers mixed with a host of other mental illnesses and disorders in an attempt to predict accurately psychopaths. He found a particular pattern again and again which was striking.
“I was looking at many scans, scans of murderers mixed in with schizophrenics, depressives and other, normal brains,” he says. “Out of serendipity, I was also doing a study on Alzheimer’s and as part of that, had brain scans from me and everyone in my family right on my desk.”
“I got to the bottom of the stack, and saw this scan that was obviously pathological,” he says, noting that it showed low activity in certain areas of the frontal and temporal lobes linked to empathy, morality and self-control. Knowing that it belonged to a member of his family, Fallon checked his lab’s PET machine for an error (it was working perfectly fine) and then decided he simply had to break the blinding that prevented him from knowing whose brain was pictured. When he looked up the code, he was greeted by an unsettling revelation: the psychopathic brain pictured in the scan was his own.
Perhaps because boldness and disinhibition are noted psychopathic tendencies, Fallon didn't hide that discovery; instead he started to study it. He found that his family line included seven alleged murderers, including Lizzie Borden*. A series of genetic tests revealed the presence of high-risk alleles for aggression, violence and low empathy.
You can see normal empathy vs. psychopath scan differences in the linked source.
*Maybe you remember this childhood rhyme (I do): Lizzie Borden took an axe/ gave her mother forty whacks./ When she saw what she had done,/ She gave her father forty-one.
