I read a book in the late 80s about a man travelling from one dimension to another through portals that are "right around the corner". He ends up in a place where mind control is prevalent and people wear a number on their clothes based on their level of ability (like 1 through 10 or 12). He is woefully out of place and shielded by one of these people, but he is dominated by a child who gives him nightmares when his protector is asleep. This is all I can remember.