I can somewhat relate, when I found my stray cat I didn't immediately get attached to it. It took months and it was hard to not lash out when he peed in the kitchen sink.
What you need to understand is that right now the cat associates you with violence. When you approach it it has no way of knowing if you are going to hurt it again or not. You need to gain it trust by always petting it and being kind with it. If it behaves badly, does things you don't approve of - don't hit it, just take it away to some other room/whatever to stop the behavior. When you approach it always be kind. It would take a while, but it's possible.
And if the cat is distant - you need to give it some time. You may cheat my bringing treats and toys, but don't force your presence. The cat is in a very dependent position - it can't truly escape your company if you force it - it can't leave the house/appartment after all. So be polite and respect it if the cat wants to be alone. Give it freedom to do as it pleases - unless it does damage to the house - and the cat would slowly come out of its shell.
Also, realise that cats display affection differently from dogs - the small gestures mean much for with them. The dog would wag its tail and bark, but a cat would simply prod you with its head and sleep in the same room with you - its the way they display their sympathy.