OK. Craster lives in his Keep. His daughters/wives are just servants and cannot help with defending against hypothetical attack. This leaves Craster as the only element able to defend. Let's leave aside he looks quite old and fat (Barristan Selmy also looks old) and let's suppose he can swing with his axe really well. Still, he is alone.
If I was in his position and group of Night Watch members visited me (armed, some skilled, although tired and wounded), I would be very far from mocking them, threatening them, even attacking them. I would be clear that in case of fight, I would definitely lose.
Still, he always behaves like he is invincible and no one could (would dare to?) harm him. Why? Is he simply stupid? Or is there anything I missed? Or anything omitted in the series?
Many times I considered parts of Game of Thrones weird or illogical, but after reading Wiki or answers here, I had to admit I just missed something. This is kind of last piece I couldn't find anything about.
I've seen the series, I haven't read any of the books. If the answer is in the books, I'd be happy to hear it.