First of all, kudos to you for correctly identifying that what you need to do is find the value of $x$ that makes the denominator equal to 0. It appears to me that that this problem is designed to test the SAT taker on two components - the first being the ability to identify that $h(x)$ is undefined if and only if the denominator is equal to 0, and the second being the ability to solve algebraically for which $x$ value will make the denominator equal to 0.
All of the other correct answers certainly sufficient, but I am going to provide my method of solving as well. It is semantically no different from the other correct solutions, just different syntax, but I imagine you are in high school and when I was in high school some methods of solving equations did not click well for me and others did, so I will provide mine just in case it is the one that works for you.
$$(x-5)^2 + 4(x-5) + 4 = 0\\
\implies x^2 - 10x + 25 + 4x - 20 + 4 = 0\\
\implies x^2 - 6x + 9 = 0.$$
From here, someone with a good eye would catch that $$x^2 - 6x + 9 = (x-3)^2,$$
thus the unique solution to our problem is $x = 3$. However if a student is not as skilled in spotting factorizations, it would be just as easy to recall the quadratic equation:
$$ax^2 + bx + c = 0 \iff x = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a}, \hspace{4mm} \text{(for } a \text{ not = 0)}$$
and use this to arrive at
$$x = \frac{6 \pm \sqrt{6^2 - 4(1)(9)}}{2} = \frac{6 \pm 0}{2} = 3.$$
This answer may seem too long for the SAT where speed matters - but that is only because I was verbose in explanation. When I initially used this method to solve your problem on paper by hand it probably took about 15 seconds.
Hope you find the answer that works for you here on MSE and good luck on your SAT!