The ultimate way to verify an answer is to follow its steps and redo its calculations etc. In principle this can be done with any answer given here (and the advantage of math over other topics is maybe that - depending on what you have trained - it may be easier to perform such a confirmation without any external resources; even the notion of "right answer" may me less well defined in other topics [Just ask questions like: Do gun control laws lead to more or less crime and please back your answer with statistics?]).
Example:
Q: To show the infinitude of primes, Euklid considers the number $2\cdot 3\cdot \ldots \cdot p_n+1$. Is this really always a new prime?
A: No, all we can asy is that it is divisible by a new prime. In fact $2\cdot3\cdot 5\cdot 7\cdot 11\cdot 13+1=30031=59\cdot 509$ is not prime.
The correctness of the counterexample in the answer can be verified by everyone with pen and paper, actually the OP might have found it if he had not given up after $2\cdot3\cdot5\cdot7+1=211$.
Of course, all the answerers are only human and so an incorrect answer like
A: No, all we can asy is that it is divisible by a new prime. In fact $2\cdot3\cdot 5\cdot 7\cdot 11+1=3211=13\cdot 19$ is not prime.
might have slipped through. In fact, especially experts might even quickly upvote such an answer because thay know that some small counterexampe like this exists! Even in the presence of so many experts erring, the OP might do the calculations by hand and spot the error (which via comments and edits will lead to the correct answer at last).
Of course, answers will not always be as self-contained. Or the answers may lead to subjects the asker has no idea about (such as the functional equation $f(x+y)=f(x)+f(y)$ leads to discussions about $\mathbb R$ being a $\mathbb Q$ vector space and the Axiom of Choice). Or references to mathematical publications may be given and prevent the average reader from such a direct verification. One has to rely on a) the self-control mechanism of peer review in mathematical publication, b) the fact that the answer quoted really addresses the problem at hand c) it meets the quality standards.
Example:
Q: Playing with my atlas, I noticed that I could colour all maps using three or four colours, though sometimes it was a bit tricky. Are there maps that require five colours?
A1: This is the famous four-colour-problem, known alrady in the 19th century. The first proof was given by Alfred B. Kempe, “On the geographical problem of the four colours”, Amer. J. Math. 2:3
(1879), 193–200. An alternative proof is by Peter Guthrie Tait (1880).
A2: This is the famous four-colour-problem, it was ultimately proved by Appel and Haken in 1989. This proof has become famous because it was the first that involved massive computer calculations to deal with tons of special cases, but no human being can really comprehend it ...
A3: I just found a five page paper "Proof of the Four-Colour-Theorem without extensive computer calculations" on arXiv by an author named Craig Pott ...
Verification of answers of this kind is certainly not trivial for the average asker, but I am confident that the voting system will quickly sort things out in such a case.
All in all one could say: There certainly are checks and balances, and the readers themselves play a great part in it.
Nevertheless it may still happen that an answer that is in fact wrong gets upvoted and/or accepted.
