The problem can be defeated by Geometric concepts. As you indicated, it looks like the desired locus is simply all points on the perpendicular bisector of the two points. The following analysis verifies this.
A circle will pass through both points if and only if the circle has a center that is equidistant between the two points. This is because the circle will have a center with a radius, and the distance of each of the two points to the center of the circle must equal the radius. Therefore, the center of the circle must be equidistant between the two points.
So, the problem reduces to showing that any given 3rd point will be equidistant between the the two points if and only if the 3rd point is on the perpendicular bisector.
If the point is on the perpendicular bisector, you can form two triangles. Each triangle includes two of the pertinent vertices, with the 3rd vertex being the midpoint of the line segment connecting the first two points. It is easy to see that the two triangles are congruent. Therefore, any point on the perpendicular bisector must be equidistant from the two points.
For the analysis in the opposite direction, assume that a point is equidistant between the two points. Form two triangles, where each of the two triangles uses two of the pertinent vertices, and the 3rd vertex is the midpoint of the line segment connecting the first two points. Then, the two triangles will have a common side, and will also have sides that are equal because each side is (1/2) the length of the line segment between the two points.
Therefore, by side-side-side analysis, the corresponding two triangles are congruent. This implies that all of the angles in the two triangles are equal. This implies that each of the two triangles must be a right triangle. This implies that the center of the circle must be on the perpendicular bisector of the two points.