Here is an excerpt from Kiselev's Geometry; Book I: Planimetry (English Adaptation) regarding tangents through a point:
Problem. Through a given point, to draw a tangent to a given circle.
Consider two cases:
(1) The given point ($C$) lies on the circle itself. Then draw the radius to this point, and at its endpoint $C$, erect the perpendicular $AB$ to this radius [...].
(2) The given point ($A$) lies outside the disk bounded by the given circle. Then, connecting $A$ with the center $O$, construct the circle with $AO$ as a diameter. Through the points $B$ and $B'$ at which this circle intersects the given one, draw the lines $AB$ and $AB'$. These lines are the required tangents, since the angles $OBA$ and $OB'A$ are right (as inscribed intercepting a diameter).
Note that the solution presented above does require a compass to draw a circle (i.e. the circle with diameter $AO$).
Here is an excerpt pertaining to the construction in (1)
of the above excerpt:
Problem. To construct a right triangle given its hypotenuse $a$ and a leg $b$.
On a line $MN$, mark $AB = a$ and describe a semicircle with $AB$ as a diameter. (For this, bisect $AB$, and take the midpoint for the center of the semicircle and $\frac{1}{2}AB$ for the radius.) Then draw an arc of radius congruent to $b$ centered at point $A$ (or $B$). Connect the intersection point $C$ of the arc and the semicircle, with the endpoints of the diameter $AB$. The required triangle is $ABC$, since the angle $C$ is right, $a$ is the hypotenuse, and $b$ is a leg.
There is also another way of constructing right angles, which is more primitive that the excerpt above.
However, as mentioned already, these constructions require a compass in addition to a straightedge.
I'm not aware of a solution to your problem with solely a straightedge.
But, remember what a compass serves to do: find the locus of points equidistant from some point (i.e. draw a circle). Can we not do the same with a straightedge? Yes, and no. You can only find a finite amount with a straightedge, so it won't really be a locus. The compass merely draws every point with just a swipe of the wrist.
Let us "emulate" a compass with a straightedge:
Answer
Pick a point $A$ on your straightedge. Pick another point $B$ on your straightedge (such that $AB$ will be the radius of your desired circle). Put your pencil tip on this $B$ and start rotating the straightedge about $A$.
My answer is more of a hack, rather than a geometric proof. If your life depended on finding a tangent to a circle going through some point, and you only had a straightedge, I recommend doing what I said above. However, don't torture yourself by not having a compass handy if you don't need to. :)