As noted in a comment, using a sketch that you yourself call a “rough diagram” is leading you astray. However, even with an accurate diagram, the question remains valid: why are the centres of the two circles collinear with the point where they touch?
When we say that two circles “touch externally”, we mean that they have a common tangent line at that point, with their centres on opposite sides of this line. A radius of a circle is always perpendicular to the tangent line it meets on the circumference. (Euclid III.18)
![A diagram of the problem. The rectangle, large quarter circle, and small full circle are shown with solid lines. Their shared tangent line, and several radiuses, are shown with dashed lines.](https://cdn.statically.io/img/i.sstatic.net/7cze1.png)
Call the small circle’s centre $O$, and the touch point $T$. The two circles’ radiuses $BT$ and $OT$ are both perpendicular to a third line, their common tangent, so $BT \parallel OT$. And they share point $T$, so they are collinear. ∎