Your voltage error doesn't matter.
You're calculating your 5V winding based on the actual number of turns, 17T, so there is no error you have to factor in. Or rather, you already did factor it in.
As for the rest, what you're looking for is your volts per turn (\$ V_{pt} \$) ratio based on the winding you intend to use as your regulation voltage. I'm assuming it is the 5V winding.
So just calculate it. The error doesn't matter. You know you'll get 5.5V if you had 5.19 turns, so you use those numbers. Plugging it into the equation, \$ V_{pt} = \frac{V_{o} + V_{d}}{N_{s}} \$, you get \$ \frac{5V + 0.5V}{5.19} \approx 1.06\$.
This value is not going to change just because you have 5 turns instead of 5.19 because the voltage on that winding will also be proportionally less as well. The ratio remains the same.
In fact, you can use the voltage per turn ratio to find the actual voltage on your 5V winding, and any other windings. It's the number of turns multiplied by the volts per turn.
Using our 1.06V per turn, we know that we'll need \$ \frac{12V + 0.9V}{1.06} = 12.17 \$ turns and \$ \frac{24V + 0.9V}{1.06} = 23.49 \$ turns respectively.
That is to get the chosen voltages, and now it is up to your discretion how to deal with the error. Integer turns means 12 and 23 would be the obvious choices. The error on the turns here is calculated exactly like we calculated the turns error earlier. Or even easier, simply multiply the chosen integer number of turns by our volts per turn constant. See how handy it is? \$ 12 * 1.06 = 12.72V \$ and \$ 23 * 1.06 = 24.38V \$. Or try 13 and 24 turns, or any other number of turns for that matter, and just multiply by the voltage per turns - that constant includes our previous sources of error so you don't have to worry about it anymore.