I first started the pedestrian way and compared the two possible rotations:
def nearest_signed(old, new):
angles = ((new - old)%360, (new - old)%360 - 360)
return min(angles, key=abs)
We check the modulo-360 angle, and its complement in the other direction.
It seems to work:
>>> nearest_signed(0, 359)
-1
>>> nearest_signed(359, 0)
1
>>> nearest_signed(180, 2)
-178
>>> nearest_signed(2, 180)
178
Now, I wanted to see how this behaves, so I made a plot of every angle combination:
import numpy as np
matplotlib.pyplot as plt
news,olds = np.ogrid[:360, :360]
rights = (news - olds) % 360
lefts = rights - 360
take_rights = abs(rights) < abs(lefts)
nearest_signed = np.where(take_rights, rights, lefts)
fig,ax = plt.subplots()
cf = ax.contourf(news.ravel(), olds.ravel(), nearest_signed, cmap='viridis', levels=np.linspace(-180, 180, 100), vmin=-180, vmax=180)
ax.set_xlabel('new angle')
ax.set_ylabel('old angle')
cb = plt.colorbar(cf, boundaries=(-180, 180))
plt.show()
![pretty plotted result](https://cdn.statically.io/img/i.sstatic.net/uL9it.png)
Now this makes it obvious that a simple modulo of the difference in angles should work. And sure enough:
>>> np.array_equal((news - olds + 180) % 360 - 180, nearest_signed)
True
Meaning that the formula you are looking for is
(new - old + 180) % 360 - 180
give or take a sign difference in your convention. If you count rotation signs the other way around, just switch the two angles:
(old - new + 180) % 360 - 180
(new - old) % 360
?