Examples where I expect I don't need a passport:
Examples where I'm not so sure:
Example where I expect you do need a passport:
Is there any general rule of thumb I can apply, or do I have to look up each case separately?
For the British islands (Isle of Man, Jersey, Guernsey, etc), you are supposed to carry ID but it is not usually checked. There is no formal border control when arriving from or going to the UK. It's like a domestic flight. I usually bring a driving licence in my wallet in case anyone asks, but I've never been asked for it.
Gibraltar has formal entrance controls; a passport or EU ID card is required. Arriving by land you would need such credentials to exit Spain; arriving by air you can get mixed up with passengers from Morocco; there's no segregation. Note that civilian flights to Gibraltar are often diverted to Málaga, so you might need appropriate credentials to enter Spain: Spain refuses to allow the use of its airspace for flights to/from Gibraltar, and the prevailing winds means the ideal landing involves flying over Spanish territory. So instead the landing involves holding steady against a sudden blast of cross wind as you come out of the wind shadow of the Rock, and then a few seconds later executing a very sharp 90 degree turn as you align with the runway. If you miss the turn you won't be aligned with the runway and you'll have to reject the landing and go around, which involves circling the Rock again. It's quite challenging, most pilots will give it two attempts before flying to Málaga instead.
Falkland Islands and other overseas territories are similar, but don't accept ID cards.
