What are the castling & e.p. conventions, and how are they used?

Question

Castling & en passant are often called "conditional moves". In the chess problem world, we frequently hear about the castling & e.p. conventions. What are these exactly, and how are they used? Particularly what does one do if there are multiple conditional moves in the same position.

Please start off with the basics, and maybe take it as far as explaining the difference between Partial Retrograde Analysis (PRA) and Retro Strategy (RS), two rather intimidating terms that I understand are important for handling multiple conditional moves.

That seems like the right scope.

And please put in as many real examples as possible!

Answer 1

The history of some chess positions can be determined with certainty and some cannot. PRA looks at each alternative history and finds their different solutions. These typically revolve around if the right to castle exist, then you have some solution; or if last move allows en-passant then another solution exists.

RS involves 'mutual dependency', for example if a pieces existence on a square can mean either one side can't castle or the other side can't castle; Then the first to castle proves the other couldn't.

There is also the AP (a posteriori) convention, which is controversial as the variations can involve moves motivated more toward proving if certain moves are legal than by real chess motivations.

Tim Krabbé's book Chess Curiosities covers these things.

Answer 2

Here is a generalization attempt, since the difference between the two isn't limited to chess, let even castling/e.p.!

To let pRA/RS rear their pretty head, we only need a) a game, b) a goal set by a composer, c) info about the game state, d) this info is incomplete. As a non-chess example, think of a Go position given to you, you shall win...but where was the last Ko?

We now have some Schrödinger cat state. Where and how do we fetch the missing information? Two possibilities:

a) pRA - We don't. We only know that we are able to (finitely) list all states of our Schrödinger cat, and by convention we know that in all states we can reach the goal. The solution is then listing all goal completions. Think of it as Everett Many-World - the wavefunction never collapses.

b) RS - We force. We make a measurement instead. E.g. by castling, the wavefunction collapses to the state where castling was possible. And we can make it collapse only by measurement, so this is "needed" to "prove" castling was possible at all.

Thus, the difference between pRA and RS is exactly the difference between Everett and Copenhagen interpretation of quantum mechanics!

Example from QM to underline my logic: Your goal is to produced some state |j>+|-j>. You are given some pure |J> on the Bloch sphere, alas, the direction isn't given. You have a Veeblefetzer gate (pat. pend.) turning any |J> into |J>+|-J>.

pRA: Implement Veeblefetzer by Hadamard+Pauli-X. Goal reached. But you still don't know |J>...but you don't need to either!

RS: Make a measurement projecting out some preferred |j> (say, |z> is Castling) which you now know. Apply a Hadamard gate. Done.

Answer 3

Castling & En Passant Conventions

Here is a simple chess problem:

[Title "#2 b) +wPh2 - Pieter ten Cate Die Schwalbe 1970"]
[FEN "r3k3/B5Q1/8/8/8/7p/8/4K2R w - - 0 1"]

a) 1. Rxh3! threat 2. Rh8# 1. ... O-O-O 2. Rc3#

1. O-O? h2+!

b) [with white pawn added on h2] 1. O-O threat 2. Rf8# 1. ... O-O-O 2. Rc1#

1. Rxh3? (blocked)

This fun miniature with no retro content relies on both players being able to castle. And how do we know that neither player's king or rook has moved? Well we don't, but as long as there exists the theoretical possibility that the rights remain, we can assume that either side may castle.

This is the castling convention, created to stop pesky legality issues from potentially interfering with problem logic , as in the above example.

Even if a problem doesn't rely on castling rights for soundness, it's still good for the solver to always know whether they should consider castling in their solving.

The castling convention is not inherently "retro", since the check that castling rights may remain is usually trivial (and too tedious to put into words). However, this convention (together with a similar one for e.p.), when combined with exact retro logic, still appears in hundreds of retro problems.

And there is an e.p. convention too, for example

[Title "#2 Sergii I. Tkachenko Die Schwalbe 1995"]
[FEN "8/K2p4/n1pBp1n1/PpPkPp2/1p1N4/1N3P2/B1P1Q3/8 w - - 0 1"]

Thematic try: 1. Sxe6? but: 1. ... f4!

Key: 1. Sxc6! Kxc6/f4/Sxc5/Sxe5/dxc6 2. Sd4#/De4#/Sxb4#/Se7#/Dd3#

Can White play 1. axb6ep, cxb6ep or exf6ep at the beginning? None of these turn out to be keys, but since we are up in the air as to whether they are legal, the e.p. convention clarifies, that no we cannot play any of these moves.

Conventions in the Codex

These conventions aren't FIDE Laws, but they are codified: you will find them in the Codex for Chess Problem Composition.

Location	Subject	Content
Art 16.1	Castling convention	Castling is permitted unless it can be proved that it is not permissible.
Art 16.2	En-passant convention	An en-passant capture on the first move is permitted only if it can be proved that the last move was the double step of the pawn which is to be captured.

One important subtlety is that castling doesn't become legal under this convention, it becomes permitted. That means that we cannot simply use the convention statically. For example, if the stipulation asks "what is the last move?", we can't use the castling convention to imply that it wasn't the king or rook that shifted.

Optimism vs pessimism

The castling convention is optimistic: it says that if histories exist for either case, we can definitely castle. The en passant convention is pessimistic: if histories exist for either case, we definitely can't capture e.p.

How these generalize

In principle, these optimistic/pessimistic conventions can generalize to many kinds of fairy chess: e.g. fuddled men, shrinking men, pocket pieces, etc. They don't apply to retractors (where you can retract anything that may be legal) or proof games (where you must provide a provably unique history for the position).

Resolving multiple conditional moves

What happens if there are multiple conditional moves on the board? There are two main protocols for resolving this situation: RS (Retro Strategy) & PRA (Partial Retro-Analysis). A loose analogy might be to quantum theory: RS is analogous to the Copenhagen Interpretation, while PRA is like the Multiple Worlds Hypothesis.

RS

RS says the person to move can make any optimistic move. Then the set of possible histories is reduced to those in which the optimistic move (typically castling) was legal. Normally, pessimistic moves can never be made under RS.

Example (1) of RS

An example is that if White & Black castling are mutually exclusive due to some complicated dependencies in the position, then whichever player castles first will prevent the other from ever castling themselves. This common special case of RS is called "mutual exclusion" (or “mutex”).

Example (2) of RS

Suppose that we know that Black’s last move was a double hop by c pawn or e pawn, but we don’t know which. So we may know that White can definitely capture ep with wPd5. But under RS White can make neither of these captures, even if White has no other legal moves. So RS is not a helpful paradigm if you want to progress this position. On the other hand it is an interesting corner-case: a position which is not stalemate but there are no playable moves.

PRA

PRA says we divide the problem into sub-problems (twins) according to the different interpretations of the conditional moves. So if there are three conditional moves, either of which might be allowed or disallowed, then we have 2^3 = 8 possible interpretations. However we discount (1) those which are an illegal combination (2) those where we could be more optimistic or pessimistic as appropriate. For any twin, all the legal rights are sorted out once and for all, before a move is played.

Example (1) of PRA

Suppose we have a position where two castlings C1 & C2 are in doubt, and we know that they cannot be both legal. So we start with 4 possible parts: {C1,C2}, {C1}, {C2}, {} where each bracket denotes a set of permitted moves. We can remove {C1,C2} from consideration because it's not a legal combination. We can also remove {} because by optimistically applying the castling convention, this could be improved to {C1} (or equivalently to {C2}). So the problem has two parts {C1} & {C2}.

Example (2) of PRA

Suppose we have a position where castling, C, and e.p., EP, are in question, and if e.p. is not permitted, the castling cannot be permitted either. Then the 4 candidates are {C, EP}, {C}, {EP} & {}. We are told that {C} is impossible, so discount it. But consider now {EP}. We could improve that to {C, EP} by the castling convention, or to {} by applying the e.p. convention. Thus we can discount {EP} too. So we are left with two parts: {C, EP} & {}.

Which applies, RS or PRA?

These days, PRA is taken to be the default, and if that doesn’t work, then RS is assumed. Yes the judgement whether PRA works out for a problem might be difficult to formalize for a computer program, but that is currently the case.

This idea that PRA is the default over RS I describe as the “meta-meta-convention” (MMC) which in truth it is.

This principle dates from 2007 and replaced an earlier idea that RS was always the default, while PRA had to be mentioned by name. So if you come across an older problem, it may be RS without directly saying.