ChaCha has clear delineations between key, nonce, counter and constants.
What is the reason for not using a XEX-like ($k=0$) approach such that the ChaCha key is 512 bits and all the other things are XOR'ed with the key, and only the key is XOR'ed with the ChaCha permutation output?
ChaCha already has some similarity to Even-Mansour with the way it is constructed, why not go all the way? Currently, it only does partial single key Even-Mansour with partially known key, known plaintext and partially exposed ciphertext.[1]
Attacker doesn't gain any additional control over the input he did not have before, and the security of the cipher when the attacker is passive becomes $2^{512}$.
Are there any downsides?
[1] If ChaCha is an Even-Mansour-like construction then:
- half of the key is effectively 0
- constants, counter, nonce are all known, rest of the plaintext is 0