I have what amounts to an educated guess about how the spelling and pronunciation diverged in the way that they did, but I cannot really prove any of it.
First, how it came to be pronounced with a sound like "kay" at the beginning. It's not the only time a Klingon word was mangled in this way. In the TNG episode "Birthright", there is a song which has the refrain yIja'Qo'. The Q is a voiceless uvular affricate, which many English-speakers don't know how to pronounce. The pronunciation guide in the script "helpfully" contained the hint "yi-ja-KKHO". If you watch the episode, the way the actors actually pronounce the last syllable of this phrase is "kay-ho". That is, they saw "KKHO" in the script, had no idea how to pronounce "K", and just said "kay". It is not unreasonable to surmise that the shooting script for "The Savage Curtain" had a pronunciation guide which said something like "K'a-less", which got mangled into "Kay-less".
[Edit: Note: The phrase yIja'Qo' comes from Marc Okrand. The name Kahless was invented before Okrand got involved with the Klingon language, and I'm not implying Okrand had anything to do with the way the name is pronounced. I'm just pointing out another instance in the production of Star Trek where (part of) an alien word beginning in "k" was pronounced with a "kay" sound which wasn't originally intended, as an illustration of the process through which this can happen.]
Second, as for why such a name was invented, I'll note that the triconsonantal root kh-l-s appears in a number of foreign names which would've been known to well-read or well-traveled English speakers of a certain generation. It exists in the Arabic boy's name "Khales", for example, which is similar enough to "Kahless". The community of baptised Sikhs is known as the "Khalsa". (Gene Roddenberry appeared to have had a Sikh war buddy who was the inspiration behind the name of Khan Noonien Singh.) The kh-l-s root exists in other "alien" languages invented by English authors, the "Khaleesi" of Game of Thrones being the most obvious one besides Kahless. "Khaless" is also a word in the Drow language of Dungeons & Dragons. In other words, "Khaless" would be a suitably foreign and yet plausible sounding name to an English-speaker.
Third, as to why it's "Kahless" and not "Khaless", English speakers misplace the "h" when it marks a non-English consonant in foreign loanwords all the time. A famous example of this is the name of the Lebanese-American writer, Kahlil Gibran. His name should've been written "Khalil", but it was written as "Kahlil" by mistake, and the mistake stuck and became his name. Mistyping "Kahless" for "Khaless" would be a very common mistake if the name was copied by hand.
There's probably no way to prove any of the above, since no one probably thought it was important to preserve any of the relevant evidence. Someone made up a foreign name for a TV show. Someone else may have made a mistake while copying it. Some actor may then have pronounced it in an unintended way. The result is suitably alien, and the spelling even suggests that the name isn't really pronounced by natives the way it is written down in English.
The only way to prove the above guess would be to find an original shooting script for "The Savage Curtain", track down the people who worked on that episode (if they're even still alive), and ask them about it (if they even remember it at this time, which is doubtful).
