Fluorine, though higher than chlorine in the periodic table, has a very small atomic sizehas a very small atomic size. This makes the fluoride anion so formed unstable (highly reactive) due to a very high charge/mass ratio. Also, fluorine has no d electrons-orbitals, which limits its atomic size. As a result, fluorine has an electron affinity less than that of chlorine.
See this.