The Grammar is Inconclusive
As you, and other answers, say, the grammar of the initial phrase is indeterminate. It could mean the caster needs to be within five feet of you when the spell is cast, when the save is made, or both.
The Feat is Inconsistent
One way to attempt to find RAI is to look at the other parts of the Feat as a cohesive whole. The saving throw effect is the third of three benefits. What do the others say? Well, the first says:
When a creature within 5 feet of you casts a spell, you can use your reaction to make a melee weapon attack against that creature.
This part specifically ties the benefit to the action of spell-casting within five feet. It is thus tempting to think that all of the benefits flow from that. After all, the feat is introduced as:
You have practiced techniques in melee combat against spellcasters,
However, this falls apart when one considers the second benefit,
When you damage a creature that is concentrating on a spell, that creature has disadvantage on the saving throw it makes to maintain its concentration
and realizes that it has nothing specifically to do with melee combat. As written, even ranged damage you do to the caster counts for this benefit. If the second benefit doesn't match the feat's introduction, we can hardly expect the first benefit to help us interpret the third.
The Lore is Unhelpful
Another potential source of RAI is to look at how the Feat worked in previous editions. For example, in 3.5 it was described as:
You have studied the ways and weaknesses of spellcasters and can time your attacks and defenses against them expertly...
Benefit: You gain a +1 bonus on Will saving throws. Spellcasters you threaten may not cast defensively (they automatically fail their Concentration checks to do so), but they are aware that they cannot cast defensively while being threatened by a character with this feat.
Here we see that while the spell-disruptive function clearly takes place in melee, the benefit to the save is not tied to caster proximity at any point. That is no help either.
A narrative approach (what I would do)
What is a Saving Throw?
The benefit of interest to you is advantage on a saving throw. A saving throw:
represents an attempt to resist a spell, a trap, a poison, a disease, or a similar threat...A saving throw can be modified by a situational bonus or penalty and can be affected by advantage and disadvantage, as determined by the DM.
Thus, the purpose of the third benefit of the feat is to make you better at your attempt to resist the power of the spell. What situational bonus would make you better at resisting a spell's power - threatening the caster when they cast the spell, or when you make your save?
Threatening the caster weakens the effect of the spell on you
In 5e, spellcasters invest power in the spell when casting, always. It makes good narrative sense for their spell to have less effect on you if you could threaten them when they were casting it.
Spellcasters sometimes continue to provide power or focus to a spell after it is cast - these types of spells require Concentration. Other spells with lasting effects require no further input from the caster - once cast successfully they are completely independent entities, and casters cannot even end them if desired. If the power of the Feat comes from weakening the spell by threatening the caster, it makes narrative sense (at least to me) that advantage to the save would be granted to any spell where the caster was within five feet of you when the spell was cast, always, and within five feet of you when you made the save, only if the spell required Concentration. If hitting (or even killing) the caster would not weaken the spell, having the caster within five feet of you when you made your save should not make a difference in your ability to resist it.
Note that some spells with delayed saves, such as web, do require Concentration, while others, such as grease, do not.
This is one possible narrative interpretation - that the spell itself is weaker because your proximity to the caster disrupts it. It is certainly not the only possibility. For example, "maybe it's the hatred that fuels you" suggests that it is not that the spell itself is weaker, but rather that your will to resist it is stronger when the caster is in proximity to you. Given that all the other approaches to determining the intent of the spell are inconclusive, I recommend the narrative approach - but a narrative approach that makes sense to you as a DM in terms of your assumptions about how magic works in your world.
