Great question! This answer is going to be heavily based in the discussion in chapter 7 of
Alkire, Ti, and Carol Rosen. Romance languages: A historical introduction. Cambridge University Press, 2010
The verbal tense system went through a reorganization in the passage from Latin to Romance. Of the original categories (roughly speaking pairs of mood and tense) only four survive with only regular sound changes (present indicative present, perfect indicative, imperfect indicative and present subjunctive), while other forms were repurposed (e.g. imperfect subjunctive) or replaced with completely new forms (e.g. indicative future, most of the perfectum family, the whole conditional mood).
This often happens when, after regular sound changes, the "old" forms cannot be distinguished anymore from other verbal forms (e.g. the indicative future cantābit becomes identical to the perfect cantāvit after the regular sound change turning intervocalic b into v, and so it had to go). This is exactly the case of the subjunctive imperfect.
The imperfect subjunctive of the verb canto has the form cantāret. After regular sound changes, in most of the persons the verb becomes indistinguishable from the infinitive cantāre. Indeed, except from Sardinian that has mantained it, the only reflex of Latin's imperfect subjunctive is in Portuguese inflected infinitive forms!
To fill in this vacuum, the speakers coopted two verbal forms: the pluperfect subjunctive and the pluperfect indicative. These tenses were in the process of being muscled out of their niche by the synthetic perfect (i.e. the perfect formed by an auxiliary verb followed by the past particle) and were ready to find a new home in the infectum system. We have early indications that this started out as a competition between the pluperfect and the imperfect when expressing potential future-in-the-past, as a reported speech equivalent of the future perfect cantā(ve)rit, thus introducing a perfectiveness distinction into reported speech. Here's an example from the first century BC:
sēque parātōs quaecumque imperāsset [instead of the usual imperāret], et libentī animō factūrōs pollicentur (De bello africano, 33)
they pledged themselves ready and willing to do whatever he commanded (lit. 'he would have had commanded'; direct speech quaecumque imperāverit faciēmus 'whatever is [lit. will have been] commanded')
Italian (as most of the other Romance languages) decided to use exclusively the perfect subjunctive for this job and went through a relatively aggressive push of regularization, eliminating the trace of the perfectum stem from all the verbs but three: essere, stare and dare (imperfect subjunctive fosse, stesse and desse). For example, today we say mettesse and not *misesse from mīsisset. This is similar to the situation for the indicative perfect (passato remoto) for which however more verbs retained their original strong stem.
As a slight digression I would like to say a couple of words about Spanish, since you name it in the question (however, I do not really speak Spanish so I'm going to trust my source for the examples). You say that Spanish has retained the Latin imperfect subjunctive, but that is not the case. It is true that Spanish has two forms, but the other comes from the Latin pluperfect indicative, not from the imperfect subjunctive. For example cantāret would give *cantare (that later presumably becomes *cantar) and not the current form cantara, coming from the pluperfect indicative cantārat. It's not 100% clear to me why Spanish retained both forms (as far as I know it is the only Romance language to do so), but that would probably be a question for Spanish.SE (or possibly Linguistics.SE), so I'll have to stop here.
congiuntivointo a tag synonym ofsubjunctive-mood?