would it be right to generalize that the only source of Magnetic Field energy is, Non Conservative Electric Field?
Not in physicist's English, because that's not how "source" of energy is thought and spoken of.
"Source of energy" refers to a body or a region of space that already contains energy, part of which then moves into another body or a region of space which we study and which is the object X in statements like "energy of/in X increases/decreases".
What you're observing is that change of magnetic energy in a fixed region of space is necessarily associated with change of magnetic field strength somewhere in there, and thus, due to Maxwell's equation, to a curl of electric field there being non-zero. This does not mean that only non-conservative part of electric field is source of energy (because field is not a body or a region of space), but it also does not mean that only non-conservative field is responsible (doing the work) for the energy transformation from electric to magnetic form. It just means that whenever there is change of magnetic energy density, total electric field is non-conservative.
Of course, when electric field in a wire has both conservative and non-conservative component, it is the total electric field that does the work on the current, thus both conservative and non-conservative component participate in the energy transformation.
When you check directions and signs in inductor where current increases in time, the induced electric field of the inductor (the major part of non-conservative part of the total electric field) in the winding actually opposes the increasing current, thus some other force is responsible for the current increase and also for magnetic energy increase. This is usually (in lumped circuits) electrostatic force due to a capacitor or a battery or other voltage source, to which the inductor is connected. So more correct thing would be to say that the source of the magnetic energy is the electric energy in the capacitor or chemical energy in the battery, and when inductor's magnetic energy increases, it is the conservative electric force that does work against the induced electric force, and most of this work (assuming negligible losses) would transform into magnetic energy. In practice losses aren't negligible, so some part of the work transforms into internal energy of the winding (manifesting via increased temperature).