I must somewhat disagree with Ignacio Vazquez-Abrams. Since no DAC is mentioned in the question, I'll assume that there isn't any.
In that case, backward compatibility of DVI is used (there's the DVI-A mode). Some (I'd say most) DVI connectors can send analog signal through the cable to work with analog receivers. Then the signal goes through the DVI->VGA converter that is just a mechanical converter which provides a VGA plug at the end of the line and the signal is actually analog the whole way.
In that case, I'd say that the difference would mostly be in the type of particular cable used and in forward compatibility needed at the end of the cable.
The DVI cable itself may be better than a VGA cable, but there's no guarantee of that. On the other hand, if you plan to switch to digital later on, than DVI may be better option.
Also be sure to check if the source can provide analog signal through the DVI port. If it doesn't, then you'd need an active converter, which can cost a lot more than passive, but you'd have digital signal up to the point at which the DVI cable ends.