definition of voltage as work done moving unit of electric charge between two points in electric field
That's almost okay, but what's missing is the concept of electrical potential. And technically, you should say "work per unit charge" to get the units correct (volt = joule per coulomb). That's often not discussed in a lower level (high school) first course, but is the basis for voltage. Electrical potential is defined for a point in space and is the potential energy per unit charge added to a system of charge(s) by adding a "test" charge at the point in space (bringing it in from infinitely far away).
Imagine a space with only $5$ nC charge. The is no force on the charge and no potential energy defined. But there is an electric field everywhere. If we add a $1$ C charge, starting it infinitely far away, and move it to a distance $3$ m from the $5$ nC charge, the E-field of the $5$ nC charge does $-15$ J of work on the test charge. Therefore, the potential energy of the system changes from zero to 15 J (by definition of potential energy).
If I calculate the change in potential energy per charge added I get
$$\phi=\frac{15~\mathrm{J}}{1~\mathrm{C}} = 15~\mathrm{V}.$$ A volt is a joule per coulomb. That's the potential at the point $3$ m from the $5$ nC charge *due to the * $5$ nC charge only. If I take my test charge, again starting at infinity, and move it to $5$ m, I will find that the potential energy added is $9.0$ J, so the potential at that point is $9.0$ V.
Now voltage is the difference in potential between two points in a system, $$V=\Delta\phi$$so voltage ends up being the difference in [work per unit charge done by the electric field] in moving a test charge from [infinity to point A] and [infinity to point B], which is also the net work per unit charge done by the field while moving the test from B to A. The voltage between $3$ m and $5$ m from the $5$ nC charge is $6.0$ V with $3$ m being the higher potential point. The voltage across that gap is $6.0$ V.
An electrical circuit is a complex set of charges and materials which affect how space responds to those charges, but the net result is an approximation model known as Ohm's Law. As @BobD details, the voltage, V, in Ohm's Law is a difference in potential between two points in the circuit, usually across some resistor or set of resistors.
Most importantly, even if the reference potential ($V_D$) is not zero, the voltages across the resistors will not change, and the current in the circuit will not change.