So why can I perpetually hold, say, a hot pan with oven mitts without
burning my hand?
Before attempting to answer this question, we need to be able to answer the question "how hot does something have to be to cause a skin burn injury?" The answer is it depends on the combination of the temperature of the surface of the skin that results from touching the object and the exposure time at that temperature as discussed below. These, in turn, depend on other factors.
The first figure below is from a landmark study of thermal injury. (Moritz, A.R, Henriques F.D. Jr. (1947), Studies of Thermal Injury: II. The Relative Importance of Time and Surface Temperatures in the Causation of Cutaneous Burns. American Journal of Pathology, 23, 695-720). The figure shows the thresholds of reversible and irreversible skin burns as a function of skin temperature and duration of exposure.
In this study it is important to emphasize that the temperatures in the figure represents the actual temperature of the surface of the skin, and not the temperature of an object prior to contacting the skin. The two will not necessarily be the same since the skin will generally act as a thermal load reducing the surface temperature of the object upon contact. The main factors that influence the skin temperature are the pre contact temperature of the object, its thermal inertia (especially thermal conductivity), and its heat capacity.
TEMPERATURE:
Since the temperature of the skin will generally be lower than the pre-touch temperature of the object it contacts, we can say that, for particular burn threshold, the pre-touch temperature of the object to cause a burn can be no less than the threshold skin temperature. For example, the figure below shows that the threshold skin temperature for a skin burn for a 1 second contact time is roughly between 65$^0$C and 75$^0$C depending on the burn severity. Consequently, in order for an object to cause a burn the temperature of the object can not be less than these temperatures. Basically this gives us the minimum temperature the object has to have for a potential to cause a burn. But a minimum temperature is a necessary but not sufficient condition to cause a burn. It also depends on the next two factors, the first being the thermal inertia (thermal conductivity) of the material.
THERMAL INERTIA (THERMAL CONDUCTIVITY:
The greater the thermal inertia of the material the greater its burn potential for a given pre-contact temperature. (A closer look at the individual components of thermal inertia shows that, for practical purposes, thermal conductivity is probably the main driving factor.) To look at the effect of thermal inertia, refer to the second figure below which is based on the work of Alice Stoll et al. (Stoll, Alice M., Piergallini John R., Chianta, Maria A. Thermal Conduction Effects in Human Skin:I, II, and III. Report Nos. NADC-79033-60, NADC-79034-60 and NADC-79036-60 15 January 1979).
The figure plots material temperature vs the inverse of the square root of the thermal inertia of the material. Clearly the two metals, aluminum and steel, which have considerably higher thermal inertia than the plastics polycarbonate and polystyrene, will cause pain and blister at much lower temperatures.The 2.5 second burn threshold for aluminum (from the data behind the graph)is between 60$^0$C and 63$^0$C. Note that this is very close to the skin temperature to cause blister at 2.5 seconds in the first figure. Compare this to the 2.5 second threshold pre contact temperature for polycarbonate plastic, which is about 110 $^0$C and polystyrene plastic, which is about 120 $^0$C.
HEAT CAPACITY:
Finally, even if the object has the minimum pre-touch temperature and thermal inertia to cause a burn, it still may not have the potential to actually cause a burn because of limited heat capacity.
Consider aluminum foil. Aluminum has a high thermal inertia (high thermal conductivity). Yet you can take s sheet of aluminum foil directly out of an oven at 350 $^0$F (177$^0$C) and not even experience pain. It is because the foil is so thin, there is simply not enough thermal energy available to burn the skin. An aluminum oven rack at that temperature would result in an instantaneous burn (See the second figure below).
One final point. It is not only the thermal properties of what is being touched that counts, but also the thermal properties of the skin which depend primarily on the thickness of the epidermis. For example, the back of a finger is more susceptible to burn than the finger pad, due to thinner skin. The second figure involves minimum epidermal thickness.
So now we return to your original question
So why can I perpetually hold, say, a hot pan with oven mitts without
burning my hand?
Perhaps you can, and perhaps you can't. In view of the above, it depends on many factors including but not necessarily limited to (1) the temperature of the object, (2) the thermal properties (thermal inertia and heat capacity) of the object and of the oven mitt (material + trapped air) and (3) the thermal properties of the skin in contact with the mitt . Another factor is blood circulation below the skin (good blood circulation takes heat away from the skin faster).
At a minimum the use of a mitt will increase the time you can hold the object than if you did not use one.
Hope this helps.