TL;DR: the values that you may find online will most likely only be accurate up to a few percents. I would strongly encourage the use of a reference solution of known concentration that you can further use for calibration on you own hardware (cuvette and spectrophotometer).
Your question is no trivial one. Indeed, absorption spectra may be influenced by a variety of factors, a phenomenon referred to as chromism. Among these factors are temperature, pressure (mechanical forces), solvent (including the use of heavy water instead of water, as mentioned in Ed V's answer), for instance. As an example, see below the absorbance spectra of a given molecule (termed Conmpound 1 in [1]) in different solvents.
Additionally, even when measuring the same solution, prepared in the exact same way, in identical cuvettes, on the same day, and with the same spectrophotometer, there can be a significant dispersion in the so-obtained results. For instance, below are raw spectra of diluted haemolysed blood equilibrated with CO$_2$, as described in [2] (personal measurements).
As you can see, there is a significant dispersion of the results, even if care was taken to reduce the external influences to a minimum. This kind of deviations explains why—even for a molecule as simple and ubiquitous as water—considerable amount of variation may exist in the literature between reported absorbance values coming from different papers. See below an example with Segelstein[3] and Kou[4] data.
"Is there a definite molar absorptivity for everything, or is it all experimental?"
The answer is thus mitigated: in theory, every molecule has a given absorbance spectrum in a fixed set of settings (solvent, temperature, EM-field, pressure, etc.). However, since it is often exceedingly hard to reproduce these settings or compensate for measurement artefacts (id est influence of the cuvettes, spectrophotometer, etc., even with blanking), this knowledge is in practice often experimental.
Of note, there exists some databases of absorbance and fluorescence spectra, which—alas—are often very patchy. You mentioned NIST database, below are a few more that I compiled (sorry about it being a picture and the missing references).
What does this mean in practice?
It means that even by comparing different papers measuring the same substance, the molar extinction coefficients that you can find may vary from one source to the other. This in turn makes quantitative measurements based on these coefficients unfeasible or inaccurate in practice.
What can you do?
Common practice to achieve what you intend to do is to prepare a standard solution, id est a solution of known concentration, for instance by diluting a known mass of CuSO$_4$ into a known volume of water. You can then compare the absorbance of this standard solution at a given wavelength (usually chosen at point of null derivative such as a maximum, so that a slight error in wavelength does not result in a huge absorbance change) with that of the solution which you want to characterise. Beer-Lambert law may be useful here...
References
[1] Synthesis of New 2-(2´-Hydroxyaryl)benzotriazoles and Evaluation of Their Photochemical Behavior as Potential UV-Filters, Farkas et al. (2010), DOI: 10.3390/molecules15096205
[2] Measuring hemoglobin spectra: searching for carbamino-hemoglobin, Dervieux et al. (2020), DOI: 10.1117/1.JBO.25.10.105001
[3] The complex refractive index of water, Segelstein (1981), URL: http://hdl.handle.net/10355/11599
[4] Refractive indices of water and ice in the 0.65- to 2.5-µm spectral range, Kou et al. (1993), DOI: 10.1364/ao.32.003531
