You have an incoming photon that intercepts an electron on the atom, the electron gets excited to some energy level for a few femtoseconds, and then goes back to the ground state, expelling another photon during the transition.
This is only one way that objects have color. It doesn't cover, for example, fluorescent materials.
In this scenario the emitted photon has very nearly the same frequency (and therefore color) as the original exciting photon. If the object is "red" then it means the object behaves this way when it intercepts red photons, but when it intercepts other colored photons it simply absorbs them and doesn't re-emit (the electron loses energy thermally and heats the material rather than producing a new photon).
So if we have a red plastic, and we illuminate it with white light, we do get red light reflected (or transmitted). But it means we "wasted" all the other energy in the white light that was being carried by green, yellow, blue, etc., photons. So this process will be fairly inefficient.
If we use an LED, we only produce (for example) red photons to begin with and don't waste energy producing yellow, green, and blue photons to be thrown away.
the precise physics that disables us from using plastic as an illumination source.
Physics doesn't prevent it. Back before LEDs were cheap, it was fairly common to produce red light by putting a red-dyed glass or plastic layer in front of a white lamp. It's even still done today for some applications. For example, you can still buy gels for stage lighting that have exactly this purpose.
Using LEDs is just cheaper and more efficient for many applications, given today's technology.
(There are other minor differences like it's hard to find dyes that are as narrow-band as LEDs, so the light from a filtered white lamp will typically have a wider spectrum than from an LED)
