Working through Bishop's Pattern Recognition and Machine Learning(a great read so far!) and on page 67 he says:
"One limitation of the parametric approach is that it assumes a specific functional form for the distribution which may turn out to be inappropriate for a particular application"
Why might it be that the functional form of a distribution is inappropriate for a particular application? An illustrative example would also be much appreciated.
It's not functional forms themselves are inappropriate, but the assumption part. It's a rigid approach (the parameters are often allowed to vary, but it's quite restrictive compared to other approaches).
While we have functional forms for a lot of things, many other distributions are not even expressible by (our human) functions.
If you need to assume a distribution, what happens when it's misspecified or inconsistent?
So that's the fact that Bishop is alluding to. Non-parametric or semi-parametric approaches try to bypass that.
