Bias can be good

An $\color{orangered}{\text{unbiased estimator}}$ is on average correct. A $\color{steelblue}{\text{biased estimator}}$ is on average not correct.

Why then, would you ever want to use a biased estimator (e.g. ridge regression)?

The answer is that introducing bias can reduce variance.

In the picture, for a given sample, the $\color{orangered}{\text{unbiased estimator}}$, has a $68\%$ chance to be within $1$ arbitrary unit of the true parameter, while the $\color{steelblue}{\text{biased estimator}}$ has a much larger $84\%$ chance.

If the bias you have introduced reduces the variance of the estimator sufficiently, your one sample has a better chance of yielding an estimate close to the population parameter.

"On average correct" sounds great, but does not give any guarantees of how far individual estimates can deviate from the population parameter. If you would draw many samples, the $\color{steelblue}{\text{biased estimator}}$ would on average be wrong by $0.5$ arbitrary units. However, we rarely have many samples from the same population to observe this 'average estimate', so we would rather have a good chance of being close to the true parameter.