These statements are equivalent:
- There exists some constant $C$ such that $p^2$ divides $f+C$
- There exist some constant $C$ and some polynomial $q$ such that $f+C=qp^2$
- There exists some polynomial $q$ such that $f'=q'p^2+2qp'p$
- There exists some polynomial $q$ such that $f'=(q'p+2qp')p$
- There exist some polynomials $q,r$ such that $r=q'p+2qp'$ and $f'=rp$
And (5) implies that $p$ divides $f'$. But the fact that $f'=rp$ for some polynomial $r$ implies that there exists some polynomial $q$ such that $r=q'p+2qp'$ (which would make the implication reversible) is not true. For example, if $f'(x)=p(x)=x^3$ we get that $1=x^3q'(x)+6x^2q(x)$, which implies that the constant term of $x^3q'(x)+6x^2q(x)$ is $1$, and this is impossible.
Let's try to see for which pairs of polynomials $p,r$ there exists a solution $q$. Let $m=\deg q$ and $n=\deg p$:
Let
$$p(x)=\sum_{j=0}^np_jx^j$$
$$r(x)=\sum_{k=0}^{m+n-1}r_kx^k$$
$$q(x)=\sum_{h=0}^{m}q_hx^h$$
Then
$$p'(x)=\sum_{j=0}^{n-1}(j+1)p_{j+1}x^j$$
$$q'(x)=\sum_{h=0}^{m-1}(h+1)q_{h+1}x^h$$
So,
$$qp'(x)=\sum_{k=0}^{m+n-1}\left(\sum_{\substack{0\le i\le k\\k-n \le i\le m-1}}(i+1)p_{k-i}q_{i+1}\right)x^k$$
$$2pq'(x)=\sum_{k=0}^{m+n-1}\left(\sum_{\substack{0\le i\le k\\k-n \le i\le m-1}}2(i+1)p_{i+1}q_{k-i}\right)x^k$$
Then, for each $k$, $0\le k\le n+m-1$,
$$r_k=\sum_{\substack{0\le i\le k\\k-n \le i\le m-1}}\left[(i+1)p_{k-i}q_{i+1}+2(i+1)p_{i+1}q_{k-i}\right]$$
$$=\sum_{\substack{0\le i\le m\\k-n+1\le i\le k+1}}\left[ip_{k-i+1}+2(k-i+1)p_{k-i+1}\right]q_i$$
$$=\sum_{\substack{0\le i\le m\\k-n+1\le i\le k+1}}(2k-i+2)p_{k-i+1}q_i$$
This is a linear system for the unknowns $q_i$. It has $m+n$ equations and $m+1$ unknowns, and each equation has at most $\min(m+1,n+1)$ terms. 'Probably' it has no solution. The smaller is $\deg p$, the more probable is that the system has a solution.
Nevertheless, this gives a finite algorithm to decide if there exists $q$ and compute it, but finding a simple, general criterion seems difficult, to say the least.