This is more of a long comment than answer. First, the analogous statement for the first derivative is already non-trivial, although not very difficult, see Aull, Charles E. "The first symmetric derivative." The American Mathematical Monthly 74.6 (1967): 708-711.
Second, denote
$$
(L_th)(x)=\frac{h(x+t)+h(x-t)-2h(x)}{t^2};
$$
then if $\phi$ is a test function, we have
$$
\int_\mathbb{R}f\phi''=\int_\mathbb{R}f\lim_{t\to 0}L_t\phi\stackrel{(1)}{=}\lim_{t\to 0}\int_\mathbb{R}fL_t\phi=\lim_{t\to 0}\int_\mathbb{R}(L_tf)\phi\stackrel{(2)}{=}\int_\mathbb{R}\lim_{t\to 0}(L_tf)\phi=\int_\mathbb{R} g\phi.
$$
This would imply that $f''=g$ in the distributional sense. That is, $f$ coincides with a suitable second anti-derivative of $g$ in distributional sense. By applying Du Bois Reymond's lemma twice, we see that they actually coincide as functions. But since $g$ is continouos, its second anti-derivative is twice continuously differentiable.
Of the two exchanges of the limit and the integral, (1) is clear, since for smooth functions $L_t\phi\to \phi''$ uniformly, and (2) is a trouble - I don't see how to deduce it from the conditions at hand. But the argument shows that if we are willing to assume a bit more, e.g., that the convergence $L_t f\to g$ is uniform, then the result follows easily.
UPD: the actual argument is much simpler. Let $a<b$, and let $G$ be the second anti-derivative of $g$ such that $(f-G)(a)=(f-G)(b)=0.$ We need to show that $f\equiv G$ on $[a,b]$. Assume that $\max_{[a,b]}(f-G)>0$. Then, $$\max_{[a,b]}H_\epsilon>0$$ for $\epsilon>0$ small enough, where $H_\epsilon(x)=(f(x)-G(x)-\epsilon (b-x)(x-a))$. But $\lim_{t\to 0}L_tH_\epsilon \equiv 2\epsilon$ for any $t$ and any $\epsilon.$ So, for any $x,$ we have $H(x+t)+H(x-t)-2H(x)>0$ for $t$ small enough. Applying this to the point on $(a,b)$ where $H_\epsilon$ attains its maximum gives a contradiction.
