There is no function $f$ with $f(x+y) = f(x)-f(y)+xy(x+y)$ for all $x,y \in \mathbb{R}$, since then with $x = 0$ it holds $f(y) = f(0)-f(y)$ for all $y \in \mathbb{R}$, so $f =\frac{f(0)}{2}$ constant. But then $\frac{f(0)}{2} = xy(x+y)$ for all $x,y \in \mathbb{R}$. A contradiction.
Suppose that $f$ instead satisfies $f(x+y) = f(x)+f(y)+xy(x+y)$ for all $x,y \in \mathbb{R}$. Then it holds with $x= 0= y$: $f(0) = f(0) + f(0)$, thus $f(0) = 0$, and with $y = -x$: $0=f(0) = f(x)+f(-x)$, thus $f(-x) = -f(x)$ for all $x \in \mathbb{R}$. Suppose $f$ is at least $C^1$. Then take the partial derivative $\frac{\partial}{\partial x}$ on both sides and get $f'(x+y) = f'(x)+2xy+y^2$ for all $x,y\in \mathbb{R}$. With $x = 0$ it follows
$$f'(y) = f'(0)+y^2$$
for all $y\in \mathbb{R}$ and therefore
$$f(y) = f'(0)y + \frac{1}{3}y^3 +c$$
for some $c \in \mathbb{R}$ and all $y \in \mathbb{R}$. It is $f(0) = 0$, so
$$f(x) = f'(0)x + \frac{1}{3}x^3.$$
Now, define $f(x):=ax+\frac{1}{3}x^3$ for $a, x \in \mathbb{R}$. Then $$f(x+y) = a(x+y) + \frac{1}{3}(x+y)^3 = (ax + \frac{1}{3}x^3) +(ay + \frac{1}{3}y^3) + \frac{1}{3}(3x^2y+3y^2x) \\ = f(x)+f(y)+xy(x+y)$$
for all $x,y \in \mathbb{R}$, so satisfies the functional equation.
Thus all $C^1$ functions satisfying the functional equation for all $x,y \in \mathbb{R}$ are given by the family $f_a(x) = ax+\frac{1}{3}x^3$ for $a, x \in \mathbb{R}$.
We further know for an $f$ satisfying the functional equation that $f$ is $C^1$ if and only if the limit $\lim_{h \to 0} \frac{f(h)}{h}$ exists, i.e. $f$ is differentiable in $0$ (since $f(0) = 0$). This is because $f(0) = 0$ and from the functional equation it follows
$$\frac{f(x+h)-f(x)}{h} = \frac{f(h)}{h}+x(x+h),$$
i.e. in this case
$$f'(x) = a +x^2$$ for $x \in \mathbb{R}$ with $a = \lim_{h \to 0}\frac{f(h)}{h}$ (in particular, $f'$ is continuous).
Therefore, all functions $f:\mathbb{R} \to \mathbb{R}$ with existing limit $\lim_{x \to 0} \frac{f(x)}{x}$ satisfying the functional equation are given by $f_a(x) = ax+\frac{1}{3}x^3$ for $x \in \mathbb{R}$ with $\lim_{x \to 0} \frac{f(x)}{x} = a \in \mathbb{R}$.
Now assume $\lim_{x \to 0} \frac{f_a(x)}{x} = 1$. Then $a = f_a'(0) = \lim_{x \to 0} \frac{f_a(x)}{x} = 1$. Therefore
$$f(x) = f_1(x) = x+\frac{1}{3}x^3.$$
It then follows:
$$\sum_{i=11}^{17} f'(i) = \sum_{i=11}^{17} (1+i^2) = (17-10)+\frac{1}{6}(17(17+1)(2\cdot 17 +1) - 10(10+1)(2\cdot 10 +1)) = 7 +\frac{1}{6}(17\cdot 18 \cdot 35 - 10\cdot 11 \cdot 21) = 7 + 17 \cdot 3 \cdot 35 - 5 \cdot 11 \cdot 7 = 7+1785-385 = 7+1400 = 1407.$$