The capacity of a channel of bandwidth $W$ is known since the 1940‘s to be given by the Shannon limit:
$$C=W \log (1+\mathit{SNR}), $$
where $\mathit{SNR}$ is the signal to noise ratio. In an optical fibre in the linear regime, the noise is essentially independent of the signal, and that means one just has to increase the signal power to increase the rate. Mathematically, an infinite rate seems possible, but the logarithmic dependence makes the increase really expensive. That is the reason why, historically, the capacity increase of fibers has been more focused on increasing the bandwidth $W$,through WDM. However this $W$ increase has some limits, and people also work at increasing the power.
The Shannon formula ensures that, to have a linear increase in capacity, one needs an exponential increase in $\mathit{SNR}$, and therefore, an exponential increas of optical of power. Which means that, at some point, nonlinear effects are noticable, and create what is sometimes named “the nonlinear Shannon limit”. The main effect is that the channel is non longer an additive channel: the signal as the output of the channel cannot be analyzed as the addition of the input signal and an independent noise, and the above formula is no longer valid. Effectively, some part of the signal acts as noise to other parts of the signal, creating an effective $SNR$ which decreases with increasing signal power past a power threshold, and therefore a decreasing capacity above a given optimal input power (at -$5\text{ dBM}\simeq300\text{ µW}$ on your graph.
As far as I understood (this is not my research topic), the concept of nonlinear Shannon limit is an active research topic: it has no clean formulation, and some researcher claim that, contrasting to the Shannon limit, this is not a universal limit but an artifact of current encoding techniques.