I am wondering about a non-GSM transmitter leaking into the GSM1800 and whether these limits can be derived from the above document in the first place.
They can't. This is subject to local laws, not communication standards – and usually formulated and enforced by the regulatory body of a country (FCC in the US, OFCOM in UK, BNetzA in Germany, MIC in Japan, …) with laws governing the guidelines for that regulation.
As a general "rule of why things are legislated": These laws exist to protect services (in your case, cellular telephony and data transfer) from interference from other services and unintentional radiators.
Thus, it's realistic to assume that local laws are written so that the interference levels do not mitigate normal operation even in the low signal strength regimes of the protected services.
GSM handsets need to work down to -120 dBm per standard. That kind of implies that you might get into trouble if your out-of-band radiation is significantly more that; the definition of "significantly" will heavily vary between countries.
In other words, if you can measure it when using the smallest resolution bandwidth of your spectrum analyzer, chaaaaaances are it's too much. Again, for more information, you'll need to refer to your local regulator.