The electron of negative ion in p-type semiconductor is in valence band which means it constitutes covalent bond. So it can't move to other atom freely. It must go up to conduction band of p-type to become "free electron" first. And this takes energy which is not available "on average". So there is no current from p-type to n-type macroscopically.
![enter image description here](https://cdn.statically.io/img/i.sstatic.net/ob76k.png)
This is the situation. Reverse bias, p-type has no hole and negatively charged, and n-type has no "free electron" and positively charged.
The purple arrow (2) is the what you are asking. And this is not happening because the electron is in covalent bond. (it can actually happen in quantum effect, but the depletion area have to be thin enough and this is not the general situation what you are wondering about)
For electron to move, it have to take route of up arrow (1) first. Overcoming this band gap is like the current of intrinsic semiconductor(pure, no doping) which is not able in low temperature.
The electrical potential is even favorable to the current(p to n) but it's stuck at the first step.
In analogy, it's like a chemical reaction whose reactant is higher energy than product, but the activation energy is too high for the reaction to happen.
PS.
On the other hand, the hole can move, though it is in valence band.
For charge to move, it should have a room to move which means it's not octet. And the hole is the room. But the electron in your question have no more room in valence band and it have to go up to conduction band.
If you see periodic table, you will notice the metals are in middle, which means it's hard to make covalent bond to get octet, so there are free electron(conduction band).
(Non metal can easily have octet because they have to get or remove "few electrons")
Edit for comment by oliver
![enter image description here](https://cdn.statically.io/img/i.sstatic.net/7OW7V.jpg)
As you see in this graph, the conductivity of intrinsic semi-conductor is very low in room temperature. The x-axis start from 700K which is much higher than room temperature.