How to release the occupied GPU memory when calling keras model by Apache mod_wsgi django?

Question

My server configuration is as follows:

1. apache 2.4.23.
2. Mod_wsgi 4.5.9

By using the Django framework and apache server, we call the Keras deep learning model. And after the successful calling of the model, the model has been always running in the GPU memory, which causes the GPU memory can not be released except by shutting down the apache server.

So, is there any way to control the release of GPU memory when calling a Keras model by Apache+Mod_wsgi+Django?

Thanks!

Runtime memory footprint screenshots

Answer 1

For people who fail to make K.clear_session() work, there is an alternative solution:

from numba import cuda
cuda.select_device(0)
cuda.close()

Tensorflow is just allocating memory to the GPU, while CUDA is responsible for managing the GPU memory.

If CUDA somehow refuses to release the GPU memory after you have cleared all the graph with K.clear_session(), then you can use the cuda library to have a direct control on CUDA to clear GPU memory.

Answer 2

from keras import backend as K
K.clear_session()

This will clear the current session (Graph) and so the stale model should be removed from GPU. If it didn't work, you might need to 'del model' and reload it again.

Answer 3

from numba import cuda
device = cuda.get_current_device()
device.reset()