Server GPUs are not being detected by Blender Cycles CUDA?

Question

I have a shared server with 8 RTX A5000s and am trying to enable GPUs allocated via CUDA_VISIBLE_DEVICES=... when running blender. The code I have so far is:

def enable_gpus():
    # Enable CUDA
    bpy.context.preferences.addons['cycles'].preferences.compute_device_type = "CUDA"
    # Enable GPUs
    for device in bpy.context.preferences.addons['cycles'].preferences.devices:
        print(device['name'])

which should just list out the available GPUs that blender is detecting, however this comes back blank with no print statements because there are supposedly no devices blender is finding to iterate through. Any ideas on how I can get this to work? I've tried making sure that cycles is enabled, and have tried different configurations of GPUs to make visible, and am not quite sure where to go from here to troubleshoot. On my own PC without CUDA_VISIBLE_DEVICES this code is correctly finding my GPUs.

Here is the server output from nvidia-smi:

Answer 1

Your code is not executing. I know this because it contains errors. For example, the last line should show this error:

NameError: the name 'd' is undefined.

Because you are using device in the for statement, not d.

So you should see this error, if it is not true, then your code is not executing.

And the correct ask for devices will be like this:

bpy.context.preferences.addons['cycles'].preferences.get_devices_for_type('CUDA')