Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Torch Version UNetEncoderBlock causes multi-card training error #40

Open
Yeeesir opened this issue Jun 6, 2023 · 0 comments
Open

Torch Version UNetEncoderBlock causes multi-card training error #40

Yeeesir opened this issue Jun 6, 2023 · 0 comments

Comments

@Yeeesir
Copy link

Yeeesir commented Jun 6, 2023

When I use maxim_pytorch provided by link, and try multi-gpu training, the following error occurred

  File "/home/miniconda3/lib/python3.8/site-packages/torch/autograd/__init__.py", line 154, in backward
    Variable._execution_engine.run_backward(
RuntimeError: Function BroadcastBackward returned an invalid gradient at index 76 - got [0] but expected shape compatible with [0, 32, 2, 2]

I found that the key problem lies in the torch implementation of UNetEncoderBlock, other network structures did not introduce errors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
1 participant
@Yeeesir