All Questions
Tagged with neural-network backpropagation
168
questions
0
votes
1
answer
37
views
My custom neural network is converging but keras model not
in most cases it is probably the other way round but...
I have implemented a basic MLP neural network structure with backpropagation. My data is just a shifted quadratic function with 100 samples. I ...
1
vote
1
answer
49
views
CS 224N Back Propagation and Margin Loss in Neural Networks
I was going through Stanford CS 224 lecture notes on Back propagation.
Page 5 states:
We can see from the max-margin loss that:
∂J /∂s = − ∂J/∂s(c) = −1
I'm not sure I understand why this is the ...
- 113
0
votes
0
answers
143
views
Runtime Error: one of the variables needed for gradient computation has been modified by an inplace operation:
I have the following code for a reinforcement learning using proximal policy optimization. It gives the following run time error.
...
- 37
1
vote
0
answers
21
views
Are "textbook backpropagation" still relevant?
The above backpropagation algorithm is taken from Shalev Shwartz and Ben-David's textbook: Understanding Machine Learning. This algorithm is described in the same way as the one in Mostafa's textbook, ...
- 119
0
votes
1
answer
61
views
This simple python Feed forward Neural Network isn't learning. What am I doing wrong?
The backpropagation procedure is taken from the approach outlined in here.
Here is the code, commented:
...
- 103
0
votes
1
answer
188
views
How is the backward propagation is done in pytroch? When to use torch.no_grad, also when and where is the gradinte calcuated?
I have this training loop in pytorch. the loss_fn = nn.CrossEntropyLoss() and optim = torch.optim.Adam(net.parameters(), lr=lr)
<...
2
votes
1
answer
407
views
Gradients of lower layers of NN when gradient of an upper layer is 0?
Say we have a neural network with an input layer, a hidden layer and an output layer.
Say the gradients with respect to the weights and biases of the output layer are all 0.
Then, by backpropagation ...
- 147
0
votes
0
answers
47
views
Backpropagation and Gradient Descent: Questions on math behind it
I watched this video which goes over backpropagation calculus and read the Wikipedia page on it. This is my understanding of the equations for the algorithm. I have questions regarding the equations ...
- 101
0
votes
1
answer
44
views
Neural Nets: Difference between activation and activation function, error on Wikipedia?
I'm reading the Wikipedia page on backpropagation and have some questions about the following equations:
$$
\frac{d C}{d a^L}\cdot \frac{d a^L}{d z^L} \cdot \frac{d z^L}{d a^{L-1}} \cdot \frac{d a^{L-...
- 101
2
votes
2
answers
6k
views
What exactly is Gradient norm?
I found that there is no common resource and well defined definition for "Gradient norm", most search results are based on ML experts providing answers which involves gradient norm or papers ...
- 23
0
votes
1
answer
117
views
GAN Generator Backpropagation Gradient Shape Doesn't Match
In the TensorFlow example (https://www.tensorflow.org/tutorials/generative/dcgan#the_discriminator) the discriminator has a single output neuron (assume batch_size=1).
Then over in the training loop ...
- 3
0
votes
1
answer
189
views
Can a Simple Neural Network Predict a 0 or 1 Output by Looking Only at the Last Input?
I wrote a simple neural network that functions similarly to many of the C# examples I've seen online. It uses weights and biases and can be trained using backpropagation. It works well for ...
- 101
0
votes
1
answer
145
views
How does Back Propagation in a Neural Net Work?
I understand that, in a Neural Net, Back Propagation is used to update the model's weights and biases to lower loss, but how does this process actually work?
- 661
0
votes
0
answers
94
views
Why backpropagation is done in every epoch when loss is always scalar?
I understand the backpropagation algorithm that it calculates the derivate of loss with respect to all the parameters in the neural network. My question is this derivate is constant right because the ...
- 101
0
votes
1
answer
88
views
Back propagation matrix shape error using Python
I wanna implement the back-propagation algorithm in python with the next code
...
- 3