All Questions
Tagged with bias neural-networks
32
questions
1
vote
0
answers
17
views
Why is the threshold term incorporated into the weight vector in linear classifiers?
In the context of linear classifiers, such as the perceptron or logistic regression, I understand that the decision boundary is defined by a linear combination of input features and weights, plus a ...
2
votes
1
answer
23
views
Using threshold and bias at the same time in NN
I'm using NN with sigmoid binary activation. And for threshold I using 0,5. So if output < 0,5, it classified as 0. And if output >= 0,5 it classified as 1. But I'm using bias too at the same ...
- 21
2
votes
1
answer
375
views
Derivative error with respect to bias in binary cross entropy
I will do research using NN with 1 hidden layer. To calculate loss using binary cross entropy and for the activation function using sigmoid. I found the derivative formula from Sadowski, 2016 (link: ...
- 47
0
votes
0
answers
100
views
Training on biased dataset, when the bias is quantitively known
I have a machine learning model (A neural network here) which minimizes MSE loss. The model should fallow an unbiased distribution. Nevertheless, the training set is biased, but fortunately by a known ...
- 567
1
vote
1
answer
181
views
How does SGD training error decrease in subsequent epochs with non-iid samples when it is recommended that samples in subsequent epochs be iid?
I have been reading the Deep Learning book by Ian Goodfellow and on pg. 277, they mention:
It is also crucial that the minibatches be selected randomly.
Computing an unbiased estimate of the expected ...
- 131
5
votes
3
answers
175
views
Confusion about the training procedure while using transfer learning
Suppose that we have a trained CNN, there is 5 conv layers and 3 fully connected layers. We take the first 5 conv layers as it is (with their parameter settings: like kernel size, activation etc) and ...
- 223
3
votes
1
answer
177
views
Why is the bias neuron in neural network always initialised to 1?
I'm just starting with neural networks wherein this towards data science article mentions that bias neuron is always initialized to 1. My question is why is the bias neuron in Neural networks is ...
- 181
0
votes
0
answers
25
views
How to explain huge bias on unseen data?
I've trained a CNN to do a binary classification based on 2D radar spectra. I've tried different dataset sizes (reaching 200.000 samples per class) and always make sure that the classes are ...
2
votes
2
answers
3k
views
Do Neural Networks suffer from high bias or high variance
For most ML models we say they suffer from high bias or high variance, then we correct for it. However, in DL do neural networks suffer from the same concept in the sense that they initially have high ...
- 633
11
votes
3
answers
7k
views
Batch normalization and the need for bias in neural networks
I've read that batch normalization eliminates the need for a bias vector in neural networks, since it introduces a shift parameter that functions similarly as a bias. As far as I'm aware though, a ...
- 213
1
vote
0
answers
315
views
can a model outperform on test data then on training data
I am training Deep Neural Networks on a classification problem. N while choosing the no of epochs, I get below graph :
So my question is that this case neither comes in high bias and nor in high ...
- 225
0
votes
0
answers
22
views
Statistical proof to exclude less frequent records from data during analysis
I am working on reviewing the results of an automated task. For ex, To give you an idea, the data that I have to review looks like as shown below
Let's say from the downstream analytics perspective, ...
- 3,302
1
vote
0
answers
34
views
What does the famous bias-variance figure actually represent?
Below figure is generally used to explain bias-variance tradeoff.
But something which is not clear and not explained anywhere is: What does the dots represent ?
Do they represent:
1. predictions on ...
- 1,815
0
votes
0
answers
787
views
How to predict new data in Matlab neural network regression when output vs. target is not diagonal
In the ideal case, we expect the output vs. target plot to be diagonal. In Matlab, using the neural network regression app, the plot comes with the non-diagonal best fit (i.e., output=m x target+...
2
votes
2
answers
332
views
Network learns bias during the first iterations if parameter initialization is not good
Andrej Karpathy in his blog post "A Recipe for Training Neural Networks" states that initialization is important for convergence. I get that but when he says:
init well. Initialize the final layer ...
- 121