Questions tagged [loss-function]
A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.
544
questions
0
votes
1
answer
18
views
Gradient output through custom loss function
I’m very new to Pytorch (and ML in general), so I’m having difficulty understanding what is going on WRT a custom loss/cost function I’m looking at. I understand what’s going on in the function, but I ...
0
votes
0
answers
13
views
why YOLO models multiple the loss by batch size in detection head?
here
return loss.sum() * batch_size, loss.detach() # loss(box, cls, dfl)
This line is from yolov8, but I saw similar thing in v5 too.
So far I only see this kind ...
- 101
2
votes
0
answers
22
views
Stuck with constant loss and network not learning
I am trying to predict certain function coefficients (output: a, b) based on its curve (input: frequency_response) with the help of https://github.com/Blealtan/efficient-kan (Kolmogorov-Arnold Network)...
- 151
0
votes
0
answers
9
views
How to handle sequences with crossEntropyLoss
fist of all i am ne wto the whole thing, so sorry if this is superdumb.
I'm currently training a Transformer model for a sequence classification task using CrossEntropyLoss. My input tensor has the ...
- 101
1
vote
1
answer
29
views
Is it appropriate to use KL Divergence as a loss function for a 1x3 regression model?
I have a regression model with a 1x3 output, which means it predicts three continuous values. I'm wondering if it would be appropriate to use the Kullback-Leibler (KL) Divergence as the loss function ...
- 175
2
votes
2
answers
58
views
Is there a max absolute error loss function?
I'm fitting a small model and need to use a custom loss function. I want to avoid large errors. I thought of a max absolute error function
$$ \text{MaxAE} = \max(|y-y_{pred}|) \, ,$$
with y the ...
- 123
2
votes
0
answers
45
views
Transformer model conditional probability distribution of sub-sentences
I have a simple transformer model (decoder only) which is trained on some dataset containing sentences to do next-word prediction. The model captures a probability distribution $P_{\theta}(\mathbf{a})$...
- 21
6
votes
2
answers
588
views
How do regression loss functions like MAE and MSE work although they remove the plus/minus sign?
I have a question about regression loss functions like Mean Absolute Error (MAE) and Mean Squared Error (MSE) used in deep learning.
When we calculate these losses, we remove the plus/minus sign from ...
- 175
1
vote
1
answer
29
views
Can you use the Euclidean Distance as a loss function?
While building an auto-encoder that preserves distances, i accidentally used the euclidean norm as the loss for the difference between the x and z distances that im trying to minimize. (I hope you can ...
- 11
0
votes
0
answers
21
views
Wrote a PINN to try to simulate 1-D heat transfer through a medium with constant diffusivity. The model doesn't even learn the boundary conditions!
I divide my training into 2 parts. a) Training on the boundary conditions within the domain. b) Training on the pde_loss.
The problem goes something like this: Both the ends of a 1 dimensional ...
0
votes
0
answers
14
views
Why does the TensorFlow docs use a different GAN generator loss?
As per the original paper that introduced GANs, the generator loss is given as:
$$
L_{G} = L _{BCE}(\mathbf{\vec 0}, \mathbf{D}(\mathbf{G}(\mathbf{\vec z}))) = \log(1 - \mathbf{D}(\mathbf{G}(\mathbf{\...
1
vote
2
answers
84
views
Mean Absolute Error from Scratch in NumPy
I recently tried implementing MAE from scratch in NumPy. The loss value and the slope seem to be equivalent to what Scikit-learn outputs, but for some reason the intercept value seems to converge to ...
- 11
0
votes
0
answers
9
views
How to implement CP tensor completion with extra calculations?
I am new to tensor decomposition. I want to know from a practical point of view, how to use an already known tool/library to compute CP factorization for tensor completion. Specifically, I want to ...
- 1
0
votes
0
answers
21
views
Autoencoders failing to recreate MNIST numbers
I have been having trouble trying to get a working (non-variational) autoencoder to reproduce images from the MNIST dataset. The two biggest issues is an averaging of the samples to yield a single ...
- 1
1
vote
1
answer
111
views
How to choose a loss function and how to calculate the loss for Text Generation in Generative AI?
For the classification problems, what loss functions can I choose ?
For the translation problem how do I decide whether the translation is good and how to choose a loss function?
And what about the ...
- 133