Custom Loss function Keras Tensorflow

Question

I need a custom weighted MSE loss function. I defined it in keras.backend

from keras import backend as K
def weighted_loss(y_true, y_pred):
    return K.mean( K.square(y_pred - y_true) *
    K.exp(-K.log(1.7) * (K.log(1. + K.exp((y_true - 3)/5 ))))      
    ,axis=-1  )

However, a test run returns

    weighted_loss(1,2)
ValueError: Tensor conversion requested dtype int32 for Tensor with dtype float32: 'Tensor("Exp_37:0", shape=(), dtype=float32)'  

or

    weighted_loss(1.,2.)
ZeroDivisionError: integer division or modulo by zero   

I wonder what mistakes am I making here.

Answer 1

Whether you are using Tensorflow or Theano is irrelevant for your question. Google the meaning of 'tensor' if the term confuses you.

Take a look at how Keras own loss function tests have been implemented here:

def test_metrics():
    y_a = K.variable(np.random.random((6, 7)))
    y_b = K.variable(np.random.random((6, 7)))
    for metric in all_metrics:
        output = metric(y_a, y_b)
        print(metric.__name__)
        assert K.eval(output).shape == (6,)

You can't simply feed a float or int into tensor calculations. Note also the use of K.eval to obtain the result you're looking for.

So try something similar with your function:

from keras import backend as K
import numpy as np

y_a = K.variable(np.random.random((6, 7)))
y_b = K.variable(np.random.random((6, 7)))
output = weighted_loss(y_a,y_b)
result = K.eval(output)

There is also no need to define your custom function in keras.backend - what if you decide to update Keras later on?

Instead you could do the following in your own code: define a function that returns your loss function

def weighted_loss(y_true, y_pred):
        return K.mean( K.square(y_pred - y_true) * K.exp(-K.log(1.7) * (K.log(1. + K.exp((y_true - 3)/5 )))),axis=-1  )

Then when you want to compile your model with your loss function, you can do:

model.compile(loss = weighted_loss)

In case you want to define a more general loss function, where the weighting depends on some input, you'll need to wrap the function. So for example:

def get_weighted_loss(my_input):
    def weighted_loss(y_true, y_pred):
        return K.mean( K.square(y_pred - y_true) * K.exp(-K.log(1.7) * (K.log(1. + K.exp((y_true - 3)/my_input )))),axis=-1  )
    return weighted_loss

Then when you want to compile your model with your loss function, you can do:

model.compile(loss = get_weighted_loss(5))