How to get same accuracy with identical models in Keras and Tensorflow?

Question

As we all know Keras backend uses Tensorflow and so it should give out some kind of results when we provide the same parameters, hyper-parameters, weights, and biases initialization at each layer, but still, the accuracy is different.

This may be because the batches of images that are fed at each step in both the models are not identical and get shuffled randomly.

Is there any way in which we can make sure that the same batch of images is fed into the model while eliminating the randomness?

I have tried using all the same parameters, hyperparameters, same weights, and biases initialization with seed values.

The accuracy of both the models is not the same.

Answer 1

I created a rule to achieve reproducibility:

• Works for python 3.6, not 3.7
• First install Keras 2.2.4
• After install tensorflow 1.9

And finally in the code:

import numpy as np
import random as rn
import tensorflow as tf
import keras
from keras import backend as K

#-----------------------------Keras reproducible------------------#
SEED = 1234

tf.set_random_seed(SEED)
os.environ['PYTHONHASHSEED'] = str(SEED)
np.random.seed(SEED)
rn.seed(SEED)

session_conf = tf.ConfigProto(
    intra_op_parallelism_threads=1, 
    inter_op_parallelism_threads=1
)
sess = tf.Session(
    graph=tf.get_default_graph(), 
    config=session_conf
)
K.set_session(sess)
#-----------------------------------------------------------------#

Answer 2

The issue is not with the framework you are using, but with the random number generator running in the background. If you are splitting your validation set in runtime, then make sure that you have a random seed set to some specific number. In order to eliminate the randomness of the random number generator, we should set seed.

You can set your random seed for TensorFlow using tf.random.set_seed(seed) variable and give it some value. The same could be achieved in pytorch using torch.manual_seed(0). But when you do this, other dependencies such as numpy and random also generates their own random number, so you need to set their seed as well. You can also use the function below to set all your seed at once. Just keep the consistency check when you run anything.

def set_seeds(seed=SEED):
    os.environ['PYTHONHASHSEED'] = str(seed)
    random.seed(seed)
    tf.random.set_seed(seed)
    np.random.seed(seed)