Tensorflow regression model predicts the same value for every input

Question

Code reference: https://github.com/adamski234/tensorfailure. It contains the code, training data and sample inputs for predictions.

I have created a model with a few layers:

normalizer = tensorflow.keras.layers.Normalization(axis=-1)
normalizer.adapt(source_data)
model = tensorflow.keras.models.Sequential([
    normalizer,
    tensorflow.keras.layers.Dense(128, input_shape=(source_data.shape[1],), activation="sigmoid"),
    tensorflow.keras.layers.Dropout(0.1),
    tensorflow.keras.layers.Dense(64, activation="relu"),
    tensorflow.keras.layers.Dropout(0.1),
    tensorflow.keras.layers.Dense(32, activation="relu"),
    tensorflow.keras.layers.Dense(16, activation="relu"),
    tensorflow.keras.layers.Dense(8, activation="relu"),
    tensorflow.keras.layers.Dense(4, activation="relu"),
    tensorflow.keras.layers.Dense(2, activation="relu"),
    tensorflow.keras.layers.Dense(2)
])
model.compile(optimizer=tensorflow.keras.optimizers.Adam(learning_rate=0.1), loss="mape")
model.fit(source_data, source_data_reference, epochs=50)

And trained it on the data from the training_data directory in the linked repository (main.py). However, when trying to run that model using data located in the input_data directory, the model returns the same pair of numbers for every single set of input values: [1128.1345 618.5919].

What I have tried:
Removing layers, changing the learning rate (0.0001 to 10), changing epoch count (3 to 1000), changing the loss algorithm (mae, mape, mse)

What happened: Every change resulted in a different pair of numbers being output. Extremely low epoch counts resulted in the output being varied, but the predictions were far outside what could be considered acceptable. Some combinations of parameters caused the predicted values to be very close to the mean of expected values.

What I expected to happen: That the model would start outputting data that is close to the data found in columns named reference__x and reference__y.

Answer 1

You have used sigmoid as the input layer it changes the value to 0.0-1.0 in the start of the layer.

Try the below model architecture,

model = tensorflow.keras.models.Sequential([
    tensorflow.keras.layers.Dense(128, input_shape=(source_data.shape[1],), activation="relu"),
    tensorflow.keras.layers.BatchNormalization()
    tensorflow.keras.layers.Dense(64, activation="relu"),
    tensorflow.keras.layers.Dense(32, activation="relu"),
    tensorflow.keras.layers.Dense(16, activation="relu"),
    tensorflow.keras.layers.Dense(2)
])

model.compile(optimizer=tensorflow.keras.optimizers.Adam(learning_rate=0.001), loss="mae")

Train the model for 20 Epoch. Make sure you have the validation dataset to validate the model during the training process